Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry pick Choose log router tag prior to resolution in commit path for version vector. (#11628) #11635

Draft
wants to merge 3 commits into
base: release-7.3
Choose a base branch
from
Draft

Cherry pick Choose log router tag prior to resolution in commit path for version vector. (#11628) #11635

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b2b5fa3
Choose log router tag prior to resolution in commit path for version …
dlambrig Sep 6, 2024
d47caa9
Merge branch 'release-7.3' into r73
dlambrig Sep 9, 2024
440ed45
Merge branch 'release-7.3' into r73
dlambrig Sep 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions fdbserver/CommitProxyServer.actor.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -744,6 +744,9 @@ std::set<Tag> CommitBatchContext::getWrittenTagsPreResolution() {
}
}

toCommit.storeRandomRouterTag();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be called only log routers are present?

Copy link
Contributor Author

@dlambrig dlambrig Sep 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you have a simulation failure where you found this that I can use to test?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test: "tests/fast/Unreadable.toml -b on -s 598569344"

Or you can run a joshua job with version vector enabled and you would see failures caused by this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the above test failure is on "main", I didn't run any tests over this change on this branch.

transactionTags.insert(toCommit.savedRandomRouterTag.get());

return transactionTags;
}

Expand Down
2 changes: 1 addition & 1 deletion fdbserver/LogSystem.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ void LogPushData::writeMessage(StringRef rawMessageWithoutLength, bool usePrevio
if (!usePreviousLocations) {
prev_tags.clear();
if (logSystem->hasRemoteLogs()) {
prev_tags.push_back(logSystem->getRandomRouterTag());
prev_tags.push_back(chooseRouterTag());
}
for (auto& tag : next_message_tags) {
prev_tags.push_back(tag);
Expand Down
9 changes: 8 additions & 1 deletion fdbserver/include/fdbserver/LogSystem.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -813,6 +813,9 @@ struct LogPushData : NonCopyable {
// getAllMessages() and is used before writing any other mutations.
void setMutations(uint32_t totalMutations, VectorRef<StringRef> mutations);

Optional<Tag> savedRandomRouterTag;
void storeRandomRouterTag() { savedRandomRouterTag = logSystem->getRandomRouterTag(); }

private:
Reference<ILogSystem> logSystem;
std::vector<Tag> next_message_tags;
Expand All @@ -834,13 +837,17 @@ struct LogPushData : NonCopyable {
// true on a successful write, and false if the location has already been
// written.
bool writeTransactionInfo(int location, uint32_t subseq);

Tag chooseRouterTag() {
return savedRandomRouterTag.present() ? savedRandomRouterTag.get() : logSystem->getRandomRouterTag();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am noticing that this is resulting in an assertion failure in "TagPartitionedLogSystem::getRandomRouterTag()" -> "deterministicRandom()->randomInt()" when "logRouterTags = 0".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this appears to be because a pseudo tag is set.

}
};

template <class T>
void LogPushData::writeTypedMessage(T const& item, bool metadataMessage, bool allLocations) {
prev_tags.clear();
if (logSystem->hasRemoteLogs()) {
prev_tags.push_back(logSystem->getRandomRouterTag());
prev_tags.push_back(chooseRouterTag());
}
for (auto& tag : next_message_tags) {
prev_tags.push_back(tag);
Expand Down