You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The changes in #5122 improved the performance of seeding fate split operations. However there is still further room for improvement. This code will seed a single split operation which will write a single conditional mutation. Writing single conditional mutations at a time requires waiting on them to committed in the write ahead log. Could get much better performance by writing many conditional mutaitons at once as this would allow many to be written to the write ahead log and then wait for all of them to be committed.
Describe the solution you'd like
The code for seeding split fate operations is restructured so that is takes a batch of tablets that need split operations seeded and writes the entire batch to a conditional writer. This would avoid the single writes. The current code avoids two problems and we would want to continue avoiding these when restructuring the code.
The first problem the current code avoids is attempting to seed the same tablet for split when its currently in the processes of seeding. This can happen when there is a situation like 100 tablets that need to split and tablet group watcher keeps finding these and queueing them up for seeding over and over. This will not cause problems with correctness, it just waste resources.
The second problem the current code avoids is making the tablet group watcher wait on seeding split operations. If there is a problem writing to the fate table to seed split, this will not block the tablet group watcher. Instead it eventually starts dropping/ignoring request to seed split operations.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
The changes in #5122 improved the performance of seeding fate split operations. However there is still further room for improvement. This code will seed a single split operation which will write a single conditional mutation. Writing single conditional mutations at a time requires waiting on them to committed in the write ahead log. Could get much better performance by writing many conditional mutaitons at once as this would allow many to be written to the write ahead log and then wait for all of them to be committed.
Describe the solution you'd like
The code for seeding split fate operations is restructured so that is takes a batch of tablets that need split operations seeded and writes the entire batch to a conditional writer. This would avoid the single writes. The current code avoids two problems and we would want to continue avoiding these when restructuring the code.
The first problem the current code avoids is attempting to seed the same tablet for split when its currently in the processes of seeding. This can happen when there is a situation like 100 tablets that need to split and tablet group watcher keeps finding these and queueing them up for seeding over and over. This will not cause problems with correctness, it just waste resources.
The second problem the current code avoids is making the tablet group watcher wait on seeding split operations. If there is a problem writing to the fate table to seed split, this will not block the tablet group watcher. Instead it eventually starts dropping/ignoring request to seed split operations.
The text was updated successfully, but these errors were encountered: