Improve performance of FateLock code for the case when there are lots of locks. #5181
Labels
enhancement
This issue describes a new feature, improvement, or optimization.
Milestone
Is your feature request related to a problem? Please describe.
Attempted to intiate many compaction for many small ranges of a table. As more compactions were started the fate operations for each took longer and longer to get a FateLock. Looking at jstacks the code was here and called this and would spend time there. Each fate operation that was added would make the getData calls for all others that had been added before.
Describe the solution you'd like
Attempt to avoid the calls to getData by encoding the information in the node name. The information stored in the data for the node is a fate uuid and whether its a read or write lock. This information could be encoded in the node name. Then the single RPC to zookeeper to getChildren would get call needed information. This would avoid making the RPC for each child to getData.
#5180 will improve the situation somewhat, but there could still be a lot of RPCs.
Additional context
Not sure if this would be suitable for an IT, but would may be good to try. Could try to make an IT for this creates a table w/ 1K tablets and then spins up 1K compactions via API one per tablet. Can try running this before and after the change and looking at the CompactRange.isReady times. Could try increasing 1K also and looking at the time differences as that increases.
The text was updated successfully, but these errors were encountered: