Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support splitting tablet with lots of files #5100

Open
keith-turner opened this issue Nov 22, 2024 · 0 comments
Open

Support splitting tablet with lots of files #5100

keith-turner opened this issue Nov 22, 2024 · 0 comments
Labels
enhancement This issue describes a new feature, improvement, or optimization.
Milestone

Comments

@keith-turner
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Currently in the main branch for Accumulo 4.0 there is no support for splitting tablets with lots of files. This exist in earlier versions of Accumulo but was not carried forward. Need to readd support for this.

Describe the solution you'd like

In earlier version of accumulo when a tablet had lots of files the split code would merge the rfile indexes in batches and then iterated over this merged view. For example if a tablet had 1000 files it might read 200 files at time merging their indexes into a new file, then it would read over the intermediate merged index files to compute the split point.

@keith-turner keith-turner added the enhancement This issue describes a new feature, improvement, or optimization. label Nov 22, 2024
@keith-turner keith-turner added this to the 4.0.0 milestone Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement This issue describes a new feature, improvement, or optimization.
Projects
None yet
Development

No branches or pull requests

1 participant