Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add repetition index to 2.1 format #3106

Open
Tracked by #2856
westonpace opened this issue Nov 8, 2024 · 0 comments
Open
Tracked by #2856

Add repetition index to 2.1 format #3106

westonpace opened this issue Nov 8, 2024 · 0 comments
Assignees

Comments

@westonpace
Copy link
Contributor

westonpace commented Nov 8, 2024

The repetition index is a general purpose structure that is used in the following situations:

  • Miniblock encoding when there are repetition levels (in this case the index will store chunk offsets)
  • Zipped encoding when there are repetition levels OR when the data is variable-width (the index will store byte offsets)
  • Either miniblock or zipped encoding when there is RLE (this will be added much later)

The repetition index is not read during full scans. However, it is read during a partial scan of a page. The repetition index introduces "indirect I/O" back into the 2.1 format ( 🫠 )

@westonpace westonpace changed the title Add repetition index Add repetition index to 2.1 format Nov 8, 2024
@westonpace westonpace self-assigned this Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant