bitset: add range counting and parallel bit extract/deposit operations #184
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add three new operations to BitSet:
- OnesBetween(from, to uint) counts set bits in range [from, to)
- Extract/ExtractTo compacts bits from positions specified by a mask
- Deposit/DepositTo spreads bits to positions specified by a mask
OnesBetween operates at the word level where possible, handling edge
words with masks. It returns 0 for invalid ranges where from >= to.
Extract takes bits from positions where mask is set and packs them into
consecutive positions starting at 0. For example, if mask has bits set
at positions 1,4,5 then Extract will take bits at those positions from
the source and pack them into positions 0,1,2.
Deposit is the inverse of Extract - it spreads bits from consecutive
positions into positions specified by mask.
The parallel bit operations are implemented using byte-wise lookup
tables rather than CPU-specific instructions like PEXT/PDEP, making
them portable while maintaining good performance.
Added extensive testing:
- Property-based tests with 1M iterations verifying correctness
- Edge cases like single words, cross-word boundaries, empty sets
- Invariant checking (e.g. result bits ⊆ mask bits)
- Benchmarks across sizes from 64 to 65536 bits
These operations are useful for:
- Efficient bit field compression/decompression
- Sparse set operations
- Bit permutation tasks
The implementation aims to be both correct and fast, operating at
the word level where possible while properly handling edges.
Benchmarks
```
goos: darwin
goarch: arm64
pkg: github.com/bits-and-blooms/bitset
cpu: Apple M3 Max
│ bench │
│ sec/op │
BitSetOnesBetween/size=64/density=0.1-16 3.026n ± 1%
BitSetOnesBetween/size=64/density=0.5-16 2.959n ± 3%
BitSetOnesBetween/size=64/density=0.9-16 3.058n ± 1%
BitSetOnesBetween/size=256/density=0.1-16 3.154n ± 2%
BitSetOnesBetween/size=256/density=0.5-16 3.179n ± 1%
BitSetOnesBetween/size=256/density=0.9-16 3.146n ± 1%
BitSetOnesBetween/size=1024/density=0.1-16 4.938n ± 1%
BitSetOnesBetween/size=1024/density=0.5-16 4.934n ± 1%
BitSetOnesBetween/size=1024/density=0.9-16 4.881n ± 1%
BitSetOnesBetween/size=4096/density=0.1-16 8.661n ± 1%
BitSetOnesBetween/size=4096/density=0.5-16 8.787n ± 2%
BitSetOnesBetween/size=4096/density=0.9-16 8.582n ± 1%
BitSetOnesBetween/size=16384/density=0.1-16 29.05n ± 0%
BitSetOnesBetween/size=16384/density=0.5-16 30.11n ± 0%
BitSetOnesBetween/size=16384/density=0.9-16 29.34n ± 0%
PEXT-16 3.243n ± 2%
PDEP-16 3.219n ± 1%
BitSetExtractDeposit/size=64/fn=ExtractTo-16 10.55n ± 0%
BitSetExtractDeposit/size=64/fn=DepositTo-16 9.812n ± 1%
BitSetExtractDeposit/size=256/fn=ExtractTo-16 35.30n ± 1%
BitSetExtractDeposit/size=256/fn=DepositTo-16 38.16n ± 1%
BitSetExtractDeposit/size=1024/fn=ExtractTo-16 134.5n ± 1%
BitSetExtractDeposit/size=1024/fn=DepositTo-16 157.2n ± 1%
BitSetExtractDeposit/size=4096/fn=ExtractTo-16 580.1n ± 2%
BitSetExtractDeposit/size=4096/fn=DepositTo-16 642.0n ± 1%
BitSetExtractDeposit/size=16384/fn=ExtractTo-16 2.412µ ± 4%
BitSetExtractDeposit/size=16384/fn=DepositTo-16 2.459µ ± 2%
BitSetExtractDeposit/size=65536/fn=ExtractTo-16 10.12µ ± 1%
BitSetExtractDeposit/size=65536/fn=DepositTo-16 9.832µ ± 1%
geomean 30.55n
```
tsenart
force-pushed
the
ext-dep-ones-between
branch
from
03e3172 to
5a7da69
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add three new operations to BitSet:
OnesBetween operates at the word level where possible, handling edge words with masks. It returns 0 for invalid ranges where from >= to.
Extract takes bits from positions where mask is set and packs them into consecutive positions starting at 0. For example, if mask has bits set at positions 1,4,5 then Extract will take bits at those positions from the source and pack them into positions 0,1,2.
Deposit is the inverse of Extract - it spreads bits from consecutive positions into positions specified by mask.
The parallel bit operations are implemented using byte-wise lookup tables rather than CPU-specific instructions like PEXT/PDEP, making them portable while maintaining good performance.
Added extensive testing:
These operations are useful for:
The implementation aims to be both correct and fast, operating at the word level where possible while properly handling edges.
Benchmarks