Blocked BitPacked: fuse FoR decode over blocked children

[mhk197]

Tracking Issue: #10167

Summary

FoR now decodes a blocked BitPacked child in one pass, the same as a global-width child. Before, the child was decoded to a primitive array first and the references were added in a second pass.

A block and a FoR chunk are both 1024 values, so each block unpacks at its own width and adds its chunk's reference directly.

Changes

• unpack_chunks walks the blocks of a blocked child, each at its own width.
• execute_block_offsets and for_each_packed_block are shared with the blocked BitPacked decoder.
• Block offsets widen to u64 directly instead of through a lazy cast.

Benchmarks

for_decode, per-chunk references, local M5 Max:

	Global width	Blocked, before	Blocked, after
u32	9.62 µs	18.54 µs	9.87 µs
i64	11.95 µs	20.04 µs	12.29 µs

Testing

• fused_decode_is_wrapping_add draws a bit width per block.
• for_fuses_blocked_child covers partial blocks, zero and native widths, and constant and per-chunk references.

Follow-up

FoRScheme still bit-packs its child with BITPACKING_V1, so the compressor doesn't write this combination yet. That needs #10245.

[codspeed]

Merging this PR will improve performance by 11.2%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 1 benchmark measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. This result is not comparable, so it counts as unchanged.

Preventing compiler optimizations

• bench_compare_primitive[(10000, 2)]

⚠️ 11 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• random_i128[0.01]
• pcodec_compress_f64
• multipolygon_mixed_validity
• take_varbin
• decompress[u16, (4000, 256)]
• ilike_contains
• decompress[u32, (1000, 256)]
• zstd_compress_string
• filter_run_end[len=4096_run=16_density=0.1]
• from_vec_drop_vortex[65536]
• …and 1 more

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
✅ 2211 untouched benchmarks
🆕 6 new benchmarks

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	WallTime	bitpack_blocked_decompress_neon[u32]	14.3 µs	12.7 µs	+12.38%
⚡	WallTime	bitpack_blocked_decompress_avx512[u32]	9.4 µs	8.6 µs	+10.04%
🆕	WallTime	decode_blocked_bitpacked_chunked_avx2[i64, 524288]	N/A	20.8 µs	N/A
🆕	WallTime	decode_blocked_bitpacked_chunked_avx2[u32, 524288]	N/A	20.6 µs	N/A
🆕	WallTime	decode_blocked_bitpacked_chunked_avx512[i64, 524288]	N/A	18 µs	N/A
🆕	WallTime	decode_blocked_bitpacked_chunked_avx512[u32, 524288]	N/A	17.5 µs	N/A
🆕	WallTime	decode_blocked_bitpacked_chunked_neon[i64, 524288]	N/A	27.7 µs	N/A
🆕	WallTime	decode_blocked_bitpacked_chunked_neon[u32, 524288]	N/A	26.7 µs	N/A

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mk/for-bitpacked-blocked-fused (8f30906) with develop (d42e7c9)}

[myrrc]

If I understand correctly, the issue may be that dst.len() may be less than chunk size. If it's greater, we currently copy to a scratch buffer although we can copy it to dst as there's sufficient capacity.

When exactly can dst.len() be less than a chunk size? When we are exporting the tail?

This is not exactly related to your change, but the exiting code seems suspicious to me

[mhk197]

There are three cases when dst.len() < FL_CHUNK_SIZE:

1. offset > 0, which happens for an array that has been sliced. so head.len() < FL_CHUNK_SIZE
2. tail.len() < FL_CHUNK_SIZE
3. both of the above

e.g.:

offset = 100, len = 2500

                   chunk 0                chunk 1                 chunk 2
position  0                       1024                   2048          2600       3072
          ┌───────────────────────┬──────────────────────┬────────────────────────┐
packed    │ offset ┆     924      │         1024         │     552     ┆  unused  │
          └───────────────────────┴──────────────────────┴────────────────────────┘
                   ▼              ▼                      ▼             ▼
output             0              924                    1948          2500
                   ├──────────────┼──────────────────────┼─────────────┤
                       dst: 924          dst: 1024          dst: 552
                     via scratch      unpack in place      via scratch

[mhk197]

It will never be greater