Repository navigation

GH-48701: [C++][Parquet] Add ALPpd encoding - #48345

Open

prtkgaur wants to merge 165 commits into

apache:mainfrom

prtkgaur:gh540-alp-pseudoDecimal-encoding

Commits on Oct 7, 2026
- Add the ALP floating-point codec
  sfc-gh-pgaur
  committed
  4a83a46
- Wire ALP into the Parquet encoder and decoder
  sfc-gh-pgaur
  committed
  526b415
- Add an ALP encoding benchmark
  sfc-gh-pgaur
  committed
  2bac3ef
- Add floating-point datasets for the benchmark
  sfc-gh-pgaur
  committed
  c4a0f53
- Add the ALP sources to CMake
  sfc-gh-pgaur
  committed
  a580d2f
- Rename the ALP headers from .hpp to .h
  sfc-gh-pgaur
  committed
  303c11d
- Draw the ALP diagrams in ASCII
  sfc-gh-pgaur
  committed
  f83358a
- Rename the ALP .cpp files to .cc
  sfc-gh-pgaur
  committed
  be8d267
- Use Arrow's Doxygen style in the ALP docs
  sfc-gh-pgaur
  committed
  642d61a
- Follow Arrow naming style in ALP
  sfc-gh-pgaur
  committed
  d419ec8
- Follow Arrow style in the ALP tests
  sfc-gh-pgaur
  committed
  59f1a67
- Update ALP callers for the renamed methods
  sfc-gh-pgaur
  committed
  ed0a119
- Fuse FOR and decode loop
  sfc-gh-pgaur
  committed
  6aeb406
- Reduce memory allocation in the decompress call
  sfc-gh-pgaur
  committed
  e7bb7c0
- Try SIMD for ALP decoding
  sfc-gh-pgaur
  committed
  1db3182
- Revert "Try SIMD for ALP decoding"
  sfc-gh-pgaur
  committed
  261498a
- Point CMake at the renamed .cc files
  sfc-gh-pgaur
  committed
  9f7882f
- Move the ALP datasets to parquet-testing
  sfc-gh-pgaur
  committed
  5cc4fa6
- Update the ALP benchmark data path
  sfc-gh-pgaur
  committed
  465ecde
- Use Arrow file naming for the ALP sources
  sfc-gh-pgaur
  committed
  dd4836c
- Update ALP includes for the renamed files
  sfc-gh-pgaur
  committed
  7fb7006
- Read ALP sizes from the page header
  sfc-gh-pgaur
  committed
  6059bbb
- Drop redundant ALP namespace qualifiers
  sfc-gh-pgaur
  committed
  2790163
- Better pack the compression block header
  sfc-gh-pgaur
  committed
  1a6bbf6
- Rename CompressionBlockHeader to AlpHeader
  sfc-gh-pgaur
  committed
  0014b5a
- Reorder ALP vector metadata for packing
  sfc-gh-pgaur
  committed
  7b05d39
- Add the ALP specification drafts
  sfc-gh-pgaur
  committed
  241b899
- Add more ALP codec tests
  sfc-gh-pgaur
  committed
  33e75ca
- Move the element count to the ALP page header
  sfc-gh-pgaur
  committed
  bfe0913
- Keep the ALP format version at 1
  sfc-gh-pgaur
  committed
  1526c60
- Set kAlpVersion back to 1
  sfc-gh-pgaur
  committed
  0e4792f
- Derive each ALP vector's element count
  sfc-gh-pgaur
  committed
  e065e69
- Update the ALP specification drafts
  sfc-gh-pgaur
  committed
  69ff2b4
- Copy ALP exception arrays into aligned storage
  sfc-gh-pgaur
  committed
  134e9ef
- Use SafeCopy for the FOR subtraction
  sfc-gh-pgaur
  committed
  985cdd8
- Add Parquet-level ALP tests
  sfc-gh-pgaur
  committed
  b31cd27
- Templatize AlpEncodedVectorInfo
  sfc-gh-pgaur
  committed
  75ea9e5
- Size ALP vector headers by kStoredSize
  sfc-gh-pgaur
  committed
  8c1f5b7
- Cap the ALP page element count at INT32_MAX
  sfc-gh-pgaur
  committed
  8c1e72e
- Rename the ALP layout to IntegerEncoding
  sfc-gh-pgaur
  committed
  2f0a563
- Rename kBitPack to kForBitPack
  sfc-gh-pgaur
  committed
  82060b7
- Make AlpHeader exactly 8 bytes
  sfc-gh-pgaur
  committed
  ab5c6b4
- Move ALP vector data toward the start
  sfc-gh-pgaur
  committed
  d1bcd28
- Read ALP metadata through a wrapper
  sfc-gh-pgaur
  committed
  f740ec2
- Separate the ALP and FOR metadata
  sfc-gh-pgaur
  committed
  04b991a
- Drop the padding byte from the FOR metadata
  sfc-gh-pgaur
  committed
  3b73329
- Put FOR-specific code behind a switch
  sfc-gh-pgaur
  committed
  cb5d981
- Test empty input and the FOR metadata size
  sfc-gh-pgaur
  committed
  189d04c
- Store per-vector offsets in ALP pages
  sfc-gh-pgaur
  committed
  fcba8d5
- Document the offset-based ALP layout
  sfc-gh-pgaur
  committed
  0261c9a
- Add ALP sampling presets
  sfc-gh-pgaur
  committed
  58bb648
- Update ALP comments for the offset layout
  sfc-gh-pgaur
  committed
  607d72f
- Add an ALP reference blob generator
  sfc-gh-pgaur
  committed
  adf80b6
- Add a tool that writes ALP Parquet files
  sfc-gh-pgaur
  committed
  65e43d9
- Drop the version byte from the ALP header
  sfc-gh-pgaur
  committed
  45d23e2
- Add ALP test files to parquet-testing
  sfc-gh-pgaur
  committed
  6d3a520
- Test reading ALP-encoded Parquet files
  sfc-gh-pgaur
  committed
  25a7ede
- Test reading Java-written ALP files
  sfc-gh-pgaur
  committed
  4c2f127
- Test FLOAT ALP files across languages
  sfc-gh-pgaur
  committed
  4296be7
- Check the Status returned by ALP Decode
  sfc-gh-pgaur
  committed
  a1be2f7
- Bounds-check each ALP vector before reading
  sfc-gh-pgaur
  committed
  77c0204
- Rename ALP types and tidy the codec API
  sfc-gh-pgaur
  committed
  2182a48
- Fix narrowing and call sites after the renames
  sfc-gh-pgaur
  committed
  9886e8a
- Batch the ALP size estimate for vectorization
  sfc-gh-pgaur
  committed
  4ec9771
- Use signed sizes and byte spans in ALP
  sfc-gh-pgaur
  committed
  8e8ef6f
- Move decoder creation outside benchmark loop
  sfc-gh-pgaur
  committed
  8d30342
- Rename alp_wrapper to alp_codec
  sfc-gh-pgaur
  committed
  5ebc724
- Return Status from ALP decode and load paths
  sfc-gh-pgaur
  committed
  c1185a2
- Document the ALP Store preconditions
  sfc-gh-pgaur
  committed
  1443bde
- Use signed integer types in ALP
  sfc-gh-pgaur
  committed
  4eb1c28
- Put ALP out parameters last
  sfc-gh-pgaur
  committed
  10ba334
- Remove the ALP specification drafts
  sfc-gh-pgaur
  committed
  9259112
- Rename ALP codec params to input and output
  sfc-gh-pgaur
  committed
  029349b
- Copy ALP header fields one at a time
  sfc-gh-pgaur
  committed
  362fcdf
- Decode any power-of-two ALP vector size
  sfc-gh-pgaur
  committed
  8ee30f0
- Make the ALP encode vector size configurable
  sfc-gh-pgaur
  committed
  169527a
- Test Parquet ALP decoding at other vector sizes
  sfc-gh-pgaur
  committed
  79809ff
- Use int64_t sizes in the AlpCodec API
  sfc-gh-pgaur
  committed
  51a7e99
- Round negative values correctly in FastRound
  sfc-gh-pgaur
  committed
  95fad0c
- Use SafeLoad/SafeStore and classes in ALP
  sfc-gh-pgaur
  committed
  1a8ac27
- Make the ALP vector members private
  sfc-gh-pgaur
  committed
  f71025d
- Remove AlpMetadataCache and GetNumElements
  sfc-gh-pgaur
  committed
  657d21f
- Drop private inheritance from ALP classes
  sfc-gh-pgaur
  committed
  af4cca7
- Spell out LoadHeader's return type
  sfc-gh-pgaur
  committed
  90fb841
- Use SafeLoadAs for all header fields in LoadHeader
  sfc-gh-pgaur
  committed
  133f369
- Validate header fields in LoadHeader
  sfc-gh-pgaur
  committed
  ff04d74
- Count AlpCodec input in elements, not bytes
  sfc-gh-pgaur
  committed
  4598521
- Use uint8_t* in the AlpCodec API
  sfc-gh-pgaur
  committed
  e53e420
- Drop trailing return types in ALP
  sfc-gh-pgaur
  committed
  2d5d7fd
- Cite the ALP paper for the tie-break rule
  sfc-gh-pgaur
  committed
  4015a9e
- Pin AlpMode's underlying type to uint8_t
  sfc-gh-pgaur
  committed
  d48496b
- Tweak ALP docs, add an incremental-decode TODO
  sfc-gh-pgaur
  committed
  d9c87f4
- Convert AlpSampler uint64_t to int64_t
  sfc-gh-pgaur
  committed
  edd18bb
- Drop the issue link from the decode TODO
  sfc-gh-pgaur
  committed
  a81b637
- Document how ALP encode reports bad input
  sfc-gh-pgaur
  committed
  88dff38
- Use bit_util::CeilDiv for ALP vector counts
  sfc-gh-pgaur
  committed
  363d0b5
- Compare ALP test output with IsBitwiseEqual
  sfc-gh-pgaur
  committed
  d00cab8
- Type-parameterize the ALP integration tests
  sfc-gh-pgaur
  committed
  29d00cb
- Type-parameterize the AlpEncodingAdHoc tests
  sfc-gh-pgaur
  committed
  058add5
- Widen the RandomData range to hit exceptions
  sfc-gh-pgaur
  committed
  44426ea
- Remove the uncompiled ALP generator tools
  sfc-gh-pgaur
  committed
  b0e8965
- Add an incremental-encode TODO to AlpEncoder
  sfc-gh-pgaur
  committed
  51cc19d
- Rename EncodeAlp parameter combinations to preset
  sfc-gh-pgaur
  committed
  2042bac
- Use bit_util helpers for ALP bit math
  sfc-gh-pgaur
  committed
  6209f19
- Remove 'non-templated' from ALP comments
  sfc-gh-pgaur
  committed
  427c03e
- Inline GetBitPackedSize at its callers
  sfc-gh-pgaur
  committed
  968ca17
- Assert kStoredSize against the member sizes
  sfc-gh-pgaur
  committed
  8a8038c
- Return Status from the AlpCodec encode API
  sfc-gh-pgaur
  committed
  ca187cc
- Load ALP benchmark data with the CSV reader
  sfc-gh-pgaur
  committed
  51a6ff0
- Derive kStoredSize from the member sizes
  sfc-gh-pgaur
  committed
  34aa9b4
- Make ALP exception counts unsigned
  sfc-gh-pgaur
  committed
  48e6526
- Enforce the [3, 15] ALP log_vector_size range
  sfc-gh-pgaur
  committed
  8c346f2
- Use array matchers in the ALP tests
  sfc-gh-pgaur
  committed
  3e0603b
- Document the ALP vector_size range
  sfc-gh-pgaur
  committed
  a2888f5
- Put EncodeAlp output parameters last
  sfc-gh-pgaur
  committed
  a1235e7
- Test that ALP rejects malformed page headers
  sfc-gh-pgaur
  committed
  e6e5cd4
- Use std::countr_zero in ALP
  sfc-gh-pgaur
  committed
  0865523
- Apply clang-format 18 to the ALP sources
  sfc-gh-pgaur
  committed
  5c9bf12
- Validate ALP vector metadata before decoding
  sfc-gh-pgaur
  committed
  3e6ad01
- Correct inaccurate ALP comments
  sfc-gh-pgaur
  committed
  0d347fd
- Type-parameterize the ALP compression tests
  sfc-gh-pgaur
  committed
  e97584c
- Drop the ALP benchmark that needs fork data
  sfc-gh-pgaur
  committed
  80a99cd
- Test ALP against the conformance file
  sfc-gh-pgaur
  committed
  4bf8b21
- Put ALP behind an opt-in writer flag
  sfc-gh-pgaur
  committed
  4926c77
- Tighten the ALP tests and document the encoding
  sfc-gh-pgaur
  committed
  7b7b520
- Validate the ALP count and offset chain
  sfc-gh-pgaur
  committed
  cae830f
- Take the ALP count from the page header
  sfc-gh-pgaur
  committed
  cb34d5e
- Encode an all-null ALP page as a bare header
  sfc-gh-pgaur
  committed
  ae23e6e
- Build the ALP sources once, and under meson too
  sfc-gh-pgaur
  committed
  751f8b7
- Serialize ALP as little-endian on any host
  sfc-gh-pgaur
  committed
  843fb96
- Move AlpMode out of the public header
  sfc-gh-pgaur
  committed
  9beaafc
- Test the ALP count and offset validation
  sfc-gh-pgaur
  committed
  f61722f
- Test batched ALP decoding
  sfc-gh-pgaur
  committed
  e7fec3d
- Decode ALP pages one vector at a time
  sfc-gh-pgaur
  committed
  44e6db1
- Move the ALP end-to-end tests to their own file
  sfc-gh-pgaur
  committed
  1cdd4af
- Name the ALP headers _internal.h
  sfc-gh-pgaur
  committed
  50b5d52
- Fold the ALP vector-size test into the suite
  sfc-gh-pgaur
  committed
  ac261dd
- Reuse ALP decode scratch across vectors
  sfc-gh-pgaur
  committed
  9bb9201
- Drop dead ALP code and unpause the benchmarks
  sfc-gh-pgaur
  committed
  79984fb
- Export AlpCodec and flatten its namespace
  sfc-gh-pgaur
  committed
  734f70f
- Make AlpSampler's constants static
  sfc-gh-pgaur
  committed
  ca0507c
- Guard the ALP loop pragmas by compiler
  sfc-gh-pgaur
  committed
  0702557
- Export the nested VectorReader too
  sfc-gh-pgaur
  committed
  9d6e8ae
- Reject ALP values beyond the definition levels
  sfc-gh-pgaur
  committed
  b3d79de
- Size the sample buffer with integer division
  sfc-gh-pgaur
  committed
  1455886
- Describe the decode target type in prose
  sfc-gh-pgaur
  committed
  aa2e431
- Export the ALP decode member templates
  sfc-gh-pgaur
  committed
  4219b5b
- Cast the element count the ALP test passes
  sfc-gh-pgaur
  committed
  d9fa958
- Export the internal ALP classes for tests
  sfc-gh-pgaur
  committed
  4df7226
- Widen the shift the ALP test takes a modulus with
  sfc-gh-pgaur
  committed
  c32fd46
- Rename the ALP roundtrip helper
  sfc-gh-pgaur
  committed
  9f2b73e
- Read the stored size through its accessor
  sfc-gh-pgaur
  committed
  2e356bc
- Skip the byte copies that have nothing to copy
  sfc-gh-pgaur
  committed
  01b6386
- Keep ALP bounds one ulp below the int limit
  sfc-gh-pgaur
  committed
  bbba931
- Drop the parquet-testing submodule bump
  sfc-gh-pgaur
  committed
  df83dfb
- Trim the ALP comments
  sfc-gh-pgaur
  committed
  e0bcc57
- State the constraints the ALP tests check
  sfc-gh-pgaur
  committed
  892b195
- GH-48701: [C++][Parquet] Refactor and harden ALP encoding
  
  wgtmac
  authored and
  sfc-gh-pgaur
  committed
  3aff81c
- Restore ALP wire layout diagrams
  sfc-gh-pgaur
  committed
  c0dd888
- Document the metadata layouts at the structs
  sfc-gh-pgaur
  committed
  ed6ed63
- Test the factory throw; keep the fallback TODO
  sfc-gh-pgaur
  committed
  51e36be
- Document that ALP is never chosen automatically
  sfc-gh-pgaur
  committed
  dff2ea5
- Fix ALP builds across compilers
  sfc-gh-pgaur
  committed
  642ee4e
- Fix constrained ALP template exports
  sfc-gh-pgaur
  committed
  2fb0c5f
Commits on Oct 8, 2026
- Fix ALP vector metadata bounds on 32-bit targets
  sfc-gh-pgaur
  committed
  e2275a1

Commits

Commits on Oct 7, 2026