Repository navigation
Conversation
…facade Adds the graphar-reader module, the first read vertical over graphar-info, graphar-core and the graphar-io-api PhysicalReader contract, without JNI. GraphReader opens a graph from its YAML and hands out an OrderedSourceEdgeReader. neighbors(vertexId) locates the offset chunk with OrderedAdjacencyResolver, reads only the two bounding offsets, and requests only that row range of the adjacency chunks it touches. scanEdges streams every topology row chunk by chunk. Edge property chunks are read with the same row ranges and joined by row position; a chunk that returns a different row count than its range fails the read. io-parquet stays a test dependency. Fixture tests compare scans of testing/ldbc_sample with values read through Arrow C++, and check per-vertex reads by recording requests and storage opens. Part of apache#978; vertex property scans follow in a separate change. Constraint: io-parquet refuses a partial row-group range without a Parquet Offset Index, and the Arrow C++ edge chunks in testing/ldbc_sample have none, so per-vertex reads are tested on a copy rewritten by ParquetPhysicalWriter. Rejected: caching whole offset chunks | reads more than one vertex needs Not-tested: per-vertex reads of C++ edge chunks without an Offset Index
keksmd
force-pushed
the
java-reader-facade
branch
from
November 12, 2026 19:23
e6964f9 to
40a889f
Compare
3 of 4 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reason for this PR
Part of #978: the first read vertical of #947. It turns a GraphAr dataset's
ordered_by_sourceadjacency and edge properties into edge and neighbor streams without JNI. Vertex property scans are a stated follow-up and are not in this PR.What changes are included in this PR?
graphar-readermodule.GraphReadersits overgraphar-info,graphar-coreand thePhysicalReadercontract fromgraphar-io-api.graphar-io-parquetis a test dependency only.OrderedSourceEdgeReader.neighbors(vertexId):OrderedAdjacencyResolverlocates the offset chunk, only the two bounding offsets are read, and only that row range of the adjacency chunks it touches is requested.scanEdges()streams every topology row chunk by chunk.scanEdges(properties)and per-vertex edge property cursors read the property chunks with the same row ranges and join them by row position. A chunk that returns a different row count than its range fails the read.java-sdk.ymlruns the new module.Are these changes tested?
Yes, against the canonical
testing/ldbc_sampleParquet fixtures, compared with values read through Arrow C++. Per-vertex reads are checked by recording physical requests and counting storage opens. The Arrow C++ edge chunks have no Parquet Offset Index, whichgraphar-io-parquetcurrently needs for a partial row-group range, so per-vertex tests read a copy rewritten byParquetPhysicalWriter. That io-parquet gap is tracked in #981.mvn -pl reader -am clean verify: reader 8 tests, 0 failures.Are there any user-facing changes?
New module
graphar-reader. No changes to existing APIs.Checklist
make cpplintbefore submitting when changed files are in thecppdirectory.pre-commit runbefore commit the changed files.