Skip to content

feat(java): read ordered_by_source GraphAr topology through a reader facade - #983

Open
keksmd wants to merge 1 commit into
apache:mainfrom
keksmd:java-reader-facade
Open

keksmd wants to merge 1 commit into
apache:mainfrom
keksmd:java-reader-facade

Conversation

@keksmd

@keksmd keksmd commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

Reason for this PR

Part of #978: the first read vertical of #947. It turns a GraphAr dataset's ordered_by_source adjacency and edge properties into edge and neighbor streams without JNI. Vertex property scans are a stated follow-up and are not in this PR.

What changes are included in this PR?

  • New graphar-reader module. GraphReader sits over graphar-info, graphar-core and the PhysicalReader contract from graphar-io-api. graphar-io-parquet is a test dependency only.
  • OrderedSourceEdgeReader.neighbors(vertexId): OrderedAdjacencyResolver locates the offset chunk, only the two bounding offsets are read, and only that row range of the adjacency chunks it touches is requested.
  • scanEdges() streams every topology row chunk by chunk.
  • scanEdges(properties) and per-vertex edge property cursors read the property chunks with the same row ranges and join them by row position. A chunk that returns a different row count than its range fails the read.
  • java-sdk.yml runs the new module.

Are these changes tested?

Yes, against the canonical testing/ldbc_sample Parquet fixtures, compared with values read through Arrow C++. Per-vertex reads are checked by recording physical requests and counting storage opens. The Arrow C++ edge chunks have no Parquet Offset Index, which graphar-io-parquet currently needs for a partial row-group range, so per-vertex tests read a copy rewritten by ParquetPhysicalWriter. That io-parquet gap is tracked in #981. mvn -pl reader -am clean verify: reader 8 tests, 0 failures.

Are there any user-facing changes?

New module graphar-reader. No changes to existing APIs.

Checklist

  • I have performed a self-review of my own code.
  • I have formatted my own code using make cpplint before submitting when changed files are in the cpp directory.
  • I have performed pre-commit run before commit the changed files.
  • I have added tests to prove my changes are effective.

…facade

Adds the graphar-reader module, the first read vertical over graphar-info,
graphar-core and the graphar-io-api PhysicalReader contract, without JNI.

GraphReader opens a graph from its YAML and hands out an
OrderedSourceEdgeReader. neighbors(vertexId) locates the offset chunk with
OrderedAdjacencyResolver, reads only the two bounding offsets, and requests
only that row range of the adjacency chunks it touches. scanEdges streams
every topology row chunk by chunk. Edge property chunks are read with the
same row ranges and joined by row position; a chunk that returns a
different row count than its range fails the read.

io-parquet stays a test dependency. Fixture tests compare scans of
testing/ldbc_sample with values read through Arrow C++, and check per-vertex
reads by recording requests and storage opens.

Part of apache#978; vertex property scans follow in a separate change.

Constraint: io-parquet refuses a partial row-group range without a Parquet
Offset Index, and the Arrow C++ edge chunks in testing/ldbc_sample have none,
so per-vertex reads are tested on a copy rewritten by ParquetPhysicalWriter.
Rejected: caching whole offset chunks | reads more than one vertex needs
Not-tested: per-vertex reads of C++ edge chunks without an Offset Index

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant