Skip to content

feat(java): let a read request name what it wants pushed down - #971

Closed
keksmd wants to merge 1 commit into
apache:mainfrom
keksmd:java-io-api-read-request
Closed

keksmd wants to merge 1 commit into
apache:mainfrom
keksmd:java-io-api-read-request

Conversation

@keksmd

@keksmd keksmd commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Adds the request-side value types for a pushdown-capable read: Projection, RowRange, Filter, ComparisonOperator, Literal, ReadRequest, and the ReadCapability enum a physical reader declares support for.

This is the next slice after #961 (merged) — it only describes what a caller can ask a reader to push down. It does not add a reader, a result type, or IO. That's the follow-up PR.

Focused verify: mvn --no-transfer-progress -f maven-projects/pom.xml -pl io-api -am spotless:check and clean verify -Dspotless.check.skip=true — clean, tests green.

@keksmd

keksmd commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

cc @SemyonSinchenko @SYaoJun — next slice after #961 (merged). Green CI.

@keksmd

keksmd commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

@SemyonSinchenko @SYaoJun this is the read half of the IO API you reviewed in #961, so it may be easiest to look at with that context. Rebased on current main, no conflicts, CI green.

It adds the request vocabulary only: projection, optional row range, optional predicate and limit. No query AST and no Arrow dependency in the public surface.

@keksmd

keksmd commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (now including #966 and #968) and force-pushed, so CI here runs against what would actually merge rather than the older base. Locally mvn -pl io-api -am clean verify gives 53 tests, 0 failures, 0 errors, 0 skipped.

@keksmd

keksmd commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

@SemyonSinchenko when you have a moment — this one is green and rebased on current main. It is the smaller half of the read contract: the request object plus the pushdown vocabulary, no behaviour yet.

I cannot add reviewers on this repo, so a ping is the only lever I have. Merging this before #972 keeps that one down to a single commit.

A physical read is not only a URI. A caller knows which columns it needs,
which source rows, which values, and how many rows are enough - and a backend
that learns all four up front can skip work that a caller would otherwise have
to discard after the fact.

Add the request half of the read API: an ordered projection, a half-open row
range, an AND conjunction of row predicates over immutable scalars, and a row
limit. A request also reports which of those hints it actually carries, so a
backend can answer for each one instead of guessing.

A column is addressed through ColumnRef, not a bare String. A reference is
unresolved by construction because a request exists before its file is opened;
a backend binds it with Schema.resolve, which matches the exact name and
refuses an unknown or ambiguous column instead of picking the first hit.

A Filter is a closed predicate tree: a column comparison, or AND, OR and NOT
over predicates, built through one named factory per operator so a null check
cannot be handed a value and a comparison cannot be handed null. A backend walks
the tree through a Visitor with one callback per node kind, so adding a node
kind is a compile error in every reader rather than a silently skipped branch.

Constraint: filter literals stay immutable and lossless. NaN, infinity, and
sub-millisecond timestamps are refused rather than silently compared against a
value a stored format cannot represent.
@keksmd
keksmd force-pushed the java-io-api-read-request branch from 72abab9 to 73c7b55 Compare September 16, 2026 07:35
@keksmd

keksmd commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

Superseded: #972 was merged with these commits stacked underneath it, so the request object and pushdown vocabulary are already on main (6f97f01). Closing to avoid a duplicate.

@keksmd keksmd closed this Sep 16, 2026
keksmd added a commit to keksmd/incubator-graphar that referenced this pull request Sep 16, 2026
A projection and a filter used to reference a column by a bare string
and a filter was a single comparison; a backend could not tell a name
from a qualified path and a caller could not express AND/OR/NOT. The
read request now carries ColumnRef, a typed exact-name reference that
only a Schema resolves, and Filter is a closed predicate tree of
comparisons, junctions and negation walked through a Visitor.

Every reader, Parquet and Iceberg consumer builds its projection from
ColumnRef; the Parquet backend still declines filter pushdown, so
results are unchanged.

Constraint: mirrors apache#971 and apache#972 so the fork
stays source-compatible with the upstream io-api surface.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant