Repository navigation
Conversation
|
cc @SemyonSinchenko @SYaoJun — next slice after #961 (merged). Green CI. |
|
@SemyonSinchenko @SYaoJun this is the read half of the IO API you reviewed in #961, so it may be easiest to look at with that context. Rebased on current It adds the request vocabulary only: projection, optional row range, optional predicate and limit. No query AST and no Arrow dependency in the public surface. |
7d920aa to
72abab9
Compare
|
@SemyonSinchenko when you have a moment — this one is green and rebased on current I cannot add reviewers on this repo, so a ping is the only lever I have. Merging this before #972 keeps that one down to a single commit. |
A physical read is not only a URI. A caller knows which columns it needs, which source rows, which values, and how many rows are enough - and a backend that learns all four up front can skip work that a caller would otherwise have to discard after the fact. Add the request half of the read API: an ordered projection, a half-open row range, an AND conjunction of row predicates over immutable scalars, and a row limit. A request also reports which of those hints it actually carries, so a backend can answer for each one instead of guessing. A column is addressed through ColumnRef, not a bare String. A reference is unresolved by construction because a request exists before its file is opened; a backend binds it with Schema.resolve, which matches the exact name and refuses an unknown or ambiguous column instead of picking the first hit. A Filter is a closed predicate tree: a column comparison, or AND, OR and NOT over predicates, built through one named factory per operator so a null check cannot be handed a value and a comparison cannot be handed null. A backend walks the tree through a Visitor with one callback per node kind, so adding a node kind is a compile error in every reader rather than a silently skipped branch. Constraint: filter literals stay immutable and lossless. NaN, infinity, and sub-millisecond timestamps are refused rather than silently compared against a value a stored format cannot represent.
72abab9 to
73c7b55
Compare
A projection and a filter used to reference a column by a bare string and a filter was a single comparison; a backend could not tell a name from a qualified path and a caller could not express AND/OR/NOT. The read request now carries ColumnRef, a typed exact-name reference that only a Schema resolves, and Filter is a closed predicate tree of comparisons, junctions and negation walked through a Visitor. Every reader, Parquet and Iceberg consumer builds its projection from ColumnRef; the Parquet backend still declines filter pushdown, so results are unchanged. Constraint: mirrors apache#971 and apache#972 so the fork stays source-compatible with the upstream io-api surface.
Adds the request-side value types for a pushdown-capable read:
Projection,RowRange,Filter,ComparisonOperator,Literal,ReadRequest, and theReadCapabilityenum a physical reader declares support for.This is the next slice after #961 (merged) — it only describes what a caller can ask a reader to push down. It does not add a reader, a result type, or IO. That's the follow-up PR.
Focused verify:
mvn --no-transfer-progress -f maven-projects/pom.xml -pl io-api -am spotless:checkandclean verify -Dspotless.check.skip=true— clean, tests green.