Repository navigation
dogs3: a proxy half-join tactic over a backend, built on int_proxy's public surface - #1022
Merged
frankmcsherry merged 2 commits intoOct 24, 2026
Conversation
`debug_assert_sorted_bridge` checks the contract every proxy backend owes its tactic. A tactic built outside differential over the same bridges, such as dogs3's proxy half join, needs the same check, so it is `pub` rather than `pub(crate)`. The one change an outside implementor of a tactic over proxies needed from this module. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
…public surface The half join had no seam between its time logic and its data: dogs3's HalfJoinTactic reads the arrangement's batches itself. ProxyHalfJoinTactic implements HalfJoinTactic and owns the total-order comparison, strictness, the output time rule (the update's own time, and its data time joined with the arranged time) and blocking; ProxyHalfJoinBackend owns key matching, value proxies and output containers, like differential's join and reduce backends. Matches are pairs of positions into the block's two presented sides, which `cross` is handed back, so a backend evaluates the time rule in its own layout. It lives here, not in differential's int_proxy, and is built from that module's public surface (ProxyBridge, KeyPosition, the bridge check) the way any outside implementor would build a tactic over proxies. Arranged times are presented as stored: advancing them by a frontier moves them in the total order relative to incomparable stream times, so the backend is given no lower bound to advance by. The arranged side is a bridge with the join's contract, sorted and consolidated by ((key, val), time); the stream side is in key order only. The generic release batcher BlobList is public: every HalfJoinTactic needs a batcher that agrees with it on when an update is released, and this is the one for Vec containers. VecHalfJoinBackend is the reference backend. The tests run the proxy half join against the cursor half join in one dataflow on random inputs, strict and not, at one and two workers, and against the definition over product times, with several block sizes and colliding hashes. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
This was referenced Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #1019 and #1020: the proxy half-join tactic now lives in dogs3, next to the operator it drives, and is built only from differential's public
int_proxysurface, as any outside implementor would build a tactic over proxies.The half join had no seam between its time logic and its data: dogs3's
HalfJoinTacticreads the arrangement's batches itself, so nothing can wrap it, for example to read an arrangement as of a frontier.ProxyHalfJoinTacticimplementsHalfJoinTacticover aProxyHalfJoinBackend, in the shape of differential's proxy join and reduce.Notes:
Ord, whichTimestamppromises is a linear extension of the partial order. Nothing needsTotalOrder, so product times work.lower = {(1, 0)},(0, 5)sorts before(1, 0), but advances to(1, 5), which sorts after it. So the backend is given no lower bound to advance by.((key, val), time). The stream side is in key order only.crossis handed back, so a backend can evaluate the time rule in its own layout.BlobListis public. EveryHalfJoinTacticneeds a batcher that agrees with it on when an update is released, andBlobListis the one forVeccontainers.What it needed from differential
One change:
int_proxy::debug_assert_sorted_bridgeis nowpub. It checks the bridge contract every proxy backend owes its tactic, so an outside tactic should run the same check rather than a copy that can drift. Everything else was already public:ProxyBridgeandKeyPosition;Semigroup,Multiply,Lattice,consolidate_updatesandHashable;ChunkBatchandVecChunk.Two private helpers weren't worth exposing, so dogs3 has its own: a batch type alias over public types, and an eight-line search.
Tests
tests/proxy_half_join.rsruns the proxy half join overVecHalfJoinBackendin two ways:Product<u64, u64>times, with several block sizes and colliding hashes.The dogs3 tests and
cargo test -p differential-dataflowpass, and clippy reports nothing in the files touched.DDIR's port to this tactic is #1021, which builds on this PR.
🤖 Generated with Claude Code
https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o