Repository navigation
Skip irrelevant count scans during lazy progress reporting - #830
Draft
frankmcsherry wants to merge 2 commits into
Draft
frankmcsherry wants to merge 2 commits into
frankmcsherry wants to merge 2 commits into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lazy progress reporting evaluates visibility before checking whether an update is a subtraction, and
is_globalscans retained timestamp counts even after an earlier frontier element establishes dominance.Check the scope-boundary exception and subtraction sign before asking the tracker about visibility, and short-circuit
PortInformation::is_globalwhen dominated. The predicates return the same results and add no state. Queries that survive the dominance check still inspect multiplicity.All three cases occur in the existing Differential BFS example. In one instrumented baseline run (10k vertices, 100k initial edges, ten waves of 1k separately timestamped replacements, four workers), 7,158 lazy-broadcast candidates included 3,373 dominated candidates, 764 nonnegative candidates, and 308 boundary candidates. These disjoint categories avoid scans over 14,986 raw batch entries; the remaining queries scan 9,230 entries. Dominated cases principally came from
Leave,Arrange: Reduce, andFeedback, and nonnegative cases fromArrangeByKey. Counts vary with scheduling and are diagnostic, not performance measurements.Seven interleaved, uninstrumented release runs of existing Timely PageRank, hash-join, barrier, and Differential BFS examples showed median execution times within 2% of baseline. The examples establish that the cases occur, but their small retained progress state does not establish a material application speedup. No benchmark example is added.
A product-timestamp regression test exercises dominance, redundant frontier counts, and visibility after the frontier advances. Workspace tests, doctests, and Clippy passed before removal of the standalone benchmark; production and test code are unchanged. Existing application examples were additionally rebuilt and run against this independent draft.