Repository navigation
feat: pin ASAPPlanner main 05fb36e4 and execute its joint plans: #787 coverage, ignore_accuracy_guarantees, pane-width SDS keys, --stream-start-ms - #844
Merged
Conversation
…selection Pin ASAPPlanner 4111e620 (#787: StoredOutput gives its SummaryCoverage, definition id and selection, and its inputs instead of a pane). - A definition's identity is the plan's coverage.definition_id; the SDS's own sub-DAG hash is gone (and with it the sha2 dependency). - The definition id leaves the selection out, so outputs of one id with other label constraints are other SummaryDefinitions (DefinitionKey: id and label constraints), and a summary instance is keyed by the id and its concrete Selection: the label constraints and the pane end the plan's relative time binds to. - A definition's pane width and each output's offset come from its selection's relative time; its RawSampleWindow is the stored output's raw-sample input; its stored inputs are the plan's inputs.stored_outputs. A selection that is not one left-open time box, or several raw-sample windows, is refused.
…guarantees - Both Planner crates move to ef52e175 (sketchlib planner-pin/0.4.0-plus: interpolated and signed DDSketch quantiles, top-k with series labels, Stage 3's executor compile check, joint search, shared stored summaries). - control_plane: --plan-selection-settings (the Planner's PlanSelectionSettings JSON) and --ignore-accuracy-guarantees; prints the queries accepted without a guarantee. --measured-operation-costs reads MeasuredOperationCosts. - serde_json float_roundtrip: a plan file reloads to the plan it was written from (its predicted usage is floats). - data_plane/tests/planner_executor_answers.rs: the data plane answers as the Planner's executor for DDSketch panes over zero and signed samples, interpolated quantiles, top-k labels, counter top-k without guarantees, and jointly planned shared states.
zzylol
force-pushed
the
feat/planner-787-coverage
branch
from
October 21, 2026 09:58
9037b7c to
1de950a
Compare
…lan compile check)
…nitions A jointly planned workload can store one computation (one definition id, the same label constraints) in panes of two widths, e.g. per-series sums in 1-second panes for `sum_over_time(data[1m])` every second and in 30-second panes for `sum_over_time(data[20m])` every 30 s. The SDS keyed definitions by (id, label constraints), so both shared one definition with the first width: the 20-minute query merged forty 1-second panes and answered 1/30 of its sum. DefinitionKey and Selection now carry the pane width: a definition is (id, label constraints, pane width), and a summary instance's concrete selection is (label constraints, width, end), the interval (end - width, end].
… at the stream's start The first pane each stored output sees is skipped, since it may have started before the data plane did. When the producers start their stream at the data plane's start (the benchmark feeds from an aligned start S), that pane is complete, and a query whose window reaches back to S was refused: e.g. `rate(data_total[20m])` and `sum_over_time(data[20m])` at S + 20 min. A producer can now declare where its stream starts (PrecomputeOptions:: stream_start_ms, `--stream-start-ms`): a first pane starting at or after it is built; one starting before it is still skipped.
zzylol
marked this pull request as ready for review
October 21, 2026 14:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The MVP three-arm benchmark needs the backend to execute what ASAPPlanner main plans. The benchmark runs ASAPQuery's seed-862 PromQL workloads with no raw data retained, accuracy guarantees ignored, and failure probability 0.3.
At the
7565c47epin, the backend arm was invalid:--no-forwardrefused them (422).ASAPPlanner main fixes both on the Planner side:
ignore_accuracy_guarantees.The backend runs the Planner's executor, so it gets these fixes by moving the pin. Running the joint plans then exposed two backend bugs, which this PR also fixes.
What
StoredOutput).coverage.definition_id.inputs.raw_sample_windows, and stored inputs frominputs.stored_outputs.05fb36e4.--plan-selection-settings FILEtakes the Planner'sPlanSelectionSettingsJSON, the same filestage_pipelinereads.--ignore-accuracy-guarantees.--measured-operation-costsreadsMeasuredOperationCosts.queries_accepted_without_guarantee.plan_installableapplies the Planner executor's compile check. That is the backend's check too, since it runs the same executor.topk(…, sum_over_time(data[1m]))every second, and in 30-second panes for…[20m]every 30 s, with the same definition id and label constraints.DefinitionKeyandSelectioncarry the pane width.--stream-start-ms(precompute). The first pane each stored output sees is skipped, because it may have started before the data plane did. A producer that starts its stream at a declared timeTmakes a first pane that starts at or afterTcomplete, and that pane is built. Without the flag, a query whose window reaches back to the feed start was refused.serde_jsonfloat_roundtripis on, so a plan file reloads to the same plan; the newpredicted_usagefloats did not round-trip before.How
No executor code changes. The data plane builds and answers with the Planner's executor:
compile,StateColumnUpdaterfor per-sample panes, andseries_labels_of_output_rowfor labels. The pin carries the DDSketch and top-k fixes into both pane builds.data_plane/tests/planner_executor_answers.rsruns the Planner's runtime-install cases through the backend:--no-forward.Before this PR
At
9037b7cb(#787 at4111e620), 7 of the 8 tests of the Planner fixes fail:pane ... is not stored).topk by (region) (2, sum_over_time(data[1m])): answers{"region": "east"}, withoutinstanceandservice.Two more failures surfaced at the new pin, before the backend fixes:
panes_of_one_definition_and_two_widths_are_stored_apart: the 20-minute sums were 40 and 80, where 1,200 and 2,400 are expected. In benchmark workload 01, q07 had max error 0.972 and passed 0 of 4 evaluations.After this PR
All tests pass. Benchmark seed-862 workload-01 (1,000 series, 20,000 samples/s, 3 rounds, arms run one after another on one machine) compares:
--no-forward.Every backend answer came from the plan, and none was refused. Each one is within its measured error of Prometheus. The backend arm's Prometheus is fed but never queried.
Verification
cargo test --workspace --locked: all pass.planner_executor_answers.rs, plusthe_first_pane_is_built_when_it_starts_at_the_declared_stream_startand the two-width store test insds.rs.ignoring_accuracy_guarantees_plans_the_undeclared_p99_query.cargo clippy --workspace --all-targets --locked -- -D warningsandcargo fmt --checkare clean.05fb36e4is identical to the one atef52e175for workloads 01 and 02.