Skip to content

feat: pin ASAPPlanner main 05fb36e4 and execute its joint plans: #787 coverage, ignore_accuracy_guarantees, pane-width SDS keys, --stream-start-ms - #844

Merged
zzylol merged 5 commits into
mainfrom
feat/planner-787-coverage
Oct 21, 2026
Merged

zzylol merged 5 commits into
mainfrom
feat/planner-787-coverage

Conversation

@zzylol

@zzylol zzylol commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Why

The MVP three-arm benchmark needs the backend to execute what ASAPPlanner main plans. The benchmark runs ASAPQuery's seed-862 PromQL workloads with no raw data retained, accuracy guarantees ignored, and failure probability 0.3.

At the 7565c47e pin, the backend arm was invalid:

  • Its three quantile queries had no plan, so --no-forward refused them (422).
  • Its top-k answers lost the chosen series' labels, so their recall against Prometheus was 0.

ASAPPlanner main fixes both on the Planner side:

The backend runs the Planner's executor, so it gets these fixes by moving the pin. Running the joint plans then exposed two backend bugs, which this PR also fixes.

What

  • refactor: route precompute outputs on StoredOutputId, not a bare u64 #787 (StoredOutput).
    • A definition's identity is the plan's coverage.definition_id.
    • A summary instance is keyed by (definition id, concrete selection).
    • Pane width and offset come from inputs.raw_sample_windows, and stored inputs from inputs.stored_outputs.
  • Pin. Both Planner crates move to ASAPPlanner main 05fb36e4.
  • Control plane.
    • --plan-selection-settings FILE takes the Planner's PlanSelectionSettings JSON, the same file stage_pipeline reads.
    • --ignore-accuracy-guarantees.
    • --measured-operation-costs reads MeasuredOperationCosts.
    • The control plane prints queries_accepted_without_guarantee.
    • plan_installable applies the Planner executor's compile check. That is the backend's check too, since it runs the same executor.
  • Fix (SDS): panes of one definition id in two widths. A joint plan can store one computation in panes of two widths. Example: per-series sums in 1-second panes for topk(…, sum_over_time(data[1m])) every second, and in 30-second panes for …[20m] every 30 s, with the same definition id and label constraints.
    • Before: the SDS keyed definitions by (id, label constraints), so both widths shared one definition. The 20-minute query merged forty 1-second panes and answered 1/30 of each sum.
    • Now: DefinitionKey and Selection carry the pane width.
  • --stream-start-ms (precompute). The first pane each stored output sees is skipped, because it may have started before the data plane did. A producer that starts its stream at a declared time T makes a first pane that starts at or after T complete, and that pane is built. Without the flag, a query whose window reaches back to the feed start was refused.
  • Plan files. serde_json float_roundtrip is on, so a plan file reloads to the same plan; the new predicted_usage floats did not round-trip before.

How

No executor code changes. The data plane builds and answers with the Planner's executor: compile, StateColumnUpdater for per-sample panes, and series_labels_of_output_row for labels. The pin carries the DDSketch and top-k fixes into both pane builds.

data_plane/tests/planner_executor_answers.rs runs the Planner's runtime-install cases through the backend:

  1. Plan with the control plane, without raw data.
  2. Install with --no-forward.
  3. Write samples over Remote Write, one request per second.
  4. Close panes with the event-time barrier.
  5. Check every answer against the exact value.

Before this PR

At 9037b7cb (#787 at 4111e620), 7 of the 8 tests of the Planner fixes fail:

  • p99 of a gauge reporting zero, the interpolated median, and jointly planned quantiles: no valid plan without raw data.
  • Signed samples: the DDSketch pane build fails on negative values (pane ... is not stored).
  • topk by (region) (2, sum_over_time(data[1m])): answers {"region": "east"}, without instance and service.
  • The other top-k forms and the counter top-k: empty answers.

Two more failures surfaced at the new pin, before the backend fixes:

  • panes_of_one_definition_and_two_widths_are_stored_apart: the 20-minute sums were 40 and 80, where 1,200 and 2,400 are expected. In benchmark workload 01, q07 had max error 0.972 and passed 0 of 4 evaluations.
  • The first evaluation of q07 and q09 (S + 20 min) was refused.

After this PR

All tests pass. Benchmark seed-862 workload-01 (1,000 series, 20,000 samples/s, 3 rounds, arms run one after another on one machine) compares:

  • Prometheus 3.5.0 alone;
  • ASAPQuery with its archived pilot plan;
  • the backend with the joint plan and --no-forward.

Every backend answer came from the plan, and none was refused. Each one is within its measured error of Prometheus. The backend arm's Prometheus is fed but never queried.

Arm Process CPU core-s, ingest phase CPU core-s, query phase Peak RSS MiB Steady RSS MiB
Prometheus Prometheus 488.8 338.5 345 305
ASAPQuery query_engine_rust 228.8 31.6 383 369
ASAPQuery its Prometheus 486.3 46.9 326 307
Backend data_plane 162.5 50.8 278 268
Backend its Prometheus (not read) 479.3 32.8 163 140
Arm Query latency, all evaluations, p50 / p99 ms Remote Write per 1 s tick to the engine, p50 / p99 ms
Prometheus 541.7 / 2005.9 61.9 / 82.7
ASAPQuery 48.9 / 1549.3 110.8 / 138.6
Backend 291.0 / 496.3 68.6 / 100.2
Query Backend max error (median) ASAPQuery max error (median)
q03 p99, avg by (region, service, job) 0.049 (0.045) 0.096 (0.093)
q04 p95, avg by (region) 0.024 (0.010) 0.046 (0.023)
q05 p75, avg by (region, service, job) 0.048 (0.043) 0.096 (0.086)
q06 / q07 top-k sums 0 / 0 0.180 (recall 0.95) / 0.101
q01, q02, q08, q09 0 0

Verification

  • cargo test --workspace --locked: all pass.
    • The new data plane tests: 9 in planner_executor_answers.rs, plus the_first_pane_is_built_when_it_starts_at_the_declared_stream_start and the two-width store test in sds.rs.
    • ignoring_accuracy_guarantees_plans_the_undeclared_p99_query.
    • Each new test failed as listed above before its fix.
  • cargo clippy --workspace --all-targets --locked -- -D warnings and cargo fmt --check are clean.
  • Benchmark: seed-862 workload-01 as above. The joint plan at 05fb36e4 is identical to the one at ef52e175 for workloads 01 and 02.

@zzylol zzylol added enhancement New feature or request area: summary execution Warm-summary binding, readout, composition, and result decoding integration: ASAPPlanner ASAPPlanner API, IR, candidate, and selected-plan integration labels Oct 21, 2026
…selection

Pin ASAPPlanner 4111e620 (#787: StoredOutput gives its SummaryCoverage,
definition id and selection, and its inputs instead of a pane).

- A definition's identity is the plan's coverage.definition_id; the SDS's
  own sub-DAG hash is gone (and with it the sha2 dependency).
- The definition id leaves the selection out, so outputs of one id with
  other label constraints are other SummaryDefinitions (DefinitionKey: id
  and label constraints), and a summary instance is keyed by the id and its
  concrete Selection: the label constraints and the pane end the plan's
  relative time binds to.
- A definition's pane width and each output's offset come from its
  selection's relative time; its RawSampleWindow is the stored output's
  raw-sample input; its stored inputs are the plan's
  inputs.stored_outputs. A selection that is not one left-open time box, or
  several raw-sample windows, is refused.
…guarantees

- Both Planner crates move to ef52e175 (sketchlib planner-pin/0.4.0-plus:
  interpolated and signed DDSketch quantiles, top-k with series labels,
  Stage 3's executor compile check, joint search, shared stored summaries).
- control_plane: --plan-selection-settings (the Planner's
  PlanSelectionSettings JSON) and --ignore-accuracy-guarantees; prints the
  queries accepted without a guarantee. --measured-operation-costs reads
  MeasuredOperationCosts.
- serde_json float_roundtrip: a plan file reloads to the plan it was written
  from (its predicted usage is floats).
- data_plane/tests/planner_executor_answers.rs: the data plane answers as
  the Planner's executor for DDSketch panes over zero and signed samples,
  interpolated quantiles, top-k labels, counter top-k without guarantees,
  and jointly planned shared states.
@zzylol
zzylol force-pushed the feat/planner-787-coverage branch from 9037b7c to 1de950a Compare October 21, 2026 09:58
@zzylol
zzylol changed the base branch from fix/repin-planner-780 to main October 21, 2026 09:58
…nitions

A jointly planned workload can store one computation (one definition id,
the same label constraints) in panes of two widths, e.g. per-series sums in
1-second panes for `sum_over_time(data[1m])` every second and in 30-second
panes for `sum_over_time(data[20m])` every 30 s. The SDS keyed definitions
by (id, label constraints), so both shared one definition with the first
width: the 20-minute query merged forty 1-second panes and answered 1/30 of
its sum.

DefinitionKey and Selection now carry the pane width: a definition is (id,
label constraints, pane width), and a summary instance's concrete selection
is (label constraints, width, end), the interval (end - width, end].
… at the stream's start

The first pane each stored output sees is skipped, since it may have started
before the data plane did. When the producers start their stream at the
data plane's start (the benchmark feeds from an aligned start S), that pane
is complete, and a query whose window reaches back to S was refused: e.g.
`rate(data_total[20m])` and `sum_over_time(data[20m])` at S + 20 min.

A producer can now declare where its stream starts (PrecomputeOptions::
stream_start_ms, `--stream-start-ms`): a first pane starting at or after it
is built; one starting before it is still skipped.
@zzylol zzylol changed the title feat(sds): adopt ASAPPlanner #787: the plan's definition id and full selection feat: pin ASAPPlanner main 05fb36e4 and execute its joint plans: #787 coverage, ignore_accuracy_guarantees, pane-width SDS keys Oct 21, 2026
@zzylol zzylol added area: control plane Planning, configuration, routing, and control-plane orchestration area: data plane Runtime ingestion, storage, and query-serving data paths bug Something isn't working labels Oct 21, 2026
@zzylol zzylol changed the title feat: pin ASAPPlanner main 05fb36e4 and execute its joint plans: #787 coverage, ignore_accuracy_guarantees, pane-width SDS keys feat: pin ASAPPlanner main 05fb36e4 and execute its joint plans: #787 coverage, ignore_accuracy_guarantees, pane-width SDS keys, --stream-start-ms Oct 21, 2026
@zzylol
zzylol marked this pull request as ready for review October 21, 2026 14:55
@zzylol
zzylol merged commit de8b915 into main Oct 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: control plane Planning, configuration, routing, and control-plane orchestration area: data plane Runtime ingestion, storage, and query-serving data paths area: summary execution Warm-summary binding, readout, composition, and result decoding bug Something isn't working enhancement New feature or request integration: ASAPPlanner ASAPPlanner API, IR, candidate, and selected-plan integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant