Repository navigation
feat(plan-selection): a setting to accept outputs without accuracy guarantees - #799
Merged
Merged
Conversation
…arantees PlanSelectionSettings::ignore_accuracy_guarantees selects the built-in IgnoreAccuracyGuarantees model in place of the caller's accuracy model. Each accepted output records the cause the guarantee check would have rejected it with, and the installable plan's report lists those queries. Executability checks still apply. stage_pipeline gains --ignore-accuracy-guarantees.
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #774.
Why
The MVP benchmark compares sketch plans for queries such as
avg by (region) (quantile_over_time(0.99, data[100s])). Stage 3 has no analytical guarantee for these plans, so it rejects every one of them withNoAccuracyGuarantee, and a deployment that doesn't retain raw data gets no plan at all. The hookAccuracyModel::accepts_output_regardless_of_guaranteealready exists. But before this PR there was no built-in model that used it and no JSON setting orstage_pipelineflag to turn it on.What
IgnoreAccuracyGuaranteesaccuracy model. It accepts every output regardless of its guarantee. Itssatisfiesis the built-in rule, andmay_accept_regardless_of_guaranteereturnstrue, so the joint search from Joint planning search: stream Stage 1 to Stage 3 and reject permanently invalid query candidates (methods 2 + incumbent) #792 rejects no candidate early for accuracy.PlanSelectionSettings::ignore_accuracy_guarantees(JSON:{"ignore_accuracy_guarantees": true}, defaultfalse). When it is set, planning usesIgnoreAccuracyGuaranteesinstead of the caller'sPlanningModels::accuracy. This coversselect_plan,e2e_planandplan_installable.stage_pipeline --ignore-accuracy-guarantees, which also works together with--plan-selection-settings.AcceptedAccuracy::guarantee_check_rejectionholds the cause (NoAccuracyGuaranteeorAccuracyTargetMissed);SelectedPlanReport::queries_accepted_without_guaranteelists the affected queries in the installable plan;guarantee_check_rejectionappears in the stage_pipeline JSON.How
PlanningModels::effective_accuracy()returnsIgnoreAccuracyGuaranteeswhen the setting is on, andmodels.accuracyotherwise. Every read of the accuracy model in plan selection goes through it, including the early-rejection checks from #792. Inaccuracy_check, the guarantee and evidence check moved intoguarantee_check. That check now always runs. When the model accepts the output regardless of its guarantee, the check's result is only recorded. Otherwise the result decides, as before.Key code interfaces added / modified
crates/plan-selection/src/accuracy.rscrates/plan-selection/src/lib.rscrates/plan-selection/src/pass/mod.rscrates/planner/src/installable.rsCLI:
stage_pipeline ... --ignore-accuracy-guarantees(new).Usage
Without the setting, the same call fails with
no valid candidate: exact plans areRawDataNotRetained, and sketch plans areNoAccuracyGuarantee. This flow is the testignoring_accuracy_guarantees_selects_a_sketch_plan_that_runs.Before this PR
The only way to use the hook was to write an
AccuracyModelin code. JSON-driven runtimes andstage_pipelinecouldn't select sketch plans that have no guarantee. The report also couldn't tell an output whose guarantee was met apart from one accepted only by the model.After this PR
One setting,
"ignore_accuracy_guarantees": true, or onestage_pipelineflag, compares candidates on cost alone. The report lists every output that is accepted only because of the setting. With the setting off, nothing changes.Caveat: DDSketch needs strictly positive inputs at run time, and the executor refuses a sample of 0 or below. When the setting accepts DDSketch over a metric not declared positive, the plan fails at run time on such a sample. The undeclared MVP p99 query selects
P5-m1(DDSketch panes).Verification
Speed mode:
cargo test --workspace --locked --no-fail-fastran on the PR head. 151 test binaries: 1,799 passed, 2 failed, 36 ignored. The 2 failures are the #795 testsa_non_negative_gauges_merged_ddsketch_panes_carry_the_relative_bound_through_avgandan_undeclared_gauges_merged_ddsketch_panes_get_no_relative_bound_through_avg. They already fail on main at 825504e, with the same assertions in main's own CI test job:P3-m1vsP5-m1, which looks like a semantic conflict between #792 and #795. This PR doesn't touch them. Main's format-and-lint job also fails, on a clippytype_complexityerror incrates/types/src/ir/properties/summary_coverage.rs, which this PR doesn't touch either.New tests:
runtime_install::ignoring_accuracy_guarantees_selects_a_sketch_plan_that_runs: without the setting, planning fails with no valid candidate. With the setting loaded from JSON, a DDSketch plan is selected, the report marks query 0, and the plan runs. Its answers equal the query-time reference and are within 5 of the exact per-group averages of p99s.runtime_install::ignoring_accuracy_guarantees_plans_the_undeclared_p99_query: the MVP p99 query over an undeclared gauge gets a plan with no raw reads, and the report marks query 0.plan_selection::tests::unproven_count_min_weights_reject_even_when_ignoring_guarantees: CmsWithHeap over a gauge is stillCountMinWeightsNotProvenNonNegative.plan_selection::tests::ignoring_guarantees_accepts_and_marks_outputs_the_check_rejects: each rejection thatNoAccuracyGuaranteeorAccuracyTargetMissedcauses becomes a valid candidate whoseguarantee_check_rejectionholds that cause. Outputs whose guarantee meets the target recordNone.stage_pipeline::ignore_accuracy_guarantees_flag_accepts_outputs_without_a_guarantee: the flag makes theno_accuracy_guaranteecandidates valid and marked.The committed dag-viewer fixture was regenerated. Its only change is the new
guarantee_check_rejectionkey.