Repository navigation
Conversation
zzylol
force-pushed
the
stack/windows-2-segments
branch
from
October 5, 2026 03:07
7148bf3 to
14921d0
Compare
4 tasks done
zzylol
force-pushed
the
stack/windows-3-pricing
branch
from
October 5, 2026 06:21
44291c9 to
9cf2956
Compare
This was referenced Oct 5, 2026
Closed
zzylol
force-pushed
the
stack/windows-3-pricing
branch
from
October 8, 2026 16:47
9cf2956 to
a87cb4d
Compare
zzylol
force-pushed
the
stack/windows-2-segments
branch
from
October 10, 2026 14:32
1d70fb2 to
7d76639
Compare
zzylol
force-pushed
the
stack/windows-3-pricing
branch
from
October 10, 2026 14:32
a87cb4d to
15bc533
Compare
…eps (Q66) Stage 3 charged every query-time TimeShift (as a pass-through) and every query-time TimeRange (as a filter) one operation per row of the scan below it. Over Example 3a's shared 5-year scan each shift and range cost as much as the scan's rows, so the shared-segment plan (10 of them) lost to the shared-input plan (8). A TimeShift now costs nothing at either timing: it only re-labels time, and the executor folds the offset into the time bounds of the read below it. A query-time TimeRange is charged one operation per row it keeps; at ingestion time it keeps every arriving row, so nothing changes there. Pane roles are unchanged. Example 3a and 4a now select the shared 1-year segments (18,104/s vs 25,112/s for shared input in 3a). Example 3b/4 Pattern B's query-time panes now take 130 ms per evaluation, within the 200 ms bound, so B2 is valid (still costlier than the selected plan). Example 1's selection is unchanged; its plans with tumbling panes get cheaper. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Example 3a and 4a now select the shared 1-year segments, and 3b's query-time panes meet the latency bound. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…/TimeRange pricing Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol
force-pushed
the
stack/windows-2-segments
branch
from
October 11, 2026 17:11
7d76639 to
335d385
Compare
zzylol
force-pushed
the
stack/windows-3-pricing
branch
from
October 11, 2026 17:11
15bc533 to
2b12377
Compare
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
The specs and tests from #590 run the current stages through the shared planner_layering_common pipeline (Pass 1 and composition, optimize_physical_asap, select_plan); tests are named after the stages. Materialization is read from the physical DAG (#667); window forms and retention stay pending adapters. Behavior that works passes (29 tests); every expectation that needs a feature not in this stack is ignored with the PR that delivers it (29 tests): Pass 2 sharing (#588, #595, #596), window composition (#601, #628, #634), materialization enumeration and MatAtQuery (#604, #625), UnivMon sizing and accuracy (#621), SQL recognition and COUNT(*) composition (#598, #603) and plan selection row estimates (#632). Each spec's "Today" column states the current output. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
3 tasks
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
The specs and tests from #590 run the current stages through the shared planner_layering_common pipeline (Pass 1 and composition, optimize_physical_asap, select_plan); tests are named after the stages. Materialization is read from the physical DAG (#667); window forms and retention stay pending adapters. Behavior that works passes (29 tests); every expectation that needs a feature not in this stack is ignored with the PR that delivers it (29 tests): Pass 2 sharing (#588, #595, #596), window composition (#601, #628, #634), materialization enumeration and MatAtQuery (#604, #625), UnivMon sizing and accuracy (#621), SQL recognition and COUNT(*) composition (#598, #603) and plan selection row estimates (#632). Each spec's "Today" column states the current output. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
…stages (#577) * test: acceptance spec and pending tests for planner-layering Example 1 Define MVP acceptance for #509 Example 1 before the Phase C stage APIs exist: 1 -> 6 -> 6 -> 1 candidates (Pass 1 x identical-expression sharing; physical operator implementation only, no materialization). - Spec with per-stage candidate tables, invariants and doc ambiguities. - Integration tests against todo!() stage stubs, ignored until the stages land; workload and Stage 0 frontend-shape tests run now. - Expected-only asap-stage-pipeline/v1 fixture for the DAG viewer. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test: run the Example 1 acceptance tests against the real stages Replace the todo!() stubs with adapters over Stage 0-3, un-ignore the 11 tests that pass, and give each still-ignored test the precise difference between the implementation and the spec. Add runtime capability checks: the selected plan compiles in the physical planner; compiling every candidate stays ignored with the two runtime gaps it finds. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test: read Example 1 logical stages as flat DAGs after the export removal Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: Example 1 acceptance on the composed stages (1 -> 3 -> 3 -> 1) The tests run Pass 1 and composition, optimize_physical_asap and select_plan through a shared tests/planner_layering_common pipeline, and follow the decided behavior: three Pass 1 candidates (Q2 exact, CMS+heap, CountSketch+heap), every sub-DAG NotMaterialized, P1 selected, CMS+heap invalid (weights not proven non-negative) and CountSketch+heap invalid (a Top-K output has no guarantee and no deployment evidence). Hydra (#605), Pass 2 sharing (#588) and the two runtime gaps (#593, #579) stay ignored with the PR that delivers them. Tests are named after the stages; the compile check no longer uses the PromQL fallback. The spec lists where the doc and the planner differ. The expected-only viewer fixture is removed: the generated four-stage fixture replaces it. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: port #590's Example 2-4 acceptance specs and tests The specs and tests from #590 run the current stages through the shared planner_layering_common pipeline (Pass 1 and composition, optimize_physical_asap, select_plan); tests are named after the stages. Materialization is read from the physical DAG (#667); window forms and retention stay pending adapters. Behavior that works passes (29 tests); every expectation that needs a feature not in this stack is ignored with the PR that delivers it (29 tests): Pass 2 sharing (#588, #595, #596), window composition (#601, #628, #634), materialization enumeration and MatAtQuery (#604, #625), UnivMon sizing and accuracy (#621), SQL recognition and COUNT(*) composition (#598, #603) and plan selection row estimates (#632). Each spec's "Today" column states the current output. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * feat(devtools): stage_pipeline examples for #509 Example 3 Ported from #590: --example planner-layering-3a (Pattern A, ad hoc batch of five p99 reports) and planner-layering-3b (Pattern B, repeating 5-min p99), the Example 3 acceptance tests' workloads. Example 1's output is unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: follow #576's per-consumer charging and exact top-k columns Plan selection charges a NotMaterialized node once per consuming query, so the charging helper checks NodeCost.multiplier and the Example 4 A3 ambiguity is recorded as decided. Example 1 checks that exact Q2 sorts by the ranked value and returns [job, topk_10]. The stage viewer guide's tab 1 row now describes the actual all-exact result. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): one planning pipeline for stage_pipeline and the acceptance tests run_planning_stages and the #509 example workloads move into the asap-devtools library; stage_pipeline serializes the run and the planner-layering tests adapt it. The shared data workload drops ingestion_rate for data at rest, which validate() requires. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * fix(devtools): select over every candidate; --max-displayed limits only the document stage_pipeline capped enumeration at 64, so Example 3a (243 candidates) selected P41 instead of the cheapest P122 (KLL for all five queries). The pipeline now enumerates, optimizes and prices every candidate; the document shows the first --max-displayed (default 64) plus the selected one, with `combinations` and `capped`. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs: Example 2's SQL candidates compose; the SQL test waits only on #598 Running the ignored SQL test composes every candidate and fails only on Q2's options (per-source counts until #598), so #603 is not a blocker. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: name stages by number and name; say query-language frontends The frontends' Run field, id and locals become stage0_query_language_frontends, and test names carry stage0..stage3 with the stage name. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: charging helper follows plan selection's per-consumer rule With a materialized node the helper accepted any multiplier >= 1. It now computes each node's computations as plan selection does (walks from query roots and materialized nodes' inputs, stopping at materialized nodes), checked on a hand-built candidate with a materialized node. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: Example 1 picks Q2 candidates by (option, no shared input) physical_with and the invalid-candidate check no longer assume one candidate per Q2 option, so #588's shared-input variants need no change. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs: Example 3 Pattern B charges each node once per consuming query Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs(stage-viewer): rows 2, 3a and 3b describe the planner's current output Read from uncapped runs: Example 2's sketch plans are rejected for having no (ε, δ) guarantee, 3a selects P122 (first of 108 tied cheapest plans, nothing shared yet) and 3b selects query-time KLL; panes and the latency check come with the window-summary PRs. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs(stage-viewer): row 2 from the SQL run; 3a ties 32 plans, not 108 Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: Example 1 counts only unshared candidates; stage3_ charging-helper names The 3-logical and 3-physical counts now hold when #588 adds shared-input variants. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): PlanningStagesRun fields carry stage number and name Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(devtools): unique temp file per stage_pipeline run cargo test runs the tests as threads of one process, so two runs of the same example shared a file name and one deleted the other's output. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
This was referenced Oct 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack: #574 → #620 → #618 → #621 → #627 → #625 → #628 → #632 → #634 → #616 → #617 → #622 → #624 → #629 → #630 → #631 → #633 → #635 → #636 → #637
Problem
Stage 3 (
crates/plan-selection/src/lib.rs) charged every query-timeTimeShift(as a pass-through) and every query-timeTimeRange(as a filter) one operation per row of the scan below them. Over Example 3a's shared 5-year scan, each shift and each range cost 2,920/s, the same as the whole scan. The shared-segment plan from #628 has 10 of them and the selected shared-input plan has 8, so segments cost 44,384/s against 39,128/s.Changes (Q66)
promql_fallback.rs,series_window(.., offset, ..)), and the per-entity summary path reads the shifted raw rows from the deployment (physical_planner/mod.rs, Pass 2 sharing and Stage 1 coverage parity with the retired MajorPass #580). So pricing it at zero is not an approximation for this executor.PaneRole) are unchanged. Retained panes and the shifts and ranges that feed only them still cost 0. The newest pane's shift is now free too. The ingestion-time pane test now expects 3 charged nodes instead of 4.docs/design_docs/proposals/stage3-cost-model.md: new "Time shifts and ranges (Q66)" paragraph. Example 1's materialized runner-up changes from 47.407 to 47.340.tools/dag-viewer/examples/planner-layering-example1.jsonis regenerated (--max-candidates 160). The selection (P82, 4.620) is the same. The 32 plans that use tumbling panes get cheaper because their shifts are now free and their ranges charge only kept rows (for example, P14 goes from 12.84 to 8.44).tools/dag-viewer/examples.json: the 3a, 3b and 4a stories now describe the new selections.stage3_b_tumbling_candidates_are_valid,stage3_b_rebuilding_a_million_series_meets_the_latency_bound). The unit testlatency_bound_rejects_slow_query_time_workstill covers the rejection.Selections (
stage_pipeline --example planner-layering-<e>)Test plan
time_shift_is_free_and_time_range_pays_for_the_rows_it_keeps: over a shared 5-year scan, the shift costs 0, and the 1-year and 5-year ranges are charged the same price per kept row.cargo fmt --all --check,cargo clippy --workspace --all-targets -D warningscargo test --workspace: 1690 passed, 0 failed, 13 ignored (after the rebase on main d4869a7; was 1679 passed, 13 ignored)tools/dag-viewer:python3 -m unittest test_viewer: 25 OKStacked on #628 (
stack/windows-2-segments).