[zzylol]

Part of #1011 (checklist 1, first item, and 5, first item). Refs #979.

Why

Whether two equal logical nodes become one memo group was decided by a hand-written list in cse.rs. A non-ASAP node was shared only when its schema had a provable unique key, or when it was a Scan, a TimeRange over a Scan, an exact PromQL TopK, or a global aggregate whose intent was on a list. A unique key is not what makes sharing safe. Sharing is safe when one copy returns each reader the rows it would have computed alone. For example, a per-series quantile without a key is safe to share. A LIMIT 10 without ORDER BY may return any 10 rows, so this PR keeps its copies separate. That is a conservative choice: one shared copy is still a valid answer for each reader, the way a database may materialize a CTE once.

Production impact is small, and I measured it. On main, the CLI and facade path (stage_pipeline) already gives #1011's example 7 groups (G3 reads [2, 2]). The frontend gives the per-series quantile a $promql_series_identity column and the unique key [[3, 0]], and the SQL frontend returns one shared node for identical sub-plans before CSE runs. #1011's 8 groups only appear with the test helper test_support::lower_promql, whose output has no key. Running the same 8 workloads through stage_pipeline on main and on this branch gave the same groups and the same selected cost for every one: #1011's example, sum without (instance), count_values, keyless SQL projections, UNION ALL, a keyless sub-plan shared across two SQL queries, and a > 10 filter over a shared without sum. This PR therefore:

• replaces the list with one rule stated on the IR;
• keeps an unordered LIMIT unshared, conservatively (a keyed one was shared before);
• fixes the memo on paths without keys (the test helpers, and any caller whose frontend output has no key).

Where the rule lives (#1011 checklist 1). It stays in asap-types, as a property of the IR operator. Any caller can ask it, and the memo and the DAG viewer can show why a node stayed separate.

What

• New NonASAPOp::is_shareable_when_equal(), which defines what "deterministic" means for sharing.
• CSE legality is now "an ASAP node, or a non-ASAP node for which this returns true". Schema::has_unique_key no longer gates sharing.
• Never shared:
  • a Limit or offset whose child is not a Sort with the same partitions;
  • a Dedup on some but not all columns;
  • a window function that reads row positions (ROW_NUMBER, LAG/LEAD, FIRST/LAST/NTH_VALUE, or an aggregate with a ROWS frame) without an ORDER BY.
• Every other non-ASAP node is shared when it is structurally equal.

How

Equal nodes are interned children first, so two equal nodes always read one input. When an operator's output depends only on its input rows, one copy serves every reader. Ties are the one place where input order decides the output. A Sort is stable, and exact top-k ranks with it, so that tie order counts as fixed. This keeps main's sharing of exact top-k (decided in review of #1011). An operator that picks rows by input order alone is never shared.

Key code interfaces added / modified

crates/types/src/ir/operator/non_asap.rs

impl NonASAPOp {
    /// Whether two structurally equal copies of this node may be one shared
    /// node: every execution over the same input rows returns the same output
    /// rows. Tie order of a stable Sort counts as fixed; picking rows by input
    /// order alone does not. Matches every variant explicitly, so a new
    /// operator must decide.
    pub fn is_shareable_when_equal(&self) -> bool; // new
}

crates/types/src/ir/cse.rs

/// Hash-conses the workload's roots; legality is now
/// `node.is_asap() || NonASAPOp::is_shareable_when_equal`.
pub fn share_common_sub_dags<Id>(
    roots: Vec<(Id, Rc<OperatorNode>)>,
) -> Vec<(Id, Rc<OperatorNode>)>; // unchanged signature; legality was unique key + hand-written list

crates/types/src/ir/schema/mod.rs

/// Whether this schema has any provable unique key. No longer the CSE gate.
pub fn has_unique_key(&self) -> bool; // unchanged signature; doc updated

Usage (from memo::tests::equal_keyless_deterministic_results_share_one_group, which runs at this commit):

// The user's query, every minute, max_error 0.01, failure_probability 0.001,
// lowered without source labels, so the per-series quantile has no unique key.
let roots = vec![(0, QueryRoot::Operator(lower_promql(
    "avg by (job) (quantile_over_time(0.5, data[5m]) / quantile_over_time(0.5, data[5m]) + quantile_over_time(0.9, data[5m]))",
    AccuracyTarget::max_error(0.01).failure_probability(0.001),
)))];
// Stage 0 output: share_common_sub_dags asks each node
// is_shareable_when_equal(); the quantile returns true.
let memo = LogicalMemo::from_roots(&roots);
// The `G2 / G2` group: the one BinaryOp dividing.
let division = memo.groups.iter().find(|group| matches!(group.node.non_asap(),
    Some(NonASAPOp::BinaryOp { operator, .. })
        if operator.kind == BinaryOpKind::Arithmetic(ArithmeticOpKind::Div))).unwrap();
// The division's two operands are one group (main: groups 2 and 3).
assert_eq!(division.children[0], division.children[1]);
// One fewer group than on main's test path (8).
assert_eq!(memo.groups.len(), 7);

Major algorithm changes

CSE legality (Stage 0 group identity).

• Problem: with the test helper lowering, q(0.5) / q(0.5) kept two groups. Each was planned and sized separately, with three budget shares instead of two. A keyed unordered LIMIT was shared.
• Before: share iff ASAP, or has_unique_key(), or one of five hand-listed shapes.
• After: share iff ASAP, or is_shareable_when_equal().
• Correctness: each reader of a shared node reads the rows it would read alone, because equal nodes have one input and the operator's output depends only on its input rows (stable tie order counts as fixed). Checked by cse::tests (unordered_limits_are_not_shared, limits_over_a_sort_with_their_partitions_are_shared, dedup_shares_only_on_every_column, without_grouping_is_shared) and by main's memo_enumeration, which measures the production memo and passes unchanged.
• Complexity: unchanged, O(1) per node on top of the structural hash. The memo can only get smaller. On the top-k snapshot workloads (test helper), 69 and 504 candidates, down from 313 and 994.
• Limits:
  • "Same rows" treats floating-point sums as equal regardless of summation order.
  • It assumes every scalar function the IR accepts is deterministic (unregistered SQL functions are rejected) and that AggIntent::Extension intents return the same rows.
  • No production-path workload I tried changes its plan (see Why).
  • Not addressed here: budget splitting counts a group once even when a query reads it twice (x + x), so the half-budget sketch read twice is never offered. That gap predates this PR (found in review); see the follow-up.

Before this PR

The CSE gate was a unique key plus a hand-written list, and a keyed unordered LIMIT could be shared.

After this PR

One rule on the IR decides group identity. planner-layering.md states it, including what "same rows" means. Comments and docs that described the unique-key gate are updated (pre-asap-ir.md, the memo example, memo/groups.rs, asap_aware_cse.rs, summary_inputs.rs).

Verification

Rebased onto main 61ee27f30; head d9821d272.

• Remote runner at the head: cargo fmt --check passed; clippy (-D warnings) passed on asap-types, asap-logical-asap-optimization and asap-planner.
• nextest at the head:
  • lib tests of asap-types, asap-logical-asap-optimization and asap-plan-selection: 822 passed;
  • asap-planner --test summary_sharing and asap-integration-tests --test memo_enumeration: all passed.
• memo_pricing.rs (from feat(devtools): stage_pipeline --price-enumerated prices each enumerated memo choice and flags any cheaper than the selected plan #1028), keyless ratio of quantiles: the memo now has 25 choices (9 whole-budget), not 343 (27). Its only valid DAG is the exact one. That DAG prices at 0.0037502/s with the series-identity column and 0.0032169/s without it, so the keyless price is no longer compared with the identity-lowered selection; that cross-lowering gap is PromQL planner test helpers skip the series-identity step production runs, so ~400 Stage 1–3 tests plan over a schema production never produces #1035. The test is renamed ratio_of_quantiles_25_pass1_choices_without_identity_only_exact_is_valid, and the now-unused assert_none_cheaper is removed. ratio_of_quantiles_no_listed_choice_is_cheaper_than_the_selected_plan keeps the "none cheaper" check on the production lowering; it passes, and so does all of memo_pricing (5/5).
• Updated tests: the ratio-operand test (the plain query's quantile is now the numerator's group), and the two top-k snapshot pins. The pins now also assert that the readers' sum_over_time is one group.
• Full workspace before the rebase (16f07f896): the only failures were tests that also fail on main at the base d728d05cf, or planning-time-budget tests that pass when run on their own.
• stage_pipeline, main vs this branch, on the 8 workloads in Why: identical groups and selected costs (Stage 0 review: frontends and the logical memo (hash-consed groups) #1011's example: 0.000470097/s on both).
• Independent review (asap-code-review) done. Its fixes are in 7f292b351; the chosen interface changes (rename, explicit match, Dedup length check) are in 18218ad06. Re-ran fmt, clippy and the 822 lib tests at 18218ad06: all passed.
• Screenshots: not applicable.

🤖 Generated with Claude Code

[zzylol]

Merging with test red by explicit decision. At head d9821d272, format-and-lint passed. test stops at planner_fuzz::finds_issue_902_filter_on_group_key_not_reading_the_grouped_result, which also fails on main (#992 updates it). The tests covering this PR's changes pass at this head:

• the lib tests of asap-types, asap-logical-asap-optimization and asap-plan-selection (822);
• memo_enumeration, summary_sharing and memo_pricing.

🤖 Generated with Claude Code