fix(asap-tools): give sketchdb its own SQL in ClickHouse runs

[milindsrivastava1997]

Part of #874 (1 of 3; #900 and #901 stack on this).

Problem: in a ClickHouse run, sketchdb always ran baseline's sql_file. For sketch-bench tables that is ClickHouse-only SQL (quantileExactInclusive, uniqExact, ...), while ASAPQuery needs the table's own SQL (asapquery.sql), which exists only for structured entries. Three things block giving sketchdb its own SQL:

• results pair across modes by global query index, so a sketchdb arm with fewer queries would compare the wrong queries;
• a saved SQL plan (asap-planner --planner saved) stores each query on NOW(), while the client config holds a templated time literal and a -- description line, so copy_prebuilt_controller_output's text check rejects it;
• calculate_fidelity indexes the estimate's results for every baseline query.

Fix:

• _mode_sql_files replaces _resolve_sql_file. sketchdb runs its own sql_file entry and skips groups without one, but only trailing groups (otherwise it raises), so its query indices are a prefix of baseline's. Every other mode still needs an entry in every group.
• The SQL planner input plans sketchdb's SQL, the text the engine receives.
• Validation also checks sketchdb's SQL files exist.
• A mode's SQL file must hold as many statements as baseline's for the same group, or the run fails: a mismatch would shift every later query index. A plain-string sql_file fails with a clear error instead of an AttributeError.
• The SQL plan check compares queries without comment lines or a trailing ;, with quoted templated time literals read as NOW().
• calculate_fidelity scores only the queries the estimate mode ran; a gap among them still fails.

Behaviour change: a hand-written baseline + sketchdb run must list sketchdb: in sql_file (e.g. the same file as baseline for ClickBench). No checked-in config uses sql_file; the docs' examples are updated (they still showed the old plain-string form).

Tests: new tests/test_clickhouse_client_configs.py; additions to test_prebuilt_controller_output.py, test_sql_planner_input.py, test_calculate_fidelity.py, test_experiment_run_clickhouse.py.

🤖 Generated with Claude Code

[milindsrivastava1997]

Code review findings (Claude Code).

[milindsrivastava1997]

_mode_sql_files checks that sketchdb's groups are a prefix of baseline's, but not that each sketchdb sql_file has as many statements as baseline's file for the same group. Results are paired by global query index, so a count mismatch shifts every later index.

Example: group 0 has baseline {q0,q1,q2} and sketchdb {q0,q2}, and group 1 is the same for both. Sketchdb's query 1 is then scored against baseline's q1, and all of its group-1 queries against the wrong baseline queries. calculate_fidelity truncates to len(estimate_results) and prints scores with no error. Suggest checking the statement counts per group here.

[milindsrivastava1997]

Fixed in bbcdac9: _mode_sql_files now fails when a mode's SQL file holds a different number of statements than baseline's for the same group (test: test_statement_count_mismatch_is_refused).

[milindsrivastava1997]

This truncation assumes the estimate mode ran a prefix of baseline's queries, but nothing ties the scored queries to the groups the mode actually ran; pairing goes only by count. A misconfigured sql_file or a statement-count mismatch is scored against the first N baseline labels, and the only sign is the 'Scoring the first N of M' message.

Both client configs keep 'id': idx, so pairing by group id plus per-group query index would remove the need for both the prefix-only rule and this truncation.

[milindsrivastava1997]

Declined for now. With the prefix rule and the per-group statement-count check (bbcdac9), the estimate's queries are exactly baseline's first N, so truncating is safe. A gap among them still fails on the lookup. Pairing by (group id, index) would change the client output and every analysis script that indexes by position.

[milindsrivastava1997]

_comparable_sql rewrites every quoted templated literal to NOW(), so queries that differ only in their time literals compare equal. Examples: SQL floored to different intervals ('{{ end_datetime[:17] }}...' vs '{{ end_datetime[:14] }}...:00'), or BETWEEN '{{start}}' AND '{{end}}' against a plan that uses NOW() for both bounds.

copy_prebuilt_controller_output can then accept a saved plan for different windows, and the mismatch shows up only at query time.

[milindsrivastava1997]

Known limitation, kept as is. The saved plan holds NOW(), so the flooring interval can't be recovered from its text. The ponytail: comment on _comparable_sql notes this. The check catches a wrong plan; the engine matches queries to the plan structurally at query time.

[milindsrivastava1997]

The template body below (~line 177) still shows sql_file: ??? # path to SQL file as a plain string. _mode_sql_files calls sql_file.get(mode) on it and fails with AttributeError: 'str' object has no attribute 'get' instead of a clear error. The checked-in clickhouse.yaml and quantile_demo.yaml also use a string sql_file. Suggest updating the template to the dict form, and either rejecting a string with a clear message or accepting it as {baseline: path}.

[milindsrivastava1997]

Fixed in bbcdac9: a plain-string sql_file now fails with a clear error (test: test_plain_string_sql_file_is_refused_clearly), and the docs template uses the dict form. No checked-in YAML sets sql_file, and there is no clickhouse.yaml or quantile_demo.yaml in the repo.

[milindsrivastava1997]

This passes only {baseline:...}. Under the new rule, any sketchdb mode in experiment_params.experiment (as in the repo's clickhouse.yaml) needs its own entry, so this command now fails with 'No query group has a sql_file entry for mode sketchdb'. Before, sketchdb fell back to baseline's SQL.

[milindsrivastava1997]

That example runs baseline only, so it still works. bbcdac9 adds a note below it: with a sketchdb mode, give sketchdb its own entry, e.g. sql_file={baseline:...,sketchdb:...}.