Skip to content

DDIR: mechanical tidies (stale code and docs, one-off results, console.html) - #1011

Merged
frankmcsherry merged 10 commits into
TimelyDataflow:master-nextfrom
frankmcsherry:interactive-tidy
Oct 24, 2026
Merged

frankmcsherry merged 10 commits into
TimelyDataflow:master-nextfrom
frankmcsherry:interactive-tidy

Conversation

@frankmcsherry

Copy link
Copy Markdown
Member

Mechanical tidies from the read-only map of interactive/. Nothing here changes behaviour. 10 commits, 28 files, +75/−1,478.

Deleted

  • ir::Id, which nothing referenced.

  • server/console.html, 687 lines, untouched since August and untested. It's retired for now; a console can be rebuilt on newer mechanisms, such as the diag.* traces.

  • examples/import_bench.rs and examples/triangles_bench.rs.

  • Leftover DDIR_DIAG_PORT settings, in the two-session demo, bench_txn.py and server/tests/multiworker.rs.

  • One-off recorded results, removed because anything worth knowing should be reproducible:

    • bench/ldbc/observations/sf1-memory-2026-10-21.{csv,json};
    • baselines/RESULTS.md and results-mini03-sf1.csv;
    • the tables of a single run in the LDBC MEMORY.md, UPDATES.md and METHOD.md.

    Method docs, bindings, tiny.json and every script stay.

Fixed

  • Stale comments:
    • lib.rs on the retired folded module;
    • explain/mod.rs:412;
    • backend/corgi.rs on VecBackend;
    • projection.rs:1-3, which pointed at an unmerged example.
  • examples/server/sessions/generated.txt: a generated source is now reclaimed when its last importer drops, so the manual drop is gone.

Rewritten: the README's Architecture section.

  • It covers language, analyses, runtime and host.
  • It states that the stack aims to stay generic over the expression language and the backend: corgi is today's expression language, and the columnar runtime is today's backend, behind Backend.

Moved: lib.rs is now just its module list.

  • hash_u64 and gen_field_seeded move to the server's generated-source code, where they're used.
  • load_program is replaced by read_to_string at its four call sites.

Kept, with a reason: tests/explain.rs's SCC_ROW copy now says why it differs from examples/programs/scc.ddp: its tests key edges by one-field tuples.

CI: the release job now runs bench/ldbc/authored/check.py --fixture. It checks that the 41 authored plans give the generated plans' answers.

Checks:

  • cargo test -p interactive -p ddir-server: 199 passed.
  • Clippy reports nothing on added lines.
  • All 16 LDBC CI steps pass, including the new check.py step.
  • check.py: 41/41.
  • Release explain suite: 29/29.
  • examples/aoc2023/run.sh: 33/33.
  • server/demo/two_sessions.py passes.

🤖 Generated with Claude Code

https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o

frankmcsherry and others added 10 commits October 23, 2026 20:10
- ir::Id was unreferenced.
- lib.rs and explain referred to the retired `folded` module.
- backend/corgi.rs said CorgiBackend mirrors VecBackend, which is deleted.
- projection.rs pointed at examples/self_opt, which isn't merged.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
The section described a `vec` row backend that is deleted, and didn't
mention shapes, projection or diagnostics. It now walks the four parts
(language, analyses, runtime, host), and says where an expression
language and a backend plug in.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
- hash_u64 and gen_field_seeded move beside the `random:` generated
  source in server.rs, their one library user; a test and an example
  name them as interactive::server::.
- load_program, a read_to_string wrapper, is inlined at its four uses.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
…r drops

generated.txt dropped the source by hand after its consumers, which now
fails with "no program": the server reclaims it when the last importer
drops. The session and the server examples README say so.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
Its tests query result edges by one-field keys, where scc.ddp keys bare
integers and counts its result, so an include_str! would need edits
throughout. The comment now says so.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
The release job now runs authored/check.py on the fixture graph at four
workers, once each, so the 41 hand-written plans can't drift from the
generated plans as the language and runtime change.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
The browser console was last changed in August, isn't run or tested, and
has likely drifted from the protocol. A future console would read the
diag.* collections and the current protocol, so it would be rebuilt
rather than kept up to date. The server README no longer points to it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
TimelyDataflow#1009 replaced the WebSocket diagnostics with the diag.* collections, so
the server no longer reads DDIR_DIAG_PORT. The two-session demo, the txn
bench and the multi-worker test still set it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
Recorded results with no lineage go; anything here should be
reproducible from the scripts and their method docs.
- observations/sf1-memory-2026-10-21.{csv,json}: one profiling run.
- baselines/RESULTS.md and results-mini03-sf1.csv: one SQL comparison run.
- MEMORY.md keeps how profile_memory.py measures and how to run it; its
  table of one run's observations, and the notes on that run's plans and
  binary, go.
- UPDATES.md drops a note about one revision's SF1 run; METHOD.md no
  longer points to RESULTS.md.

Kept: the bindings (sf1-memory.json and the rest), tiny.json, METHOD.md,
and every script.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
examples/import_bench.rs and examples/triangles_bench.rs were one-off
Rust benches over the library, built by CI but run by hand. Frank: "Those
two Rust programs can probably go." With import_bench gone, hash_u64's
only user is gen_field_seeded, so it is private to server.rs.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
@frankmcsherry
frankmcsherry merged commit ae17dd9 into TimelyDataflow:master-next Oct 24, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant