Skip to content

refactor(optimizer): move Stage 1 modules into pass1/ and pass2/ (#958 step 1) - #960

Merged
zzylol merged 1 commit into
mainfrom
refactor/958-step1-move
Oct 29, 2026
Merged

zzylol merged 1 commit into
mainfrom
refactor/958-step1-move

Conversation

@zzylol

@zzylol zzylol commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Part of #958 (step 1).

Why

Stage 1 (crates/logical-asap-optimization/src) has no stage structure. Pass 1 and Pass 2 modules sit side by side at the top level with accuracy helpers, and two different files are both named state_configurations.rs. #958 (decided 2026-10-27) reorganizes the crate by pass. Step 1 is the move alone, so later steps (the deployment-input split, the single memo-based Pass 2, Stage 1 finishing the memo) land in the right place and review as behavior changes only.

What

A move-only change, with no behavior change:

How

The commit is exactly this script's output on origin/main (97b192bf), followed by cargo fmt --all, then rebased without conflicts onto 37dacce2 (#956 touches no moved file). Compared with the first version, the script also rewrites relative doc paths (memo/..., memo.rs, bare memo:: paths) in docs/**/*.md.

958-step1-move.sh
#!/usr/bin/env bash
# #958 step 1: move-only reorganization of crates/logical-asap-optimization/src.
# Run from the repository root on a clean checkout of the base commit.
set -euo pipefail
S=crates/logical-asap-optimization/src
mkdir -p $S/pass1 $S/pass2
git mv $S/per_computation_candidates.rs $S/pass1/per_computation_candidates.rs
git mv $S/realization.rs                $S/pass1/realization.rs
git mv $S/function_rules.rs             $S/accuracy/function_rules.rs
git mv $S/asap_aware_cse.rs             $S/pass2/asap_aware_cse.rs
git mv $S/workload_candidates.rs        $S/pass2/workload_candidates.rs
git mv $S/memo.rs                       $S/pass2/memo.rs
git mv $S/memo                          $S/pass2/memo
git mv $S/pass2/memo/state_configurations.rs       $S/pass2/memo/shared_configurations.rs
git mv $S/pass2/memo/rules/state_configurations.rs $S/pass2/memo/rules/shared_configuration_readers.rs
# Present only once #952 has merged.
if [ -f $S/finer_aggregate.rs ]; then git mv $S/finer_aggregate.rs $S/pass2/finer_aggregate.rs; fi
# Module paths in every Rust file and doc path references.
python3 - <<'PY'
import pathlib, re
CRATE = "asap_logical_asap_optimization"
MOVED = {  # top-level module -> new parent
    "per_computation_candidates": "pass1", "realization": "pass1",
    "asap_aware_cse": "pass2", "workload_candidates": "pass2", "memo": "pass2",
    "finer_aggregate": "pass2",
}
names = "|".join(MOVED)
def fix_paths(text):
    # crate::memo, asap_logical_asap_optimization::memo, ... (not already moved)
    text = re.sub(r"\b(crate|%s)::(%s)\b" % (CRATE, names),
                  lambda m: f"{m[1]}::{MOVED[m[2]]}::{m[2]}", text)
    text = re.sub(r"\bcrate::function_rules\b", "crate::accuracy::function_rules", text)
    text = re.sub(r"\bmemo::state_configurations\b", "memo::shared_configurations", text)
    text = re.sub(r"\brules::state_configurations\b", "rules::shared_configuration_readers", text)
    # Any other `state_configurations::` path, including nested `memo::{…}` imports,
    # names the memo-level module (the rules-level one is private to `rules`).
    text = re.sub(r"(?<!rules::)\bstate_configurations::", "shared_configurations::", text)
    # Items of grouped imports: `asap_logical_asap_optimization::{ memo::X, ... }`.
    out, i = [], 0
    for m in re.finditer(r"\b%s::\{" % CRATE, text):
        start = m.end()
        depth, j = 1, start
        while depth:
            depth += {"{": 1, "}": -1}.get(text[j], 0)
            j += 1
        body = text[start:j - 1]
        res, d, k = [], 0, 0
        while k < len(body):
            c = body[k]
            if d == 0:
                mm = re.match(r"(%s)\b" % names, body[k:])
                prev = body[:k].rstrip()
                if mm and (prev == "" or prev.endswith(",")):
                    res.append(f"{MOVED[mm[1]]}::{mm[1]}")
                    k += len(mm[1])
                    continue
            d += {"{": 1, "}": -1}.get(c, 0)
            res.append(c)
            k += 1
        out.append(text[i:start]); out.append("".join(res)); i = j - 1
    out.append(text[i:])
    return "".join(out)
for path in pathlib.Path("crates").rglob("*.rs"):
    if "target" in path.parts:
        continue
    old = path.read_text()
    new = fix_paths(old)
    if new != old:
        path.write_text(new)
# Module declarations.
S = pathlib.Path("crates/logical-asap-optimization/src")
def edit(path, pairs):
    text = path.read_text()
    for old, new in pairs:
        assert old in text, (path, old)
        text = text.replace(old, new, 1)
    path.write_text(text)
edit(S / "pass2/memo.rs", [("pub mod state_configurations;", "pub mod shared_configurations;")])
edit(S / "pass2/memo/rules.rs", [("mod state_configurations;", "mod shared_configuration_readers;")])
edit(S / "accuracy/mod.rs", [("pub mod evidence;\n", "pub mod evidence;\npub(crate) mod function_rules;\n")])
lib = S / "lib.rs"
text = lib.read_text()
blocks = {}
for name in ["per_computation_candidates", "asap_aware_cse", "workload_candidates", "memo", "realization", "finer_aggregate"]:
    # The `mod` line with the doc comment above it.
    m = re.search(r"((?:^///.*\n)*)^(pub )?mod %s;\n" % name, text, re.M)
    if m:
        blocks[name] = m[0]
        text = text[:m.start()] + text[m.end():]
text = re.sub(r"^mod function_rules;\n", "", text, flags=re.M)
text = text.replace("pub use realization::summary_candidates;", "pub use pass1::realization::summary_candidates;")
text = re.sub(r"\[`(per_computation_candidates|realization)`\]", r"[`\1`](pass1::\1)", text)
text = re.sub(r"\[`(asap_aware_cse|workload_candidates)`\]", r"[`\1`](pass2::\1)", text)
text = re.sub(r"\n{3,}", "\n\n", text).rstrip("\n") + "\n\npub mod pass1;\npub mod pass2;\n"
lib.write_text(text)
def module(name, doc, order):
    body = doc + "\n" + "".join(blocks[n] if blocks[n].startswith(("pub", "///")) and "pub mod" in blocks[n]
                                  else blocks[n].replace("mod ", "pub(crate) mod ", 1)
                                  for n in order if n in blocks)
    (S / f"{name}.rs").write_text(body)
module("pass1", "//! Stage 1, Pass 1: per-computation candidate generation and composition,\n//! and the summary realizations it offers.\n",
       ["per_computation_candidates", "realization"])
module("pass2", "//! Stage 1, Pass 2: ASAP-aware sharing over the Pass 1 candidates, through the\n//! logical memo and its rules.\n",
       ["asap_aware_cse", "finer_aggregate", "workload_candidates", "memo"])
# File paths mentioned in docs and comments.
FILES = {
    "src/per_computation_candidates.rs": "src/pass1/per_computation_candidates.rs",
    "src/realization.rs": "src/pass1/realization.rs",
    "src/function_rules.rs": "src/accuracy/function_rules.rs",
    "src/asap_aware_cse.rs": "src/pass2/asap_aware_cse.rs",
    "src/workload_candidates.rs": "src/pass2/workload_candidates.rs",
    "src/memo/rules/state_configurations.rs": "src/pass2/memo/rules/shared_configuration_readers.rs",
    "src/memo/state_configurations.rs": "src/pass2/memo/shared_configurations.rs",
    "src/memo/": "src/pass2/memo/", "src/memo.rs": "src/pass2/memo.rs",
}
for path in list(pathlib.Path(".").rglob("*.md")) + list(pathlib.Path("crates").rglob("*.rs")):
    if "target" in path.parts or ".git" in path.parts:
        continue
    old = path.read_text()
    new = old
    for a, b in FILES.items():
        new = re.sub(r"logical-asap-optimization/" + re.escape(a), "logical-asap-optimization/" + b, new)
    new = new.replace("rules,state_groups,state_configurations,window_candidates",
                      "rules,state_groups,shared_configurations,window_candidates")
    if new != old:
        path.write_text(new)
PY
# Relative paths and bare module paths in docs (and one test comment).
python3 - <<'PY'
import pathlib, re
targets = list(pathlib.Path("docs").rglob("*.md")) + [pathlib.Path("crates/integration-tests/tests/memo_enumeration.rs")]
for path in targets:
    old = path.read_text()
    new = old
    new = new.replace("memo/rules/state_configurations.rs", "memo/rules/shared_configuration_readers.rs")
    new = new.replace("memo/state_configurations.rs", "memo/shared_configurations.rs")
    new = re.sub(r"(?<![\w/.-])memo/", "pass2/memo/", new)
    new = re.sub(r"(?<![\w/.-])memo\.rs", "pass2/memo.rs", new)
    new = re.sub(r"(?<![\w/.-])(per_computation_candidates|realization)\.rs", r"pass1/\1.rs", new)
    new = re.sub(r"(?<![\w/.-])(asap_aware_cse|workload_candidates|finer_aggregate)\.rs", r"pass2/\1.rs", new)
    new = re.sub(r"(?<![\w:])memo::(candidates|window_candidates|enumeration|state_groups|rules|groups)\b", r"pass2::memo::\1", new)
    if new != old:
        path.write_text(new)
PY

The script does four things:

  1. git mv the files;
  2. rewrite crate::X and asap_logical_asap_optimization::X paths, including items of grouped use …::{…} imports;
  3. move the mod declarations from lib.rs into the new pass1.rs and pass2.rs, with their doc comments;
  4. update file paths mentioned in docs and comments.

Key code interfaces added / modified

Module paths only. No item changes its name, signature or visibility, except that the crate-private function_rules becomes pub(crate) under accuracy.

Old path New path
asap_logical_asap_optimization::per_computation_candidates asap_logical_asap_optimization::pass1::per_computation_candidates
asap_logical_asap_optimization::realization asap_logical_asap_optimization::pass1::realization (summary_candidates stays re-exported at the crate root, unchanged)
asap_logical_asap_optimization::asap_aware_cse asap_logical_asap_optimization::pass2::asap_aware_cse
asap_logical_asap_optimization::workload_candidates asap_logical_asap_optimization::pass2::workload_candidates
asap_logical_asap_optimization::memo asap_logical_asap_optimization::pass2::memo
asap_logical_asap_optimization::memo::state_configurations (finite shared sizes and pane configurations) asap_logical_asap_optimization::pass2::memo::shared_configurations
memo::rules::state_configurations (private: expands readers at those configurations) pass2::memo::rules::shared_configuration_readers
crate-private function_rules accuracy::function_rules (pub(crate))

The new module files:

// crates/logical-asap-optimization/src/pass1.rs  (new)
/// Stage 1, Pass 1: per-computation candidate generation. …
pub mod per_computation_candidates;   // was crate::per_computation_candidates
pub mod realization;                  // was crate::realization

// crates/logical-asap-optimization/src/pass2.rs  (new)
/// Stage 1, Pass 2: ASAP-aware common-subexpression elimination …
pub mod asap_aware_cse;               // was crate::asap_aware_cse
pub mod workload_candidates;          // was crate::workload_candidates
/// Logical group identity available before Pass 1 candidate generation.
pub mod memo;                         // was crate::memo

The following stay where they are:

Usage: a caller that read asap_logical_asap_optimization::memo::state_groups::expand_concrete_summary_state_groups now reads asap_logical_asap_optimization::pass2::memo::state_groups::expand_concrete_summary_state_groups. Planning a workload end to end (asap_planner::e2e_plan, stage_pipeline) is unchanged.

Major algorithm changes

None.

Before this PR

All Stage 1 modules sat at the top level of the crate: per_computation_candidates.rs, realization.rs, function_rules.rs, asap_aware_cse.rs, workload_candidates.rs, memo.rs and memo/. memo/state_configurations.rs and memo/rules/state_configurations.rs held different things under one name.

After this PR

  • src/pass1/{per_computation_candidates, realization}.rs
  • src/pass2/{asap_aware_cse, workload_candidates}.rs
  • src/pass2/memo.rs and src/pass2/memo/…, with shared_configurations.rs and rules/shared_configuration_readers.rs
  • src/accuracy/function_rules.rs

Verification

Run on the regenerated commit (base 97b192bf), before the conflict-free rebase onto 37dacce2:

  • cargo build --workspace --all-targets: clean;

  • cargo test -p asap-logical-asap-optimization --lib: 235 passed;

  • cargo test -p asap-plan-selection --lib: 379 passed;

  • cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --all --check: clean.

  • git diff -M --stat origin/main: 111 files, every moved file a rename ({ => pass1}/…, { => pass2}/…, { => accuracy}/function_rules.rs). The other changed lines are new paths, imports, and cargo fmt reflows of lines whose path length changed.

  • A grep over crates, docs, tools and the root *.md files finds no old path (crate::memo, workload_candidates, asap_aware_cse, state_configurations, old src/… file paths).

🤖 Generated with Claude Code

Move only (#958 step 1): per_computation_candidates and realization to
pass1/, asap_aware_cse, workload_candidates and memo to pass2/,
function_rules to accuracy/, and the two state_configurations files
renamed by role (memo::shared_configurations,
memo::rules::shared_configuration_readers). Callers use the new paths;
no re-exports at the old ones. Generated by the move script in the PR
body, then cargo fmt.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@zzylol
zzylol force-pushed the refactor/958-step1-move branch from 53dad60 to 0584da4 Compare October 29, 2026 04:14
@zzylol
zzylol merged commit ecb360d into main Oct 29, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

optimizer Workload optimization and plan selection refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant