Skip to content

Add a multi-agent version of the pruning loop - #2

Merged
mmshad merged 17 commits into
mainfrom
multi-agent-loop
Oct 24, 2026
Merged

Add a multi-agent version of the pruning loop#2
mmshad merged 17 commits into
mainfrom
multi-agent-loop

Conversation

@mmshad

@mmshad mmshad commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Adds a multi-agent version of the pruning loop, next to the single-agent loop.

  • Four roles run as read-only claude -p steps: a planner, a critic, a writer, and an auditor. experiment_multi.yaml sets the model and the effort level of each role, and the limits.
  • The rounds of each wave run as one SLURM array job on A100 MIG slices.
  • The final steps add two confirmation seeds and the 3-seed means. The auditor and check_report check the report against the records.
  • The single-agent loop gets the model and effort settings, the model check, a deny rule for the README, an optional GPU constraint, and fixes for stopped and restarted runs. Its commands stay the same.
  • The README describes the multi-agent loop and compares three designs. No result holds, so the comparison shows no gain from the extra agents.
  • 106 unit tests pass.
  • The README links to the handbook chapter Multi-Agent Workflow Example, which is not live yet. A separate handbook PR adds it and updates End-to-End Agentic Workflow Example. Merge the two PRs together.

Closes #1

mmshad added 14 commits October 21, 2026 19:12
@mmshad
Add model and effort settings, array jobs, and seed records
7b1ca2d
@mmshad
Add the multi-agent loop
0e488dd
@mmshad
Describe the multi-agent version in the README
7761aa0
@mmshad
Fix the findings of the code review
eebb4a0
@mmshad
Set the version to 0.2.0
458a949
@mmshad
Fix the findings of the cluster tests
2c17f49 
- Run the round array jobs on A100 MIG slices, with 150 GPU minutes.
- Keep 450,000 of 1,250,000 tokens for the writer and the auditor.
- Read the baseline metrics in check_report.
- Count the GPU minutes of jobs that a stopped loop job canceled.
@mmshad
Add an optional GPU constraint for the round jobs
704a049
@mmshad
Link the README to the multi-agent chapter
7f8d921
@mmshad
Shorten the multi-agent section of the README
3f9cdf2
@mmshad
Refuse a new search after the final steps
ffd0d04 
- A loop or plan-only job stops in a state folder whose final steps are done.
- The plan-only job says when a role step failed.
@mmshad
Fix the README findings of the chapter review
19f307e
@mmshad
Use one bullet style in Adapt the example
5a5ccf4
@mmshad
Add the results of the comparison runs
b9d2eb7
@mmshad
Plan for 10% more tokens
48590ed
@mmshad mmshad added documentation Improvements or additions to documentation enhancement New feature or request labels Oct 23, 2026
@mmshad mmshad self-assigned this Oct 23, 2026
@mmshad
mmshad merged commit a50c496 into main Oct 24, 2026
2 checks passed
@mmshad
mmshad deleted the multi-agent-loop branch October 24, 2026 05:58
@mmshad mmshad mentioned this pull request Oct 24, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@mmshad mmshad

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature]: Add a multi-agent version of the pruning loop

1 participant

@mmshad