Skip to content

Polish batch: spend cap, starter form, compare, --follow, grouped outage alerts - #98

Merged
Adambomb210 merged 2 commits into
mainfrom
claude/polish-batch
Nov 19, 2026
Merged

Adambomb210 merged 2 commits into
mainfrom
claude/polish-batch

Conversation

@Adambomb210

Copy link
Copy Markdown
Collaborator

Builds the improvements list (2, 3, 5, 6, 8) and the earlier polish list.

Members

  • Spend cap per job: the form's "Stop once it has cost ($)", max_spend_cents in the API, and --max-spend in the CLI (job files: max-spend).
    • Once a job's accrued spend reaches the cap, the scheduler fails the job and destroys its machine. A box is stopped instead.
    • The CLI cancels the job when the Krater it's talking to doesn't know the field, so the cap is never silently dropped (production doesn't know it until this ships).
  • Try the starter job on the new-job form: puts examples/starter/train.py in the project's inputs as starter/, then fills in the form.
  • krater submit --follow: prints the output until the job finishes. If it succeeded, it downloads the outputs here. The exit code is the same as krater wait's.
  • krater compare <job>...: each job's outputs/metrics.json side by side.
  • krater logs says when the newest attempt has no output yet; the logs API now includes latest_attempt.
  • krater run gives one plain line when it can't read the project's secret names (signed out or offline).
  • Job page: "Last checkpoint stored", with an overdue badge when nothing has arrived for twice the upload interval.
  • Pause wording: now "almost always because it was outbid", instead of stating outbid as fact.
  • CLI 0.7.0.

Ops

  • code/<hash>/ input folders are deleted after 7 days once no unfinished job uses them (in the hourly compute_storage_cleanup). A rerun of a pruned version is refused at submit, as for any missing input.
  • When two or more ops alerts open at once and each says DNS failed or the network is unreachable, they go out as one Slack post.
  • /healthz includes "revision": the image's KRATER_REVISION build arg, set by CI.
  • The Deploy workflow is split into pick and deploy, so a run for a commit that is no longer main's tip shows skipped instead of a green success that deployed nothing.
  • CLI tests now share a conftest.py that keeps sign-ins out of the real config folder. test_updates_cli and the new tests had been writing fake tokens to %APPDATA%\krater.

Migration e2b7c4d9a1f6 adds the nullable compute_jobs.max_spend_cents and nothing else.

Tests: the server suite and the CLI suite (242) pass. The new tests cover the spend cap (scheduler, API, form), the starter, checkpoint freshness, code-folder pruning, the grouped outage alert, --follow, compare, the logs note, and run when signed out.

🤖 Generated with Claude Code

Adambomb210 and others added 2 commits November 19, 2026 17:15
…age alerts

- Per-job spend cap (max_spend_cents; form "Stop once it has cost", API, CLI --max-spend):
  the scheduler fails the job (or stops a box) once its accrued spend reaches it.
- Web "Try the starter job": puts examples/starter/train.py in the project's inputs and
  fills in the form.
- krater submit --follow (logs, then outputs downloaded on success, wait's exit code) and
  krater compare <jobs> (outputs/metrics.json side by side). CLI 0.7.0.
- krater logs says when the newest attempt has no output yet (logs API: latest_attempt);
  krater run explains plainly when it can't read secret names.
- Job page: when a checkpoint last reached storage, and an overdue badge.
- Old code/<hash>/ input folders no unfinished job uses are pruned after 7 days.
- Ops alerts that all say this server's DNS/network is down are posted as one.
- /healthz reports the deployed commit (KRATER_REVISION from CI); the Deploy workflow
  skips its deploy job instead of reporting success for a commit it didn't deploy.
- Pause wording no longer assumes outbid outright.
- CLI tests: a shared conftest keeps every test's sign-in out of the real config folder.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…PI test sees latest_attempt

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@Adambomb210
Adambomb210 merged commit 98b3c70 into main Nov 19, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant