Skip to content

review-runs-tracking: 2026-10 #881

Description

@dormouse-bot

Monthly tracking issue for below-threshold findings. Each run appends findings as a comment. Future runs read these to build cumulative evidence.

Do not close manually — a new issue is created each month, and prior months are closed automatically.

Activity

  1. dormouse-bot commented on Oct 1, 2026

    @dormouse-bot
    CollaboratorAuthor

    Run 36879007937 — 2026-10-01T14:57:58Z (review-runs)

    Window 2026-09-29T14:21:40Z → run start (~48.5 h, anchored on predecessor 36582088435). The 09-30 sweep 36727612071 still reads queued and never ran. That is GitHub bookkeeping: today's run started normally, so nothing is holding the group. 298 tend runs censused (not a page boundary): tend-mention 156 (128 success, 27 skipped, 1 cancelled), tend-review 56 (54 success, 2 failure), tend-ci-fix 38 (1 success, 37 skipped), tend-mention-relay 34, tend-notifications 10, tend-triage 2, and 1 each of nightly and review-runs. Near-timeout: 2. Both failures ran 351 min against the 360-min default cap (no timeout-minutes anywhere). Nightly took 27 min. Token report: $70.71. tend-review $47.10 over 55 runs, nightly $14.36, notifications $5.28, review-runs $2.64, mention $0.88, triage $0.45. 211 runs had no readable artifact (relay and skipped runs).

    GATE-PASSING, acted on: tend-review polls hung 351 min on environment-gated Hosted PR preview jobs. Overlay skip in #882

    workflow-audit #880 re-listed 24 already-accounted commits from the same stale bound, 2026-09-03T12:03:48Z. Occurrence 2

    Outage tracker drained

    • Bot temporarily unavailable #871 held 2 rows, the two hung reviews above. I diagnosed them, commented, and closed the tracker. There was nothing to retry: both PRs merged with green CI.

    Pre-registered watches

    • tend-notifications under-delivery, occurrence 25: 10 runs in about 48 h against */15. No lever.
    • tend-mention fan-out: 156 dispatches, 1 cancellation, $0.88 across the 4 sessions that had logs. Cheap.
    • tend-weekly cron move: first scheduled Sunday 15:17 slot is 10-04. Not yet observed.
    • No local dependencies in sessions (Medium structural, 09-29): not re-counted this run.

    Maintainer corrections

    Live scan

    Run 37017894064 — 2026-10-02T14:21:38Z (review-runs)

    Window 2026-10-01T14:46:38Z → run start (~23.5 h, anchored on predecessor 36879007937). 572 tend runs censused (not a page boundary): tend-mention 317 (294 success, 15 skipped, 8 cancelled), tend-review 118 (117 success, 1 cancelled), tend-mention-relay 94, tend-ci-fix 30 (all skipped), tend-triage 6, tend-notifications 4, tend-nightly 2 (1 failure), review-runs 1. The volume came from the maintainer's spec-audit stack (#885–#907) and the Hosted stack (#867–#872). Near-timeout: 1 (the nightly failure below). Nothing else ran past 25 min. Token report: $115.74. tend-review $76.85 over 118 runs, nightly $21.88, mention $8.34, triage $3.62, review-runs $3.48, notifications $1.57.

    Nightly 36867257260 killed at 351 min. Same Hosted-preview poll hang, already fixed by #882

    workflow-audit stale bound: occurrence 3, deterministic. Acted on: #920

    Live-work handling: I opened a fix PR that conflicted with the maintainer's in-flight PR, then withdrew it

    Outage tracker drained

    Pre-registered watches

    • tend-notifications under-delivery, occurrence 26: 4 runs in ~23.5 h against */15. None ran after 08:17Z, so security-audit FAIL [security-audit] FAIL on 2026-10-04 #908 (10:38Z) went without a bot response until this sweep. No lever.
    • tend-mention fan-out: 317 dispatches, 8 cancellations, $8.34 over 26 sessions. Driven by the maintainer's @dormouse-bot review requests across the two stacks. Cheap per run.
    • tend-weekly cron move: first slot 10-04. Not yet observed.

    Maintainer corrections

    Live scan

    Run 37123854067 — 2026-10-03T12:54:00Z (review-runs)

    Window 2026-10-02T14:09:27Z → run start (~22.5 h). 665 tend runs censused (not a page boundary): tend-mention 343 (322 success, 19 skipped, 2 cancelled), tend-review 137 (all success), tend-mention-relay 88, tend-ci-fix 88 (1 success, 87 skipped), tend-notifications 5, tend-triage 2, nightly 1, review-runs 1. No long-running or near-timeout jobs: the longest was triage at 28 min wall-clock. The ~16-min tend-mention runs at 23:37–23:45Z were runner queue time: their handle jobs ran for seconds or were skipped. Both cancellations (37115523082, 37115527213) were concurrency supersession from four quick inline replies on #957. Token report: $130.75. tend-review $103.02 over 137 runs (the highest was #957 at $6.31, 33 diagrams across 24 specs with 5 verify agents, and all four findings were applied by the maintainer), nightly $16.69, review-runs $3.23, triage $2.64, mention $2.40, notifications $2.11, ci-fix $0.66.

    Sessions sampled: nightly and the most expensive review were both sound

    workflow-audit stale bound: fix #920 verified in production

    Pre-registered watches

    • tend-notifications under-delivery, occurrence 27: 5 runs in about 22.5 h against */15, none after 08:07Z. As a result the 10-03 security-audit FAIL on [security-audit] FAIL on 2026-10-04 #908 (10:02Z) had no bot response until this sweep, which is the second day running for the same issue. No lever.
    • tend-mention fan-out: 343 dispatches, 2 cancellations, $2.40 over 11 sessions with logs. Cheap.
    • tend-weekly cron move: first slot is 10-04. Not yet observed.

    Maintainer corrections

    Live scan

    Run 37205764571 — 2026-10-04T13:36:03Z (review-runs)

    Window 2026-10-03T12:44:37Z → run start. 445 tend runs censused (not a page boundary): tend-mention 236 (220 success, 13 skipped, 3 cancelled), tend-review 79 (all success), tend-mention-relay 76, tend-ci-fix 45 (2 success, 43 skipped), tend-notifications 6, nightly 1, triage 1, review-runs 1. No failures, no long-running or near-timeout jobs: longest was ci-fix 37129084869 at 27 min (200/400-run local flake loops; no workflow sets timeout-minutes, so the 360-min default applies). Token report: $86.10 — tend-review $61.96 over 79 runs (top #987 $4.73), nightly $8.33, mention $6.94 over 22 sessions, ci-fix $4.21, review-runs $2.13, notifications $1.52, triage $1.01.

    Sessions sampled: all sound

    Pre-registered watches

    • tend-notifications under-delivery, occurrence 28: 6 runs in ~23 h against */15, none after 08:47Z. The 10-04 security-audit FAIL on [security-audit] FAIL on 2026-10-04 #908 (10:57Z) had no bot response until this sweep — third day running. No lever.
    • tend-mention fan-out: 236 dispatches, 3 cancellations, $6.94. Cheap.
    • tend-weekly cron move: slot is Sunday 15:17Z, after this sweep. Not yet observed.

    Maintainer corrections

    Live scan

    Run 37340496763 — 2026-10-05T16:29:20Z (review-runs)

    Window 2026-10-04T13:28:54Z → run start. 73 tend runs censused (not a page boundary): tend-mention 39 (34 success, 5 skipped), tend-review 13, tend-mention-relay 11, tend-notifications 6 (5 success, 1 failure), nightly 1, weekly 1, review-runs 1, triage 1 (skipped). No long-running or near-timeout jobs: longest was nightly 37326344621 at 26 min (360-min default cap). Token report: $24.57: nightly $10.72, tend-review $9.46 over 13 runs (top #1009 $1.46), review-runs $1.81, notifications $1.71, mention $0.63 (2 sessions), and weekly $0.24.

    Failure: tend-notifications 37311191249 was an Anthropic 529 Overloaded on the first turn

    Sessions sampled: all sound

    Pre-registered watches

    • tend-notifications under-delivery, occurrence 29: 6 runs in ~24 h against */15 (one failed with a 529). No lever.
    • tend-mention fan-out: 39 dispatches and 2 sessions, $0.63. Cheap.
    • tend-weekly cron move: discharged. The Sunday 17 15 slot started at 18:39Z (3 h 22 m GitHub delay), still after the quota reset, and the run succeeded. The Renovate PRs that opened on Monday will reach next week's run; tend-review already reviewed them.

    Maintainer corrections

    Live scan

    Run 37479274755 — 2026-10-06T14:34:09Z (review-runs)

    Window 2026-10-05T16:23:46Z → run start. 59 tend runs censused (not a page boundary): tend-mention 34 (27 success, 6 skipped, 1 failure), tend-review 11 (9 success, 2 failure), tend-mention-relay 9, tend-notifications 3 (2 success, 1 failure), nightly 1, review-runs 1. Every tend-mention session ended at verify (no handle job ran). No long-running or near-timeout jobs: the longest was nightly 37469699567 at 28 min (360-min default cap). Token report: $15.86: nightly $7.28, tend-review $5.46 over 9 runs (top #1024 $0.89), notifications $1.84, review-runs $1.28.

    All four failures were GitHub runner starvation at 20:08–20:47Z on 10-05

    tend-notifications approved #1021 over CI that never ran; I reran it

    • Evidence level: Low (designed). Occurrence 1.
    • Run ID: 37400221926
    • Detail: poll_pr_checks.py approval 1021 returned approve and listed the 5 cancelled jobs as unverified. That is upstream's intended treatment of CANCELLED since tend#1323. The review body disclosed that CI never verified the head. The session's own narration said "checking the approval rules before rerunning its cancelled CI", but it never reran, so the APPROVE stood over an untested head. This sweep ran gh run rerun --failed on CI 37367920310 and Hosted preview 37367920325. Not a bundled defect. A second occurrence of an approval standing over a never-rerun runner-starved CI would justify an overlay line telling the reviewer to rerun it.

    Sessions sampled: all sound

    Pre-registered watches

    • tend-notifications under-delivery, occurrence 30: 3 runs in about 22 h against */15, one of them runner-starved. No lever.
    • tend-mention fan-out: 34 dispatches, none reached handle, $0. Cheap.

    Maintainer corrections

    Live scan

    Run 37639349476 — 2026-10-07T14:53:01Z

    Window 2026-10-06T14:28:36Z → run start (~24 h). 127 tend runs censused (not a page boundary): tend-mention 59 (49 success, 10 skipped — every handle job skipped at the verify gate, no session), tend-ci-fix 33 (4 success, 29 skipped), tend-review 19, tend-mention-relay 9, tend-notifications 5 (3 pre-boot exits with no unread notifications), nightly 1, review-runs 1. Zero failures, zero cancellations, no job ≥20 min. Token report: $23.37 over 27 sessions — tend-review $11.04 / 19, nightly $6.74, notifications $2.14, ci-fix $2.10, review-runs $1.35.

    Maintainer corrections

    The Step 4 script ran and returned 7 dispositions (all merges: #985, #1005, #1006, #1015, #1018, #1019, #1025), 0 reviews, and 13 comment rows, all cloudflare-workers-and-pages deploy tables or a Renovate edited/blocked notice. No maintainer corrections.

    Live scan

    ci-fix sessions on maintainer-cancelled runs — designed, no action

    37551824621 and 37551864301 ($0.65 total) opened sessions on CI runs nedtwigg cancelled while batch-merging Renovate PRs; both correctly concluded "deliberate cancel, nothing failed" and posted nothing. tend 0.2.x fires ci-fix on cancelled too (recorded 2026-09-10). Designed no-op.

    Unguarded PATCH after a failed body edit — occurrence 1, Low, no action

    Healthy signals worth keeping in the record

    Run 37796665900 — 2026-10-08T15:06:33Z

    Window 2026-10-07T14:45:13Z → run start (~24 h). 512 tend runs censused (not a page boundary): tend-mention 273 (246 success, 24 skipped, 3 cancelled by concurrency), tend-review 91, tend-mention-relay 85, tend-ci-fix 54 (4 success, 50 skipped), tend-notifications 4, tend-triage 3, nightly 1, review-runs 1. Zero failures. The longest jobs were tend-review 37699124077 (#1048, 34 min) and 37671569711 (#1031, 31 min). Both spent the time polling CI; #1048's included a rerun after an apt-mirror failure. The cap is 360 min with no override, so no job came near it. Token report: $89.53 over 132 sessions, tend-review $60.67 / 91, nightly $11.76, mention $9.47 / 30, ci-fix $3.96, review-runs $1.79, notifications $1.38, triage $0.50. The cost is high because maintainer PR throughput was high (#1031–#1068). No subject is an outlier; the most expensive is #1057 at $5.56 over 10 runs.

    ci-fix opened a hardening PR for a one-off apt-mirror stall, and the maintainer closed it as not worth the complexity. Occurrence 1, Medium, stochastic, record only

    • Evidence level: Medium
    • Occurrences this run: 1 (none found in the 2026-09/10 evidence)
    • Run ID: 37678040336
    • Workflow: https://github.com/diffplug/dormouse/actions/runs/37678040336
    • Session: eafd8be3-8287-4fe9-abb6-f53433bccc77.jsonl
    • Detail: An Ubuntu mirror stall made Argos 37674202964 on main hit its 30-min cap and be cancelled. ci-fix opened ci(argos): bound and retry the browser system-dependency apt install #1035, which added bounded, retried install-deps to argos.yml. Its verification was careful: two dispatched branch runs, both green. nedtwigg closed it: "hasn't often been a problem. Not worth the complexity right now". The same day, ci-fix handled the same mirror stalls on Standalone Smoketest as transient failure trackers, which is the cheaper path. Watch for a second ci-fix PR that hardens against a one-off infra transient. Five or more occurrences would justify an overlay line ("a single infra transient gets a tracker, not a workflow change").

    Maintainer clarified a draft-review point on #996. Low, record only

    nedtwigg replied that the page "presents the plan as live on purpose" because #996 heads a stack (#999–#1003) that builds it. The review read #996 in isolation. That is the same shape as the earlier "closed on knowledge the bot could not have had" entry, and it is not a review defect. The other review findings on #996 were applied.

    Maintainer corrections

    The Step 4 script ran and returned 4 dispositions, 75 comment rows and 0 reviews. #1028, #1029 and #1030 merged. #1046 (the bot's spec-budget fix for main) was closed without comment because nedtwigg's own merge 39702ced7 trimmed the spec under budget, so it was superseded rather than corrected. #1035 is covered above. The only correction of a bot claim is #996. Every other non-deploy comment row is a maintainer accepting a review finding ("Applied", "Fixed in", "Agreed"), about 25 in all.

    Live work

    Healthy signals

    Run 37946019792 — 2026-10-09T14:50:24Z

    Window 2026-10-08T14:56:54Z → run start (~24 h). 144 tend runs censused (not a page boundary): tend-mention 80 (65 success, 15 skipped), tend-review 28, tend-ci-fix 14 (all skipped), tend-mention-relay 11, tend-triage 5, tend-notifications 4, nightly 1, review-runs 1. Zero failures. Longest job: nightly 37935723024 at 19 min; the cap is 360 min (no override). Token report: $40.01 over 40 sessions — tend-review $23.28 / 28, nightly $9.07, triage $3.09 / 5, review-runs $2.57, notifications $1.30, mention $0.70 / 2. Outlier: review of #1085 at $6.05 in one run (a 100+ file draft); sampled below and sound.

    Live work: zizmor impostor-commit failure on the dtolnay/rust-toolchain pin. Acted on: #1088

    Nightly declined the re-pin on a cooldown that does not cover action pins. Occurrence 1, Medium, stochastic, record only

    Sessions sampled: sound

    Maintainer corrections

    Step 4 script ran: 4 dispositions (#1070, #1071, #1073, #1077 merged — the bot's rustls bump #1073 landed), 12 comment rows, 0 reviews. No maintainer corrections. nedtwigg closed #603 as superseded by the implemented history sweep (not a correction of a bot claim); on #1081 they applied a bot finding.

    Live scan

    Bot PRs #1086, #1087 (green but Workflow Lint, fixed by #1088), each reviewed at their head. Triage on #1074–#1079 each answered. Red default branch: no candidates (back to 2026-09-23T19:09Z). status=waiting: all hosted-preview.yml/hosted-production.yml, maintainer-gated. Dependabot #1 glib still tracked by #663 (#5 rustls closed by #1073). Outage trackers: none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    review-runs-trackingMonthly tracking issue for review-runs below-threshold findings

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions