Skip to content

Keep the collector out of the linear-scan test's clock - #1429

Closed
MaxGhenis wants to merge 1 commit into
mainfrom
nzhub/linear-scan-clock
Closed

MaxGhenis wants to merge 1 commit into
mainfrom
nzhub/linear-scan-clock

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Unblocks #1427, whose engine-free (3.13) check keeps failing on test_a_long_string_is_scanned_in_linear_time, though no scan changed.

Cause

The test held blocks of texts to a shared 2 s of CPU time. Two things spent that budget without any scan getting slower, and a one-job diagnostic on CI measured both. The diagnostic ran the same pytest flags on #1427 merged with main, which failed, and on main, which passed.

  • One full collection landed inside the last block on both runs. It took 1.12 s and 1.20 s, collected 381,164 objects and fell on the same text.
  • Runner speed varied. The scans took 5.58 s on one runner and 2.64 s on the other, about 2.1 times as long on every text. Tests the two runs shared took 1.4 to 1.8 times as long, so the runner accounts for this, not the code.

Even with no collection, the slower runner's largest block took 1.65 s of its 2 s. A collection on a slow runner fails the test; on a fast runner it passes. Python 3.14 passed every time on the same heads, but this PR doesn't depend on why.

Fix

  • _assert_scans_quickly stops the collector while the clock runs and restores it afterwards, even when a scan raises.
  • Each text now gets its own 2 s. The slowest text the diagnostic timed alone took 0.47 s on the slower runner, which leaves a margin of more than 4.
  • Two new tests pin the helper: it stops and restores the collector, and it refuses at exactly 2 s.

Tests only. No source or spec file changes.

Invariants

  • A scan's clock never counts the collector.
  • The helper leaves the collector as it found it, on success and on failure.
  • Any one text whose scans take 2 s or more of CPU fails the test, and the failure names the text.

A cost of this change: the shared budget also caught regressions spread thinly over many texts. A slowdown under 2 s on every text, summing to more than 2 s across a block, used to fail and now passes. The test's job stays quadratic and exponential scans, and those blow up on single texts: see below.

Does it still catch slow scans?

I put real slow scans back into spec_source_citations.py and ran the old and new tests against each, on Python 3.13.9 with a 200 s limit.

Mutant Reintroduces Old test New test
none control passes passes
finditer_everywhere a rescan from each character of a run fails fails
reverse_each_number a reverse locator tried from each number fails fails
marks_uncached finding each citation's marks from the whole string no result in 200 s fails
punycode_unbounded quadratic Punycode on a long label fails fails
quotes_backtrack backtracking over a run of quotes fails fails
possessive_off every possessive quantifier in the module fails fails
number_backtrack backtracking inside one number passes passes

number_backtrack passes both, so this PR opens no new gap there. The new test fails sooner on finditer_everywhere and marks_uncached, because it checks after each text instead of after a block.

Checks

  • test_spec_source_citations.py -k "scan_clock or linear_time": 4 passed on Python 3.13 and on 3.14.
  • ruff check and ruff format --check pass on the changed file.
  • CI: run in progress at the time of writing.

🤖 Generated with Claude Code

The test held blocks of texts to a shared 2 s of CPU time. One full
collection of a worker's heap took 1.1 to 1.2 s of that on CI, and a slow
runner took twice as long as a fast one over the same scans, so the test
failed without any scan changing. Each text now gets its own 2 s, and the
collector stays off while the clock runs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Closing. #1421 dropped test_a_long_string_is_scanned_in_linear_time from main in a62ac83 on María's ruling of 2026-10-30, so this PR has nothing left to fix, and the PR that it was unblocking gets the same result by merging main.

For the record, a CI diagnostic on two runners found one full collection taking 1.12 s and 1.19 s of the test's 2 s budget, with the collector's cost otherwise near zero. The timings also suggest runner variation. An independent review of this branch found that a budget per text misses a quadratic regression the shared budget caught, so this branch's version would have needed more work anyway.

🤖 Generated with Claude Code

@MaxGhenis MaxGhenis closed this Oct 30, 2026
@MaxGhenis
MaxGhenis deleted the nzhub/linear-scan-clock branch October 30, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant