Skip to content

test: check the tool concurrency limit without a timing margin - #226

Open
SizzleUnrlsd wants to merge 2 commits into
mainfrom
test/tool-concurrency-limit-wait
Open

SizzleUnrlsd wants to merge 2 commits into
mainfrom
test/tool-concurrency-limit-wait

Conversation

@SizzleUnrlsd

Copy link
Copy Markdown
Contributor

Closes #225.

tool_scheduling_tests checked that a tool limited to one run at a time does not keep the other tools waiting. It did so by comparing end times, with a 150 ms margin. Since the check was added on 2026-10-01, it failed 8 times in about 100 macOS attempts on GitHub's runners: 6 on macos-14 and 2 on macos-15. The last failure was on main, after #223.

Changes

  • test: the three timed fake tools record their runs through Timeline::record, which wakes a test waiting for runs. This changes no check.
  • test: the limited tool now waits instead of racing.
    • Each of its runs waits for the other tool to be done with every file. If that has not happened after 10 s, it throws.
    • The check becomes failedTools().empty(). It holds when the other tool runs while the limited one holds its worker. It fails when the scheduler makes the other tool wait.
    • The check that the limited tool never runs twice at once stays.

The 10 s limit is still a time bound. It is about 20 times the expected wait of 450 ms, against a 150 ms margin before.

Verification

On macOS (8 cores), ctrace_tool_scheduling_tests was built from main and from this branch.

  • Reproduction on main. The test runs at background priority (taskpolicy -b), where macOS coalesces timers, so sleeps overrun as they do on the runners.
    • The check failed once in 10 runs.
    • With 16 busy loops added, it failed 3 times in 20 runs.
    • CPU load alone does not reproduce the failure: there was none in 25 runs at normal priority with 16 busy loops.
  • This branch, at background priority: no failure in 10 runs. The new check no longer depends on a margin, so these runs only confirm it, they do not prove it.
  • Breaking runOnPool so that each lane waits for the previous one makes the new check fail, after 30 s: each of the three limited runs waits 10 s. The change was reverted.
  • Each commit builds, and the test passes at each commit.
  • scripts/format-check.sh passes with clang-format 20.
  • The other test programs are unchanged; this pull request's CI runs them.

Not changed

"the other files are analyzed while a slow file is" uses the same stopwatch, with a 550 ms margin. It has never failed on CI. Under background priority and load, it failed 9 times in 20 runs on main. If it starts failing on CI, it can be fixed the same way in a follow-up.

The three timed fake tools each locked the timeline and wrote their run themselves. Timeline::record does it in one place, so that recording a run can also wake a test waiting for it.
The check compared end times: the limited tool, 3 x 200 ms, had to be
done after the other tool, 3 x 150 ms, a 150 ms margin. On GitHub's
macOS runners a sleep or a worker is sometimes that late: since
2026-10-01 the check failed 8 times in about 100 macOS attempts, with
no change to the scheduler.

Each run of the limited tool now waits for the other tool to be done
with every file, and throws if it is not after 10 s. The check is that
no tool failed: it holds when the other tool runs while the limited
one holds its worker, and fails when the scheduler makes it wait.
@SizzleUnrlsd SizzleUnrlsd added the test Tests and test tooling label Nov 12, 2026
@SizzleUnrlsd SizzleUnrlsd self-assigned this Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Tests and test tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tool_scheduling_tests: the concurrency limit check fails when a sleep overruns 150 ms

1 participant