Skip to content

Stockfish 19's small net, and the best line first - #334

Merged
ashtonanderson merged 13 commits into
mainfrom
claude/perf-round-10
Nov 5, 2026
Merged

ashtonanderson merged 13 commits into
mainfrom
claude/perf-round-10

Conversation

@ashtonanderson

@ashtonanderson ashtonanderson commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

On a real iPhone 12, the analysis page survives repeated visits and reloads that make Safari give up on today's code. After a move, the four lines settle about 0.7 s sooner (about 3.4 → 2.7 s) and the best line reaches depth 18 about 2.4 s sooner. The chart comes 0.1–0.2 s later. A first visit no longer downloads 78 MB of nets, and Stockfish uses about 5× less memory.

The depth label now means something different, so its timings aren't like for like. Today, "d18" appears when four lines have reached depth 18. Here it appears when the best line alone has, with the other lines still shallower. The fair comparison of the whole display is "lines settled".

Draft: awaiting a decision on merging.

What changes

  1. Stockfish 19's small net replaces Stockfish 17. It uses one 1.2 MB net served with the site, instead of two nets totalling 78 MB. The net download code and IndexedDB cache are gone, and old nets are removed from returning visitors' storage.
  2. The best line goes deep first. The search now runs: every move to depth 6, the best line to the target depth, Maia's moves to depth 14, the other three lines one at a time to the target, every other move to depth 10, then Maia's moves to the target.
  3. A position isn't done until the search finishes. Updates carry partial until the last one. Revisits, full-game analysis, Learn From Mistakes and drills all respect it, and unfinished results aren't saved.
  4. Less memory reserved. Stockfish reserves 134 MB and Maia 268 MB, and both retry when an iPhone has none to spare.

Results

Main at 3e5ef6d against this branch's code: production builds, warm visits, 6 pairs. The phone is emulated, with every thread at a quarter of full speed; it's far slower than an iPhone 12, so use it to compare builds. Stockfish was sooner in 6 of 6 pairs on every row, and the board and Maia's moves are within the noise.

seconds phone desktop
first depth 9.02 → 5.02 1.55 → 0.65
depth 12 18.02 → 7.50 2.80 → 0.80
depth 18 29.54 → 9.46 4.87 → 1.25
  • Stepping through a game: steps reaching depth 18 go from 13 to 24 of 24 on the phone, and from 30 to 32 of 32 on a desktop.
  • After a step, desktop: the lines stop changing at 3.38 s instead of 5.36 s, and the chart is unchanged at 1.20 s.
  • First visit, phone on 4G, before the last search change: Stockfish's first depth goes from about 126 s to about 5.6 s, and Maia's moves from about 103 s to about 53 s.
  • Renderer memory, desktop, before the memory fix: peak goes from about 1,045 MB to about 555 MB.

On a real iPhone 12

Today's code against the new code, the whole site in Safari, one or two runs each:

Today's code New code
6 return visits page reloaded, both times all finished, both times
10 reloads of the whole page Safari gave up after 4 engines back on 10/10
After a step: depth 12 0.77–0.82 s 0.28–0.29 s
After a step: depth 18 3.27–3.37 s 0.87–1.18 s
After a step: chart 1.19–1.34 s 1.39–1.58 s
After a step: lines settled 3.37–3.42 s 2.59–3.12 s
First visit: Stockfish's first depth 22.0 s 3.6 s
First visit: Maia's moves 19.7 s 10.6 s
First visit: board 0.94 s 3.05 s (one load)

The reload tests are harsher than normal browsing, and this is one phone. On today's code the stepping test itself reloaded the page in 3 of 4 tries, so its step rows come from the runs that finished; the new code finished all 6. No real Android phone has been measured. Maia on WebGPU was tested and not pursued: in the one run that gave numbers, it didn't free the CPU for Stockfish.

Trade-offs to review

  • Accuracy: at depth 18 the new engine picks moves about as well as Stockfish 17 against deep native searches, but its final best move on game positions is a little worse (0.42 vs 0.30 win points lost per move against a Stockfish 19 reference, 0.49 vs 0.27 against a Stockfish 17 one).
  • Move colours: 87.9% of played moves get the same colour from both engines.
  • The depth label: "d18" now means the best line got there, and the other lines fill in afterwards.
  • The chart: 0.1–0.2 s later after a step on the iPhone, unchanged within noise on ada. It appears 40–60 ms after Maia's last answer, and rendering it at normal priority didn't help, so the likeliest cause is Maia sharing the CPU with Stockfish. That isn't confirmed, because today's code reloaded before giving the same measurement.
  • Saved analysis: it will mix Stockfish 17 and 19 results, and unfinished results are no longer saved.
  • human-bar: its loadNnueModelShared and second engine need rewriting.

Before merging

  • Run the engines and the whole site on an iPhone (public/phone-lab.html, on claude/phone-lab-base and claude/engine-bench, neither for merging).
  • Repeat the reload tests on the final search order: both engines back on 10 of 10, in both tests.
  • Try the analysis page by hand on an iPhone: real taps and scrolling aren't covered by those tests.

Checks

npm test (74 tests), npx tsc --noEmit and npm run lint pass. An independent code review of the first three code commits found five defects, which are fixed in d671b88 with tests. Full method and tables are in LOADING.md, round 10. This also carries round 9's notes from the closed #333.

🤖 Generated with Claude Code

ashtonanderson and others added 6 commits November 3, 2026 17:41
Nothing changed: on a warm phone visit the engines are short of CPU, and
each change tried either made Stockfish later when Maia came sooner or
was within the noise. Records where the time goes, the five changes tried
and reverted, how much smaller Maia models and Stockfish builds lose, and
first visits on 4G.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The staged search spent most of its time on the four-line search to the
target depth: on a desktop, ~4.8 s of ~5.8 s until depth 18 appeared. After
screening, one line now searches alone to the target depth (~1.3 s), then
Maia's moves and the four lines fill in, helped by its hash.

An update is `partial` until the last one, so a position is not done when
only its best line has reached the target depth: the page searches it again
if it is left early, full-game analysis waits, and it is not saved as done.
The depth on show no longer falls, and a shallower score cannot outrank a
deeper best line.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Stockfish 17's two nets were 78 MB: a long download on a first visit, ~8 s
on a phone to read and unpack at each visit, and 517 MB of memory. The
smallnet build of @lichess-org/stockfish-web uses a 1.2 MB net served with
the site, needs 81 MB, and at the same depth picks moves as well against a
deep native search (see LOADING.md).

Phone, warm visit, 6 paired runs: first depth 9.58 -> 5.32 s, depth 12
19.18 -> 10.18 s, depth 18 30.70 -> 19.68 s, all 6 sooner. Desktop: 1.69 ->
0.70 s, 2.93 -> 1.29 s, 5.07 -> 3.09 s, all 6 sooner. Board and Maia as before.

The net download (hosts, resume, IndexedDB cache) goes, and the old nets are
removed from a returning visitor's storage. The build starts threads as it
needs them, so the pool cap is gone too.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Every move was searched to depth 10 before the best line started: on a phone,
seconds of the time to depth 12. They now go to depth 6, the best line to the
target depth, and then every move again to depth 10, helped by its hash.

On 222 game positions on a desktop (no page, one engine each), depth 12 comes
at 0.12 s instead of 0.62 s and depth 18 at 0.98 s instead of 1.51 s, with the
same accuracy against a deep native search, and all lines are done at 7.3 s
instead of 6.6 s.

During a search a move is coloured only from a score of depth 10 or more, so
that the shallow pass cannot flash a false blunder.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
From a review of the last three commits:

- A finished search replaces a deeper unfinished one. Full-game analysis at a
  lower target could not replace it, and waited 60 s per position.
- Learn From Mistakes and drills search an unfinished position again. Its
  played move may still be at depth 6 and unjudged, so mistakes were missed.
- Unfinished searches aren't saved. Saved at depth 12 they came back looking
  finished, with no per-move depths, and shallow scores could colour moves.
- A move at a shallow depth can't outrank the lines at the target depth in the
  final answer.
- The time limit on Stockfish's net covers reading it, not just its headers,
  and its read errors get the connection message.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@vercel

vercel Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
maia-platform-frontend Ready Ready Preview Nov 5, 2026 3:27am UTC

Request Review

…o spare

On an iPhone, after several reloads of the analysis page, Stockfish and Maia
failed with "RangeError: Out of memory", and later pages could not reserve even
Stockfish's 256 MB: earlier pages' reservations are held for some time.

Stockfish now reserves 134 MB (it uses 81 MB) and Maia 268 MB (198 MB at most),
768 -> 402 MB between them. When even that fails, Stockfish tries again after
1, 3, 8 and 15 s, as Maia now does too, and the page waits up to 40 s for it.

Not yet measured on an iPhone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… after them

With Stockfish 19's small net the four lines settled later than on main (0.49 s
on an iPhone after a step). After the best line, the other three now search one
at a time among the moves not yet shown, instead of four together from depth 1,
and the second pass over every move comes after them, as it changes none of the
lines on show.

On 222 game positions, one engine per game in an empty page, the four lines are
done at 5.37 s instead of 6.46 s, with as many of a deep search's top four moves
(2.90 against 2.86 of 4). Not yet measured on the real page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@ashtonanderson
ashtonanderson marked this pull request as ready for review November 5, 2026 03:26
@ashtonanderson
ashtonanderson merged commit 43d242d into main Nov 5, 2026
6 checks passed

This branch was successfully deployed

1 active deployment
Preview — 98538f16 Deployed Nov 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant