Repository navigation
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🚀 Deploying Preview to Cloudflare 🚀Preview Deployments by commit
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1156 +/- ##
==========================================
+ Coverage 93.32% 93.55% +0.23%
==========================================
Files 273 282 +9
Lines 27050 28171 +1121
Branches 2722 2836 +114
==========================================
+ Hits 25244 26356 +1112
- Misses 1784 1793 +9
Partials 22 22 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
|
| File | Main | PR | Change |
|---|---|---|---|
addons.html |
354.63 KB | 361.27 KB | +6.64 KB (+1.9%) |
embedding.html |
59.79 KB | 61.38 KB | +1.59 KB (+2.7%) |
Performance estimate (single CI run)
- Generation time: 43.5% faster (30.15 s → 17.04 s)
- Peak memory: 25.5% lower (2.50 GB → 1.86 GB)
legacy-json Generator
Performance estimate (single CI run)
- Generation time: 2.8% faster (8.70 s → 8.46 s)
- Peak memory: 19.0% lower (1.64 GB → 1.33 GB)
llms-txt Generator
Performance estimate (single CI run)
- Generation time: 27.6% slower (4.67 s → 5.96 s)
- Peak memory: 16.5% lower (1.64 GB → 1.37 GB)
orama-db Generator
Output size: 1 file changed · net -2.00 B
File size details
| File | Main | PR | Change |
|---|---|---|---|
orama-db.json |
9.57 MB | 9.57 MB | -2.00 B (-0.0%) |
Performance estimate (single CI run)
- Generation time: 76.2% slower (4.92 s → 8.67 s)
- Peak memory: 23.2% lower (1.87 GB → 1.43 GB)
web Generator
Output size: 68 files changed · net +1.82 MB
File size details
| File | Main | PR | Change |
|---|---|---|---|
all.html |
33.07 MB | 34.89 MB | +1.82 MB (+5.5%) |
assets/SearchBox-886q7CtQ.js |
84.66 KB | — | -84.66 KB (-100.0%) |
assets/SearchBox-ClnW7TK8.js |
— | 84.66 KB | +84.66 KB |
assets/dist-CKngGbBr.js |
30.42 KB | — | -30.42 KB (-100.0%) |
assets/dist-DaJ62IKD.js |
— | 30.42 KB | +30.42 KB |
assets/SideBar-BZ0sUX8x.js |
25.61 KB | — | -25.61 KB (-100.0%) |
assets/SideBar-BHV36Nzp.js |
— | 25.61 KB | +25.61 KB |
assets/client-BXtnUTPQ.js |
24.91 KB | — | -24.91 KB (-100.0%) |
assets/client-B3VvMd8I.js |
— | 24.91 KB | +24.91 KB |
assets/config-Cx-IX3nR.js |
24.37 KB | — | -24.37 KB (-100.0%) |
assets/config-DtT2jlD0.js |
— | 24.09 KB | +24.09 KB |
assets/Combination-OVHFiUe7.js |
16.11 KB | — | -16.11 KB (-100.0%) |
assets/Combination-BXjTcf0a.js |
— | 16.11 KB | +16.11 KB |
assets/ThemeToggle-D_UY-0YN.js |
13.65 KB | — | -13.65 KB (-100.0%) |
assets/ThemeToggle-BDk05zng.js |
— | 13.65 KB | +13.65 KB |
assets/dist-C6BUFa3H.js |
10.20 KB | — | -10.20 KB (-100.0%) |
assets/dist-DAVnU38z.js |
— | 10.20 KB | +10.20 KB |
assets/Layout-C6XLgpZ3.js |
10.13 KB | — | -10.13 KB (-100.0%) |
assets/Layout-R7RUVoMG.js |
— | 10.13 KB | +10.13 KB |
assets/compat-D_gF0q9c.js |
10.13 KB | — | -10.13 KB (-100.0%) |
assets/compat-BdLDmi5I.js |
— | 10.13 KB | +10.13 KB |
assets/Tooltip-BCBLALa2.js |
7.93 KB | — | -7.93 KB (-100.0%) |
assets/Tooltip-DhSa3-SO.js |
— | 7.93 KB | +7.93 KB |
assets/dist-CEw6Vg07.js |
6.99 KB | — | -6.99 KB (-100.0%) |
assets/dist-EwcSEbA2.js |
— | 6.99 KB | +6.99 KB |
assets/jsx-runtime-nvJAqap8.js |
5.67 KB | — | -5.67 KB (-100.0%) |
assets/jsx-runtime-B2kvX5Dd.js |
— | 5.67 KB | +5.67 KB |
assets/dist-BH7R-Ve2.js |
3.93 KB | — | -3.93 KB (-100.0%) |
assets/dist-_cK0Q299.js |
— | 3.93 KB | +3.93 KB |
assets/CodeTabs-BtSs7AO4.js |
3.89 KB | — | -3.89 KB (-100.0%) |
assets/CodeTabs-CxesZulN.js |
— | 3.89 KB | +3.89 KB |
assets/CodeBox-CFjkd_W5.js |
3.44 KB | — | -3.44 KB (-100.0%) |
assets/CodeBox-BN6z2RpZ.js |
— | 3.44 KB | +3.44 KB |
assets/hooks.module-CW9C37Hs.js |
3.40 KB | — | -3.40 KB (-100.0%) |
assets/hooks.module-DV8BJgCx.js |
— | 3.40 KB | +3.40 KB |
assets/FunctionSignature-CW3k6SWp.js |
2.28 KB | — | -2.28 KB (-100.0%) |
assets/FunctionSignature-BHfSHcI7.js |
— | 2.28 KB | +2.28 KB |
assets/Banner-C05n5eub.js |
2.13 KB | — | -2.13 KB (-100.0%) |
assets/Banner-CFOw4ClD.js |
— | 2.13 KB | +2.13 KB |
assets/ChangeHistory-7LT_-gdw.js |
1.77 KB | — | -1.77 KB (-100.0%) |
assets/ChangeHistory-Dyc9FvW-.js |
— | 1.77 KB | +1.77 KB |
assets/DataTag-zNRK6tTk.js |
844.00 B | — | -844.00 B (-100.0%) |
assets/DataTag-DUkSX3Tx.js |
— | 844.00 B | +844.00 B |
assets/DocumentationIndex-C3DhS3Pm.js |
833.00 B | — | -833.00 B (-100.0%) |
assets/DocumentationIndex-DM3ImrxD.js |
— | 833.00 B | +833.00 B |
addons.html |
377.69 KB | 376.96 KB | -751.00 B (-0.2%) |
assets/ArrowUpRightIcon-Bi08b7Qh.js |
618.00 B | — | -618.00 B (-100.0%) |
assets/ArrowUpRightIcon-BaLMheMJ.js |
— | 618.00 B | +618.00 B |
assets/Badge-BR0j23fd.js |
607.00 B | — | -607.00 B (-100.0%) |
assets/Badge-oAi6J1ZB.js |
— | 607.00 B | +607.00 B |
assets/AlertBox-BNTUtRYe.js |
591.00 B | — | -591.00 B (-100.0%) |
assets/AlertBox-B0FVXSf0.js |
— | 591.00 B | +591.00 B |
assets/CodeBracketIcon-D9zS3xWH.js |
512.00 B | — | -512.00 B (-100.0%) |
assets/CodeBracketIcon-Da62_n8V.js |
— | 512.00 B | +512.00 B |
assets/dist-DHGxzEgh.js |
477.00 B | — | -477.00 B (-100.0%) |
assets/dist-Dw05tQo8.js |
— | 477.00 B | +477.00 B |
assets/ChevronDownIcon-DY_kMtyw.js |
468.00 B | — | -468.00 B (-100.0%) |
assets/ChevronDownIcon-Bv3nn1RV.js |
— | 468.00 B | +468.00 B |
assets/renderLabel-B05hHKeG.js |
447.00 B | — | -447.00 B (-100.0%) |
assets/renderLabel-MrIJcOxn.js |
— | 447.00 B | +447.00 B |
assets/Blockquote-CxiKUNKo.js |
167.00 B | — | -167.00 B (-100.0%) |
assets/Blockquote-DsF6xEUn.js |
— | 167.00 B | +167.00 B |
assets/useRemoteConfig-DGOFaw-H.js |
151.00 B | — | -151.00 B (-100.0%) |
assets/useRemoteConfig-Du1lHNBi.js |
— | 151.00 B | +151.00 B |
n-api.html |
1022.93 KB | 1023.04 KB | +113.00 B (+0.0%) |
assets/withIsland-C8b6-JCe.js |
105.00 B | — | -105.00 B (-100.0%) |
assets/withIsland-DZPeGbXA.js |
— | 105.00 B | +105.00 B |
embedding.html |
69.16 KB | 69.12 KB | -41.00 B (-0.1%) |
Performance estimate (single CI run)
- Generation time: 64.4% faster (47.31 s → 16.84 s)
- Peak memory: 33.8% lower (3.31 GB → 2.19 GB)
avivkeller
reviewed
Oct 7, 2026
ovflowd
commented
Oct 8, 2026
ovflowd
marked this pull request as ready for review
October 8, 2026 15:35
ovflowd
marked this pull request as draft
October 8, 2026 15:41
Member
Author
|
Marking as draft because #1157 should be merged first. |
4 tasks done
The Shiki plugin registered every bundled language (~250 grammars) in each thread that highlighted code, which cost ~2s and ~100MB per thread and made every highlight several times slower, as each code block was matched against grammars it never uses. Importing it also imported all of them, through `@node-core/rehype-shiki`'s `LANGS` and its plugin, on every thread loading `jsx-ast`, the main thread included. The highlighter now registers a bundled language the first time code in it is highlighted, along with the bundled languages a configured one embeds, and lists the bundled ones from their metadata alone. The rehype plugin is a port of `@node-core/rehype-shiki`'s, so importing it no longer imports every grammar, and the themes are given to Shiki by name, which it keeps parsed instead of parsing them for every highlight. The grammars now come from doc-kit's own `shiki` dependency (4.4.3) rather than the copy `@node-core/rehype-shiki` pins (4.3.1). Its C++ grammar highlights types and template arguments differently, which shows on the Node.js docs' C++ examples. Assisted-by: Claude Opus 5.5 <[email protected]>
Every highlighted token of every code block, signature and type became a hast element, then a JSX element, then generated code, only to be rendered back into the same markup: most of what `jsx-ast` allocated, and much of what the pages' code took to compile and render. Highlighted code is static, as islands adopt it without rendering it again, so it now reaches the pages as the markup Preact renders it to, held by a `<code>` through `dangerouslySetInnerHTML`. Code blocks are embedded by a plugin running after Shiki. Types and signatures are embedded as they are highlighted, before `rehype-raw`, which would otherwise parse each of their tokens again. Assisted-by: Claude Opus 5.5 <[email protected]>
The pool renders a single task on the calling thread, so `all.html`, rendered on its own after the other pages, was rendered on the main thread. That held the whole site in its heap, and kept the pool from shutting its idle workers down until it was done. It is now rendered alongside the other pages, first, as it takes by far the longest. It is no longer minified either. The minifier's memory grows to about twelve times the page it is given and is never returned, and this page is the whole site: minifying the Node.js docs' ~35MB `all.html` takes ~400MB and over a second, for a page 2% smaller once compressed. Assisted-by: Claude Opus 5.5 <[email protected]>
V8 keeps a string built by concatenation as a rope of all of its pieces, so the code `jsx-ast` generates (one write per token) and the HTML Preact renders (one write per tag) took around ten times the size of their text: 28MB of page code was held as 363MB, and `all.html` alone as ~430MB. Both are now flattened as soon as they are complete, which lets the pieces be collected. Assisted-by: Claude Opus 5.5 <[email protected]>
The main thread loads every generator of a run, but only ever calls `generate`; building the pages is the workers' job. Still, the generators imported what only their workers use, so the main thread loaded the TypeScript parser and the HTML minifier (both WASM), and what building a page takes. Those are now imported where they are used. `getFullName` also moves to a module of its own: `buildBarProps`, which `section-pages` imports, and `buildContent` only need a page's full name, not the code that builds and highlights signatures. Assisted-by: Claude Opus 5.5 <[email protected]>
The threads rendering pages load the `html` generator's module, and so everything `generate` imports, along with `processing.mjs`. That included `config.mjs`, which loads Shiki for the languages' display names and a Markdown processor of its own: ~14MB and ~75ms per worker, for what only the main thread uses, while the workers render pages, which is when the build peaks. `createVirtualImports` moves into `config.mjs`, which `generate` imports when it bundles the site, so the main thread alone loads it. Assisted-by: Claude Opus 5.5 <[email protected]>
A process gives all of its memory back when it exits, native memory included, which a worker thread does not: Rolldown, for one, only frees its allocator's memory with its process. `createChildProcess(moduleURL)` runs a module in a child process of its own through a generic host script, and calls its exports over birpc (MIT, no dependencies). Calls in flight fail once the process exits, and `close` ends it. `on` returns nothing: birpc holds its first call until whatever `on` returns settles, so returning the emitter let a `close` right after a call end the process before the call was written, failing it with EPIPE rather than with the process's exit. Assisted-by: Claude Opus 5.5 <[email protected]>
Vite bundles with Rolldown, whose native memory (~250MB building the Node.js docs) is only returned when its process exits, so it stayed in the build's process while the pages rendered, which is when the build peaks. The default Vite adapter now runs in a child process of its own (`createChildProcess`), which `generate` ends once every page program is compiled, before the pages are rendered. Bundlers can have a `close` for that: `generate` calls it once it is done bundling and compiling. An adapter passed as `bundler` still runs on the main thread, so its function-valued options keep working. Assisted-by: Claude Opus 5.5 <[email protected]>
A worker that highlights code no longer holds every grammar Shiki bundles, only those of the code it highlighted. The comment now says what holds for every generator: each worker has a heap of its own, with the libraries and the pages it is working on. Assisted-by: Claude Opus 5.5 <[email protected]>
V8 lets a heap whose limit is 2GB or more grow to up to four times its live data before collecting it, and on a machine with plenty of memory every worker's limit is 4GB, so each held several times what it used. Below 2GB, a heap only grows to about twice its live data. `workerHeapSize` (`--worker-heap-size`) sets each worker's old space limit, in MB, like `threads` sets their number. It defaults to the limit V8 gives this process, at most `DEFAULT_MAX_WORKER_HEAP_SIZE` (2047): a machine with less memory keeps the smaller limit V8 picks for it. An explicit `--max-old-space-size` still wins, as V8 prefers it. On Node core's build (4 threads), the peak goes from 2.73GB to 2.39GB on Node 26 and from 2.90GB to 2.73GB on Node 24. Lower limits gain little more: 2.29GB at 1536MB, 2.30GB at 768MB. Assisted-by: Claude Opus 5.5 <[email protected]>
Assisted-by: Claude Opus 5.5 <[email protected]>
ovflowd
force-pushed
the
perf/memory-usage
branch
from
October 10, 2026 11:59
4f2e846 to
dd8f6c1
Compare
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Important
This is a proof of concept, opened as a draft to discuss the approach. Happy to split it into smaller PRs once we agree on the direction.
Building the Node.js docs peaks at 3.7–3.9GB of memory, while a single page only needs ~100MB. Node core only builds the HTML docs on machines with more than 5GiB of memory, and silently skips them otherwise. That's what happens on its
ubuntu-slimdoc CI, whose docs artifact has no HTML at all, and most likely on the release machine too.This PR brings the peak down to 2.1–2.7GB, and the build time to less than half, by tackling where that memory went:
jsx-ast, the main thread included. The docs use about ten languages. Each language is now registered the first time code in it is highlighted, which takes highlighting from ~17s of CPU to ~4s. The plugin is a port of@node-core/rehype-shiki's, whose module imports every grammar, and the themes go to Shiki by name, so it stops parsing them again for every block.jsx-astallocated. A plugin after Shiki embeds the code blocks, and types and signatures are embedded as they're highlighted, beforerehype-rawwould parse each of their tokens again.all.htmlrenders in the worker pool, unminified. It was rendered and minified on the main thread (~1GB), and the minifier's WASM memory grows to ~12× the page without ever shrinking. Unminified, it's 5% larger, or ~2% once gzipped.createChildProcessin core runs a module in a process of its own and calls it overbirpc(MIT, no dependencies). The Vite adapter runs in one, which ends once the pages are compiled, before they render.workerHeapSizeoption,--worker-heap-size). With a limit of 2GB or more (4GB on machines with 16GB+), V8 lets a heap grow to 4× its live data before collecting it; just under 2GB, about 2×.+=held ~10× their text), and each thread only imports the modules it uses.Validation
node --run test(761 tests),node --run format:checkandnode --run lintpass. New tests cover loading grammars on demand, the static markup, the child process (including one that dies mid-call) and the workers' heap limit.main, on thewebtarget (71 pages) and on Node core's config (741), exceptall.htmland three C++ pages (addons,embedding,n-api).all.htmlis unminified, and minifying it givesmain's page back outside those C++ code blocks. Their highlighting changes because grammars now come from core's ownshiki(4.4.3) instead of the one@node-core/rehype-shikipins (4.3.1). Pinning core's to ~4.3.1 would keep the old highlighting.Benchmarks
main→ this branch, building Node core's docs with its config (legacy-json-all+section-pages) unless noted. Medians of 3 interleaved runs on an M4 Pro; peak memory is the physical footprint of every process in the build.webtarget, Node 26CI's per-generator comment compares a single run with a single run of
mainon another runner, which swings ±20% even on unrelated PRs (e.g. #1155).Related Issues
Refs: #815
Refs: nodejs/node#62045
Check List
node --run testand all tests passed.node --run format:check&node --run lint.