Repository navigation
fix(v18): pin libpg_query with checked scan allocations - #188
pyramation wants to merge 1 commit into
Conversation
…ut of memory as out of WASM memory
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
Review complete. 🟡 1 medium 📍 Findings outside the diff (1) — 🟡 1 medium — defects on lines GitHub can't attach comments to🟡 Medium — // versions/18/src/index.ts
308 if (errorPtr) {
309 // Read PgQueryError struct
310 const messagePtr = wasmModule.getValue(errorPtr, 'i32');
311 const funcnamePtr = wasmModule.getValue(errorPtr + 4, 'i32');
312 const filenamePtr = wasmModule.getValue(errorPtr + 8, 'i32');
313 const lineno = wasmModule.getValue(errorPtr + 12, 'i32');
314 const cursorpos = wasmModule.getValue(errorPtr + 16, 'i32');
315
316 const message = messagePtr ? wasmModule.UTF8ToString(messagePtr) : 'Unknown error';
317 const funcname = funcnamePtr ? wasmModule.UTF8ToString(funcnamePtr) : undefined;
318 const filename = filenamePtr ? wasmModule.UTF8ToString(filenamePtr) : undefined;
319 The new The same gap exists in This PR upgrades the binding to libpg-query v18 and improves how malloc failures are reported: when the parser error message is exactly
Reviewed commit: b30efdb |
|
Folded into #190 |
Summary
Stacked on #183. Fixes the ~50 MB
scanabort.Cause.
pg_query_scanin the fork never checked itsmallocs. At about 50 MB (18.6M tokens) one of them returns NULL inside the 1 GiB heap, and the token protobuf gets packed at address 0. That overwrites static data: the call aborts withAssertion failed: <protobuf bytes>(several MB written to stderr), and the next sync call in the same tick fails too.Changes.
versions/18(MakefileLIBPG_QUERY_COMMITandx-publish.libpgQueryCommit) moves fromb0f5f33to8076f09. That is the head offix/scan-out-of-memory-18, Report out of memory from pg_query_scan instead of writing through NULL libpg_query#12, which checks each allocation, frees what was allocated and returnsresult.error = "out of memory".readDetailedResult(templates/full/index.ts) maps libpg_query's plain"out of memory"error to the sameout of WASM memoryerror that a NULL result already throws. So scan reports one message no matter where the allocation failed.Regression test
New test in
versions/18/test/scan.test.js: "throws out of WASM memory at 50 MB, then sync and async calls work". It runs in a child process, because before this fix the abort corrupts the instance.b0f5f33): the test fails. The child exits 1 because the follow-upscanSyncthrows after the abort.pnpm testpasses 166/166 (emsdk 6.0.3).Merge order
18-constructivewith a merge commit, not squash. The Makefile clones18-constructiveand checks out8076f09, so that commit has to stay reachable on the branch.fatal: reference is not a tree, as fix: stack-depth guard and shadow-stack restore for deep recursion (follow-up to #179) #180 did.Link to Devin session: https://app.devin.ai/sessions/68876ce060664609ab6b63c8bed91e78
Open in Devin Desktop: https://app.devin.ai/desktop/session/68876ce060664609ab6b63c8bed91e78?variant=devin
Requested by: @pyramation