Repository navigation
fix(executor): constraints compile in class context — charset aliases resolve per-character - #124
Merged
Merged
Conversation
…t aliases resolve per-character
Root cause of the alalc-ell-Grek-Latn-2010 divergences (medial γκ
rendered "gk" instead of "nk"): compileItem had a single alias
expansion context, so an imported charset alias resolving to a plain
string (unicode's greek, ~400 codepoints) compiled to
regexpEscape(<whole string>) — a literal sequence. In a constraint
position that sequence became a lookbehind/lookahead no input could
ever satisfy, silently disabling every rule built on it: the map's
before any(greek) ... after any("κΚ") + any(greek) γ→n rules never
fired and the unconstrained parallel γ→g won.
Ruby/RS resolve aliases in two contexts (RS builder.rs
materialize_class vs materialize, convert_class_alias): constraints
and any() arguments are CLASS context, where an imported Item::Str
charset expands one entry per character; from/to stay sequence
context; local string aliases stay sequences even in class context.
This port mirrors that with a classCtx parameter on compileItem:
constraint compilation sites (executeSubRule + parallel megaregexp)
pass class context, the any() case class-resolves its alias arguments,
capture innards and group parts stay sequence-resolved.
Golden capture (interscript-rs tests/golden/isc-wf0.jsonl): pre-fix
exactly 5 divergent rows, all on alalc-ell-Grek-Latn-2010
(γκέγκε, Λαγκαδάς, Παγκράτι, Κονγκό, Νταγκεστάν); post-fix all
11,048 rows byte-identical — no other map's output changes.
New specs:
- test/golden-isc-wf0.test.ts — the TS mirror of the RS
golden_capture additivity gate (skips without sibling checkouts);
- test/isc/constraints.test.ts — per-shape regression units (imported
charset in class position, sum constraints, before+after combined,
capital variants, local-alias sequence preservation, nth_string
replacement) plus the five ell rows through the full path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Contextual (before/after-constrained) rules were silently inert whenever they referenced an imported charset alias —
alalc-ell-Grek-Latn-2010's medial γκ rules never fired, so TS rendered "gk" where the map's embedded tests and the RS runtime render "nk" (5 golden-capture divergences).Root cause
compileItemhad a single alias-expansion context. An imported charset alias resolving to a plain string (dependency mapunicode'sgreek, ~400 codepoints) compiled via the string case toregexpEscape(<whole charset>)— a literal sequence. In a constraint position that sequence became the lookbehind(?<=(?:<400 chars…>)), which no input can ever satisfy, so rules likenever matched and the unconstrained parallel
γ → gwon.Ruby/RS resolve aliases in two contexts (RS
builder.rs:materialize_classvsmaterialize;convert_class_alias): constraints andany()arguments are CLASS context, where an importedItem::Strcharset expands one entry per character;from/tostay sequence context; a local string alias stays a sequence even in class context. The fix mirrors exactly that with aclassCtxparameter oncompileItem; the executor's constraint compile sites pass class context,any()class-resolves its alias arguments, capture innards and group parts stay sequence-resolved.Fix
src/runtime/compile-item.ts:compileItem(item, ctx, classCtx = false); imported charset alias in class position → per-character set (stringCharset);any()arguments class-resolve alias entries; capture innards forced to sequence context (RS parity).src/runtime/executor.ts:executeSubRuleand the parallel megaregexp path compilebefore/not_before/after/not_afterwithclassCtx = true.Verification
ff173b0): exactly 5 divergent rows, all onalalc-ell-Grek-Latn-2010— γκέγκεgkegke→gkenke, ΛαγκαδάςLagkadas→Lankadas, ΠαγκράτιPagkrati→Pankrati, ΚονγκόKongko→Konnko, ΝταγκεστάνḎagkestan→Ḏankestan.cargo test --release --test golden_capturepasses; the rs crate and maps corpus were only read.vitest run— 310 passed / 4 skipped (pre-existing), 0 failed.npm run lint,npm run format:check,npm run buildall clean.New specs (9 tests)
test/golden-isc-wf0.test.ts— TS mirror of the RSgolden_captureadditivity gate: replays all 11,048 rows, requires byte-identical output (skips without the siblinginterscript-rs/mapscheckouts).test/isc/constraints.test.ts— per-shape regression units, each verified to fail pre-fix: imported charset in class position (root-cause shape),from any(charset)per-char,to any(charset)nth_string, sum constraint with charset part, boundary-sum word-final protection, before+after combined with capital-vs-small variants, local-alias sequence preservation, plus the five ell rows (and Χονγκ Κονγκ / Γκαμπόν guards) through the full path.Test plan
npx vitest run— full suite green (310 passed / 4 skipped)npx vitest run test/golden-isc-wf0.test.ts test/isc/constraints.test.ts— 9/9 with sibling checkouts presentcd ../interscript-rs && cargo test --release --test golden_capture— rust additivity gate greennpm run lint && npm run format:check && npm run build