diff --git a/TODO.sota/57-a1-authority-completeness.md b/TODO.sota/57-a1-authority-completeness.md new file mode 100644 index 00000000..c51c454e --- /dev/null +++ b/TODO.sota/57-a1-authority-completeness.md @@ -0,0 +1,23 @@ +# WO57 — A1: authority-completeness audit + +Status: PLANNED · Plane A +Goal: enumerate the UNIVERSE of authority romanization/transliteration +systems and diff it against the 293 encoded in `maps/`. Every gap is +either encoded or cited out-of-scope with an authority reference. +Method: +1. Build the catalog: UN WGRS (1968–2016 editions, ~25 systems), + BGN/PCGN current schedule (~55), ALA-LC full language tables + (~170 Romanization tables), ISO 9/233/259/3609/7098/843/9984/9985/ + 233-3, ICAO 9303, national systems (DIN 14640, RTGS, Korean RR + 2000, HK Government, ROC, Thai Royal Institute…). Sources: the + authority documents already collected in interscript-references + + the ISO 24229 code convention. +2. Extract the corpus's system codes from maps/*.isc names; normalize + per ISO 24229; join against the catalog. +3. Emit coverage matrix (authority × system × status: encoded / + gap-encoded-possible / out-of-scope+reason) to + docs/coverage/COVERAGE.md; gaps that are encodable become maps + PRs (whole-document transcription law applies). +Acceptance: coverage matrix published with zero unexplained gaps; +enconding PRs opened for every encodable gap. +Owner gates: none for the audit; map encodings are ordinary PRs. diff --git a/TODO.sota/58-a2-invertibility.md b/TODO.sota/58-a2-invertibility.md new file mode 100644 index 00000000..6fcd579c --- /dev/null +++ b/TODO.sota/58-a2-invertibility.md @@ -0,0 +1,17 @@ +# WO58 — A2: invertibility ledger + reverse maps + +Status: PLANNED · Plane A +Goal: for every system, determine whether forward∘reverse = identity; +publish the loss conditions (homophone mergers, digraph ambiguity, +case conventions, vowel elision). +Method: +1. Script against the installed gem runtime: for each map with a + reverse variant (or a generated one), round-trip the map's embedded + tests + a synthetic Unicode sample per source script. +2. Classify: lossless / lossy-with-convention / not-invertible; attach + minimal distinguishing examples. +3. Where an authority publishes a reverse system: encode it (whole- + document law). Where the forward map is provably bijective: + generate the reverse from the grammar and validate. +Acceptance: docs/invertibility/LEDGER.md covering all systems; reverse +maps shipped for every losslessly invertible one; losses cited. diff --git a/TODO.sota/59-a3-equivalence.md b/TODO.sota/59-a3-equivalence.md new file mode 100644 index 00000000..10456c06 --- /dev/null +++ b/TODO.sota/59-a3-equivalence.md @@ -0,0 +1,15 @@ +# WO59 — A3: system-equivalence checker + distinguishing corpora + +Status: PLANNED · Plane A +Goal: mechanically decide/estimate whether two systems coincide, and +produce the minimal test corpus that distinguishes them. +Method: +1. Compose maps as automata over a sampled alphabet (the ruby + interpreter as oracle); bounded-equivalence search (input-length + horizon escalation). +2. Distinguishing corpus generator: for near-equivalent systems emit + the smallest input set that yields differing outputs; use it to + justify alias/dedup decisions in the corpus. +Acceptance: checker + generator shipped in the runtime toolchain; +every claimed-equivalent map pair in maps/ justified by the checker; +CI uses it to prevent accidental duplicates. diff --git a/TODO.sota/60-a4-is1-semantics.md b/TODO.sota/60-a4-is1-semantics.md new file mode 100644 index 00000000..aeccc3c8 --- /dev/null +++ b/TODO.sota/60-a4-is1-semantics.md @@ -0,0 +1,15 @@ +# WO60 — A4: IS-1 v2 — denotational semantics + +Status: PLANNED · Plane A +Goal: complete the ISC specification with a semantics section so every +runtime behavior is derivable from the document; zero open rulings. +Method: +1. Collect every behavior where runtimes might diverge: boundary + semantics, parallel rule priority, maybe composition, capture/ref, + funcall typing, casing interactions, dependency staging order. +2. Write the denotational (or operational) semantics for the ISC + fragment in IS-1; give each ruling a test vector. +3. Property-test all three runtimes against the semantics. +Acceptance: IS-1 v2 released with the semantics section; the rulings +list emptied; each ruling has vectors in all runtimes' suites. +Depends: nothing; feeds A5. diff --git a/TODO.sota/61-a5-certification-kit.md b/TODO.sota/61-a5-certification-kit.md new file mode 100644 index 00000000..5768e12b --- /dev/null +++ b/TODO.sota/61-a5-certification-kit.md @@ -0,0 +1,15 @@ +# WO61 — A5: third-party conformance certification kit + +Status: PLANNED · Plane A +Goal: an implementable certification path so ANY runtime can +self-certify against Interscript. +Method: +1. Levels: L0 parse (ISC grammar conformance), L1 embedded tests, L2 + frozen conformance vectors (maps + models), L3 full cross-runtime + parity on the distinguishing corpora (WO59). +2. Ship the kit as a package: vector bundles from GitHub Releases + + a runner spec + a self-certification report format (machine- + checkable). +3. Certify our three runtimes at L3 as the reference demonstration. +Acceptance: kit published; report format validated; ruby/ts/py at L3. +Depends: WO58/WO59/WO60 for L3 content. diff --git a/TODO.sota/62-b1-dispatch-doctrine.md b/TODO.sota/62-b1-dispatch-doctrine.md new file mode 100644 index 00000000..8e5ca856 --- /dev/null +++ b/TODO.sota/62-b1-dispatch-doctrine.md @@ -0,0 +1,14 @@ +# WO62 — B1: the dispatch doctrine as registry defaults + +Status: PLANNED · Plane B +Goal: encode the technique-dispatch table — (script class, ambiguity +type, data regime) → recipe — as registry defaults so a new language +gets a recommended recipe automatically. +Method: map banked verdicts into src/pipeline defaults: abjad+lexicon +(sft + morph injection + oracle relabel on disagreeing arms); abugida +(lexicon-first + ByT5 fallback + rule composite); finite ambiguity +(rules); phonotactic targets (ByT5, never CTC); VRR reporting for all +vocalization tasks. Validate against every historical run: the doctrine +must RECOMMEND what actually won. +Acceptance: doctrine module + docs; backtest table (doctrine vs actual +verdicts) with zero contradictions unexplained. diff --git a/TODO.sota/63-b2-hebrew-sota.md b/TODO.sota/63-b2-hebrew-sota.md new file mode 100644 index 00000000..42ff112c --- /dev/null +++ b/TODO.sota/63-b2-hebrew-sota.md @@ -0,0 +1,8 @@ +# WO63 — B2: Hebrew to external SOTA (DER < 5, int8 < 6) + +Status: PLANNED · BLOCKED-OWNER (data-supply asks, drafts in WO43/WO17) +Goal: ReNikud-protocol DER < 5; int8 artifact < 6 (from 8.18). +Method: plane scale-up (large arm) once data lands; learned-IPA v0 +feature layer (r30/31 lines); knesset v6 weak-pretrain (Dicta +machine-label provenance per memory); VRR alongside DER. +Acceptance: external protocol numbers, protocol package published (B9). diff --git a/TODO.sota/64-b3-arabic-sota.md b/TODO.sota/64-b3-arabic-sota.md new file mode 100644 index 00000000..243c560b --- /dev/null +++ b/TODO.sota/64-b3-arabic-sota.md @@ -0,0 +1,12 @@ +# WO64 — B3: Arabic to external SOTA (DER < 2.0, WikiNews < 8) + +Status: PLANNED · launchable now +Goal: SadeedDiac DER < 2.0 (canonical 2.29), WikiNews multiref WER < 8 +(news specialist 8.85). VRR reported. +Method: r8 noisy-student chain — synthesize with the canonical r7 +teacher over large unlabeled MSA, filter by VRR-licensed consistency, +retrain with morph features (r6-morph teacher); register closure on +the news specialist; decode policy unchanged (greedy, first-alternates). +HF Jobs, --detach, --timeout 23h, step ckpts; ≤2 GPU budget shared +with B7. +Acceptance: gates green + external protocol reconciliation (B9). diff --git a/TODO.sota/65-b4-urdu-sota.md b/TODO.sota/65-b4-urdu-sota.md new file mode 100644 index 00000000..7d0802a5 --- /dev/null +++ b/TODO.sota/65-b4-urdu-sota.md @@ -0,0 +1,10 @@ +# WO65 — B4: Urdu — diac CER < 3; G2P license path + +Status: PLANNED · G2P half BLOCKED-OWNER (commercial-license data) +Goal: urd-diac 3.57 → CER < 3; urd-g2p research-only → commercial- +usable data lineage. +Method: diac — oracle relabel arms only if teachers decorrelate (banked +negative: correlated arms transfer nothing); morph/lexicon features; +VRR. G2P — owner selects licensed corpus (option drafts in WO50); +retrain; relabel artifacts non-research-only. +Acceptance: external protocol numbers; license gate green on G2P. diff --git a/TODO.sota/66-b5-persian-registers.md b/TODO.sota/66-b5-persian-registers.md new file mode 100644 index 00000000..d5e16a51 --- /dev/null +++ b/TODO.sota/66-b5-persian-registers.md @@ -0,0 +1,9 @@ +# WO66 — B5: Persian — hold < 0.7 across registers + +Status: PLANNED +Goal: fas-diac 0.603 row-mean holds < 0.7 on poetry and colloquial +registers; Arabic-loan vocalization probe (the known residual). +Method: register-stratified eval sets (build from poetry corpora, +colloquial sources); probe the scar (docs 0.52% lesson: corpus-level vs +row-mean declared); targeted data if registers degrade; VRR. +Acceptance: register-stratified external-protocol numbers published. diff --git a/TODO.sota/67-b6-thai-lexicon.md b/TODO.sota/67-b6-thai-lexicon.md new file mode 100644 index 00000000..c4927472 --- /dev/null +++ b/TODO.sota/67-b6-thai-lexicon.md @@ -0,0 +1,11 @@ +# WO67 — B6: Thai — lexicon-first, three layers (PER < 2) + +Status: PLANNED · launchable now +Goal: published PER < 2; lexicon hit-rate reported; tone accuracy for +the TTS layer (RTGS / IPA / IPA+tone tracked separately). +Method: Royal Institute dictionary (or equivalent licensed lexicon) as +first-class lexicon layer with hit-rate metric; ByT5 fallback for novel +tokens (names); RTGS rule composite over the inferred layer (banked: +CTC cannot); tiny contract for the edge tier. +Acceptance: three-layer eval published; hit-rate + PER external- +protocol certified. diff --git a/TODO.sota/68-b7-khmer-lexicon.md b/TODO.sota/68-b7-khmer-lexicon.md new file mode 100644 index 00000000..cde065a1 --- /dev/null +++ b/TODO.sota/68-b7-khmer-lexicon.md @@ -0,0 +1,10 @@ +# WO68 — B7: Khmer — Chuon Nath lexicon conditioning (CER < 15) + +Status: PLANNED · launchable now (data on disk: khmer-dict-spice) +Goal: khm-latn 27.4 → CER < 15 with hit-rate reported. +Method: build the Chuon Nath-conditioned dataset (headword→reading +pairs, syllabification-aware); lexicon-conditioned ByT5 (retrieval or +prefix conditioning); fp16 zip hygiene (khm fp32 corruption lesson); +conformance fixtures for the new artifact. +Acceptance: gates green; hit-rate + CER on held-out; release via the +strict publish path. diff --git a/TODO.sota/69-b8-community-intake.md b/TODO.sota/69-b8-community-intake.md new file mode 100644 index 00000000..6752a81e --- /dev/null +++ b/TODO.sota/69-b8-community-intake.md @@ -0,0 +1,10 @@ +# WO69 — B8: prove the contribution story (3 community languages) + +Status: PLANNED +Goal: three external-contributor languages go end-to-end through the +2-YAML path (corpora + recipe), landing as shipped models without +framework forks. +Method: pick 3 candidate languages with licensable corpora and clear +ambiguity classes; run the doctrine (WO62) to recommend recipes; +execute via the spine; document friction; fix the spine, not the fork. +Acceptance: 3 languages shipped; friction log resolved into spine PRs. diff --git a/TODO.sota/70-b9-external-benchmark-cert.md b/TODO.sota/70-b9-external-benchmark-cert.md new file mode 100644 index 00000000..1ce0c638 --- /dev/null +++ b/TODO.sota/70-b9-external-benchmark-cert.md @@ -0,0 +1,12 @@ +# WO70 — B9: external-benchmark certification + +Status: PLANNED +Goal: no self-graded SOTA claims — every headline reconciles to an +external leaderboard/protocol package. +Method: per language, pin the external protocol (Arabic: SadeedDiac + +ALTensor; Hebrew: ReNikud; Thai: PyThaiNLP G2P suite; Persian: published +papers' splits; Urdu: published baselines) as protocol packages in +interscript-ml-tools; publish prediction files for every claim (the +frontier-predictions release pattern); multiref handled per WO01 +lessons with VRR. +Acceptance: claims matrix where every row links predictions + protocol. diff --git a/TODO.sota/71-c1-conformance-matrix.md b/TODO.sota/71-c1-conformance-matrix.md new file mode 100644 index 00000000..6d43d4a4 --- /dev/null +++ b/TODO.sota/71-c1-conformance-matrix.md @@ -0,0 +1,11 @@ +# WO71 — C1: model × runtime conformance matrix in CI + +Status: PLANNED +Goal: every shipped model byte-exact in ruby/ts/py, CI-enforced, forever. +Method: fixtures already frozen in conformance/ (12 rows each; sidecar +on the release). Wire each runtime's CI to download the fixture set +from the models Releases (data channel law) and run the full matrix: +all shipped models, not just the ones curated per-repo. A red cell +blocks merge like any test. +Acceptance: matrix visible in CI summary (model × runtime grid, all +green); new model publication extends the matrix automatically. diff --git a/TODO.sota/72-c2-metric-convention-gate.md b/TODO.sota/72-c2-metric-convention-gate.md new file mode 100644 index 00000000..43babba2 --- /dev/null +++ b/TODO.sota/72-c2-metric-convention-gate.md @@ -0,0 +1,10 @@ +# WO72 — C2: gates reject undeclared metric conventions + +Status: PLANNED +Goal: quality gates refuse a metric that doesn't declare its convention +(row-mean vs corpus-level) and anchor. +Method: extend the gate registry (WO55 pattern): MetricDeclaration +required on every metrics file consumed by a gate; validate in +`isxml gate` and publish guard. Spec: real instances, no doubles. +Acceptance: gate tests proving rejection; all existing recipes updated +with declarations (they are, per WO53 — verify). diff --git a/TODO.sota/73-c3-perf-budgets.md b/TODO.sota/73-c3-perf-budgets.md new file mode 100644 index 00000000..34c4ec5b --- /dev/null +++ b/TODO.sota/73-c3-perf-budgets.md @@ -0,0 +1,11 @@ +# WO73 — C3: performance budgets in CI + +Status: PLANNED +Goal: decode latency + artifact size budgets enforced so regressions +block like test failures. +Method: measure current shipped baselines (single-row greedy, p50/p95 +per model, mobile-typical CPU); encode budgets in models.yaml metadata +(perf: {latency_p50_ms, size_mb}); runtime CI asserts the budget on the +conformance fixture set; budget changes are owner-visible recipe edits. +Acceptance: budgets in the index; CI green under budget; a synthetic +regression fails CI (proven). diff --git a/TODO.sota/74-d2-spec-release.md b/TODO.sota/74-d2-spec-release.md new file mode 100644 index 00000000..f4fd849a --- /dev/null +++ b/TODO.sota/74-d2-spec-release.md @@ -0,0 +1,10 @@ +# WO74 — D2: IS-1/ISC as a citable standard + +Status: PLANNED · Depends WO60 +Goal: versioned spec releases with a citation block; the spec becomes +an artifact others implement against. +Method: IS-1 v2 (WO60) tagged as spec release v2.0 on the canonical +repo; spec PDF/HTML artifacts attached; citation block (BibTeX) in the +README + site spec page. D1 (PyPI publisher) remains tracked in +WO53-ship-set; do not nag. +Acceptance: spec release tagged; citation renders on the site. diff --git a/TODO.sota/75-d3-catalogue.md b/TODO.sota/75-d3-catalogue.md new file mode 100644 index 00000000..6c44785b --- /dev/null +++ b/TODO.sota/75-d3-catalogue.md @@ -0,0 +1,10 @@ +# WO75 — D3: catalogue completeness on the site + +Status: PLANNED +Goal: every shipped object (293 maps, all index-v12 models, three +runtimes + API status) browsable on interscript.org. +Method: audit the site's map/model catalogue against models.yaml + +maps corpus; fill gaps; add runtime status cards (versions, conformance +level per WO61). +Acceptance: catalogue diff = zero; automated freshness check (site +build pulls the index and fails on drift). diff --git a/TODO.sota/76-e1-paper-formalism.md b/TODO.sota/76-e1-paper-formalism.md new file mode 100644 index 00000000..e33dfbe8 --- /dev/null +++ b/TODO.sota/76-e1-paper-formalism.md @@ -0,0 +1,14 @@ +# WO76 — E1: paper — the ISC formalism, corpus, and interoperability + +Status: PLANNED · Depends WO57/WO60 +Goal: a complete draft (LREC/COLING-class) presenting ISC as an +executable formalism for authority script conversion; the 293-system +corpus; the differential-testing interoperability methodology; the +conformance regime. +Method: sections — intro (conventions as unverifiable prose), the +formalism, corpus construction + transcription discipline, the +interoperability contract (parity methodology), evaluation (coverage +WO57 + case studies), related work (FST tooling, romanization surveys). +Owner reviews the draft before any external submission (external- +communication law). +Acceptance: full draft in docs/papers/; owner-reviewed. diff --git a/TODO.sota/77-e2-paper-dispatch.md b/TODO.sota/77-e2-paper-dispatch.md new file mode 100644 index 00000000..8a09f75b --- /dev/null +++ b/TODO.sota/77-e2-paper-dispatch.md @@ -0,0 +1,11 @@ +# WO77 — E2: paper — technique dispatch for underdetermined mappings + +Status: PLANNED · Depends WO62 + plane B results +Goal: a complete draft presenting the dispatch doctrine: ambiguity +class → technique, with the negative-results registry (RL, CTC, +LLM-teachers, correlated distillation) as a first-class contribution +and VRR as an evaluation proposal. +Method: results from B2-B7; backtest table from WO62; the two +structural rules (vocalization-as-composition, abugida lexicon-first) +as the conceptual frame. +Acceptance: full draft with the results matrix; owner-reviewed. diff --git a/TODO.sota/78-e3-benchmark-artifacts.md b/TODO.sota/78-e3-benchmark-artifacts.md new file mode 100644 index 00000000..e64cf786 --- /dev/null +++ b/TODO.sota/78-e3-benchmark-artifacts.md @@ -0,0 +1,9 @@ +# WO78 — E3: frozen benchmarks as citable artifacts + +Status: PLANNED +Goal: eval protocols + conformance suites published as citable, +versioned artifacts. +Method: protocol packages (WO70) + conformance vectors bundled as +tagged releases with per-file sha256 sidecars (data-channel law) and a +citation block per bundle. +Acceptance: bundles tagged; citations render; sidecars verified.