Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions TODO.sota/57-a1-authority-completeness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# WO57 — A1: authority-completeness audit

Status: PLANNED · Plane A
Goal: enumerate the UNIVERSE of authority romanization/transliteration
systems and diff it against the 293 encoded in `maps/`. Every gap is
either encoded or cited out-of-scope with an authority reference.
Method:
1. Build the catalog: UN WGRS (1968–2016 editions, ~25 systems),
BGN/PCGN current schedule (~55), ALA-LC full language tables
(~170 Romanization tables), ISO 9/233/259/3609/7098/843/9984/9985/
233-3, ICAO 9303, national systems (DIN 14640, RTGS, Korean RR
2000, HK Government, ROC, Thai Royal Institute…). Sources: the
authority documents already collected in interscript-references +
the ISO 24229 code convention.
2. Extract the corpus's system codes from maps/*.isc names; normalize
per ISO 24229; join against the catalog.
3. Emit coverage matrix (authority × system × status: encoded /
gap-encoded-possible / out-of-scope+reason) to
docs/coverage/COVERAGE.md; gaps that are encodable become maps
PRs (whole-document transcription law applies).
Acceptance: coverage matrix published with zero unexplained gaps;
enconding PRs opened for every encodable gap.
Owner gates: none for the audit; map encodings are ordinary PRs.
17 changes: 17 additions & 0 deletions TODO.sota/58-a2-invertibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# WO58 — A2: invertibility ledger + reverse maps

Status: PLANNED · Plane A
Goal: for every system, determine whether forward∘reverse = identity;
publish the loss conditions (homophone mergers, digraph ambiguity,
case conventions, vowel elision).
Method:
1. Script against the installed gem runtime: for each map with a
reverse variant (or a generated one), round-trip the map's embedded
tests + a synthetic Unicode sample per source script.
2. Classify: lossless / lossy-with-convention / not-invertible; attach
minimal distinguishing examples.
3. Where an authority publishes a reverse system: encode it (whole-
document law). Where the forward map is provably bijective:
generate the reverse from the grammar and validate.
Acceptance: docs/invertibility/LEDGER.md covering all systems; reverse
maps shipped for every losslessly invertible one; losses cited.
15 changes: 15 additions & 0 deletions TODO.sota/59-a3-equivalence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# WO59 — A3: system-equivalence checker + distinguishing corpora

Status: PLANNED · Plane A
Goal: mechanically decide/estimate whether two systems coincide, and
produce the minimal test corpus that distinguishes them.
Method:
1. Compose maps as automata over a sampled alphabet (the ruby
interpreter as oracle); bounded-equivalence search (input-length
horizon escalation).
2. Distinguishing corpus generator: for near-equivalent systems emit
the smallest input set that yields differing outputs; use it to
justify alias/dedup decisions in the corpus.
Acceptance: checker + generator shipped in the runtime toolchain;
every claimed-equivalent map pair in maps/ justified by the checker;
CI uses it to prevent accidental duplicates.
15 changes: 15 additions & 0 deletions TODO.sota/60-a4-is1-semantics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# WO60 — A4: IS-1 v2 — denotational semantics

Status: PLANNED · Plane A
Goal: complete the ISC specification with a semantics section so every
runtime behavior is derivable from the document; zero open rulings.
Method:
1. Collect every behavior where runtimes might diverge: boundary
semantics, parallel rule priority, maybe composition, capture/ref,
funcall typing, casing interactions, dependency staging order.
2. Write the denotational (or operational) semantics for the ISC
fragment in IS-1; give each ruling a test vector.
3. Property-test all three runtimes against the semantics.
Acceptance: IS-1 v2 released with the semantics section; the rulings
list emptied; each ruling has vectors in all runtimes' suites.
Depends: nothing; feeds A5.
15 changes: 15 additions & 0 deletions TODO.sota/61-a5-certification-kit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# WO61 — A5: third-party conformance certification kit

Status: PLANNED · Plane A
Goal: an implementable certification path so ANY runtime can
self-certify against Interscript.
Method:
1. Levels: L0 parse (ISC grammar conformance), L1 embedded tests, L2
frozen conformance vectors (maps + models), L3 full cross-runtime
parity on the distinguishing corpora (WO59).
2. Ship the kit as a package: vector bundles from GitHub Releases +
a runner spec + a self-certification report format (machine-
checkable).
3. Certify our three runtimes at L3 as the reference demonstration.
Acceptance: kit published; report format validated; ruby/ts/py at L3.
Depends: WO58/WO59/WO60 for L3 content.
14 changes: 14 additions & 0 deletions TODO.sota/62-b1-dispatch-doctrine.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# WO62 — B1: the dispatch doctrine as registry defaults

Status: PLANNED · Plane B
Goal: encode the technique-dispatch table — (script class, ambiguity
type, data regime) → recipe — as registry defaults so a new language
gets a recommended recipe automatically.
Method: map banked verdicts into src/pipeline defaults: abjad+lexicon
(sft + morph injection + oracle relabel on disagreeing arms); abugida
(lexicon-first + ByT5 fallback + rule composite); finite ambiguity
(rules); phonotactic targets (ByT5, never CTC); VRR reporting for all
vocalization tasks. Validate against every historical run: the doctrine
must RECOMMEND what actually won.
Acceptance: doctrine module + docs; backtest table (doctrine vs actual
verdicts) with zero contradictions unexplained.
8 changes: 8 additions & 0 deletions TODO.sota/63-b2-hebrew-sota.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# WO63 — B2: Hebrew to external SOTA (DER < 5, int8 < 6)

Status: PLANNED · BLOCKED-OWNER (data-supply asks, drafts in WO43/WO17)
Goal: ReNikud-protocol DER < 5; int8 artifact < 6 (from 8.18).
Method: plane scale-up (large arm) once data lands; learned-IPA v0
feature layer (r30/31 lines); knesset v6 weak-pretrain (Dicta
machine-label provenance per memory); VRR alongside DER.
Acceptance: external protocol numbers, protocol package published (B9).
12 changes: 12 additions & 0 deletions TODO.sota/64-b3-arabic-sota.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# WO64 — B3: Arabic to external SOTA (DER < 2.0, WikiNews < 8)

Status: PLANNED · launchable now
Goal: SadeedDiac DER < 2.0 (canonical 2.29), WikiNews multiref WER < 8
(news specialist 8.85). VRR reported.
Method: r8 noisy-student chain — synthesize with the canonical r7
teacher over large unlabeled MSA, filter by VRR-licensed consistency,
retrain with morph features (r6-morph teacher); register closure on
the news specialist; decode policy unchanged (greedy, first-alternates).
HF Jobs, --detach, --timeout 23h, step ckpts; ≤2 GPU budget shared
with B7.
Acceptance: gates green + external protocol reconciliation (B9).
10 changes: 10 additions & 0 deletions TODO.sota/65-b4-urdu-sota.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# WO65 — B4: Urdu — diac CER < 3; G2P license path

Status: PLANNED · G2P half BLOCKED-OWNER (commercial-license data)
Goal: urd-diac 3.57 → CER < 3; urd-g2p research-only → commercial-
usable data lineage.
Method: diac — oracle relabel arms only if teachers decorrelate (banked
negative: correlated arms transfer nothing); morph/lexicon features;
VRR. G2P — owner selects licensed corpus (option drafts in WO50);
retrain; relabel artifacts non-research-only.
Acceptance: external protocol numbers; license gate green on G2P.
9 changes: 9 additions & 0 deletions TODO.sota/66-b5-persian-registers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# WO66 — B5: Persian — hold < 0.7 across registers

Status: PLANNED
Goal: fas-diac 0.603 row-mean holds < 0.7 on poetry and colloquial
registers; Arabic-loan vocalization probe (the known residual).
Method: register-stratified eval sets (build from poetry corpora,
colloquial sources); probe the scar (docs 0.52% lesson: corpus-level vs
row-mean declared); targeted data if registers degrade; VRR.
Acceptance: register-stratified external-protocol numbers published.
11 changes: 11 additions & 0 deletions TODO.sota/67-b6-thai-lexicon.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# WO67 — B6: Thai — lexicon-first, three layers (PER < 2)

Status: PLANNED · launchable now
Goal: published PER < 2; lexicon hit-rate reported; tone accuracy for
the TTS layer (RTGS / IPA / IPA+tone tracked separately).
Method: Royal Institute dictionary (or equivalent licensed lexicon) as
first-class lexicon layer with hit-rate metric; ByT5 fallback for novel
tokens (names); RTGS rule composite over the inferred layer (banked:
CTC cannot); tiny contract for the edge tier.
Acceptance: three-layer eval published; hit-rate + PER external-
protocol certified.
10 changes: 10 additions & 0 deletions TODO.sota/68-b7-khmer-lexicon.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# WO68 — B7: Khmer — Chuon Nath lexicon conditioning (CER < 15)

Status: PLANNED · launchable now (data on disk: khmer-dict-spice)
Goal: khm-latn 27.4 → CER < 15 with hit-rate reported.
Method: build the Chuon Nath-conditioned dataset (headword→reading
pairs, syllabification-aware); lexicon-conditioned ByT5 (retrieval or
prefix conditioning); fp16 zip hygiene (khm fp32 corruption lesson);
conformance fixtures for the new artifact.
Acceptance: gates green; hit-rate + CER on held-out; release via the
strict publish path.
10 changes: 10 additions & 0 deletions TODO.sota/69-b8-community-intake.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# WO69 — B8: prove the contribution story (3 community languages)

Status: PLANNED
Goal: three external-contributor languages go end-to-end through the
2-YAML path (corpora + recipe), landing as shipped models without
framework forks.
Method: pick 3 candidate languages with licensable corpora and clear
ambiguity classes; run the doctrine (WO62) to recommend recipes;
execute via the spine; document friction; fix the spine, not the fork.
Acceptance: 3 languages shipped; friction log resolved into spine PRs.
12 changes: 12 additions & 0 deletions TODO.sota/70-b9-external-benchmark-cert.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# WO70 — B9: external-benchmark certification

Status: PLANNED
Goal: no self-graded SOTA claims — every headline reconciles to an
external leaderboard/protocol package.
Method: per language, pin the external protocol (Arabic: SadeedDiac +
ALTensor; Hebrew: ReNikud; Thai: PyThaiNLP G2P suite; Persian: published
papers' splits; Urdu: published baselines) as protocol packages in
interscript-ml-tools; publish prediction files for every claim (the
frontier-predictions release pattern); multiref handled per WO01
lessons with VRR.
Acceptance: claims matrix where every row links predictions + protocol.
11 changes: 11 additions & 0 deletions TODO.sota/71-c1-conformance-matrix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# WO71 — C1: model × runtime conformance matrix in CI

Status: PLANNED
Goal: every shipped model byte-exact in ruby/ts/py, CI-enforced, forever.
Method: fixtures already frozen in conformance/ (12 rows each; sidecar
on the release). Wire each runtime's CI to download the fixture set
from the models Releases (data channel law) and run the full matrix:
all shipped models, not just the ones curated per-repo. A red cell
blocks merge like any test.
Acceptance: matrix visible in CI summary (model × runtime grid, all
green); new model publication extends the matrix automatically.
10 changes: 10 additions & 0 deletions TODO.sota/72-c2-metric-convention-gate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# WO72 — C2: gates reject undeclared metric conventions

Status: PLANNED
Goal: quality gates refuse a metric that doesn't declare its convention
(row-mean vs corpus-level) and anchor.
Method: extend the gate registry (WO55 pattern): MetricDeclaration
required on every metrics file consumed by a gate; validate in
`isxml gate` and publish guard. Spec: real instances, no doubles.
Acceptance: gate tests proving rejection; all existing recipes updated
with declarations (they are, per WO53 — verify).
11 changes: 11 additions & 0 deletions TODO.sota/73-c3-perf-budgets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# WO73 — C3: performance budgets in CI

Status: PLANNED
Goal: decode latency + artifact size budgets enforced so regressions
block like test failures.
Method: measure current shipped baselines (single-row greedy, p50/p95
per model, mobile-typical CPU); encode budgets in models.yaml metadata
(perf: {latency_p50_ms, size_mb}); runtime CI asserts the budget on the
conformance fixture set; budget changes are owner-visible recipe edits.
Acceptance: budgets in the index; CI green under budget; a synthetic
regression fails CI (proven).
10 changes: 10 additions & 0 deletions TODO.sota/74-d2-spec-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# WO74 — D2: IS-1/ISC as a citable standard

Status: PLANNED · Depends WO60
Goal: versioned spec releases with a citation block; the spec becomes
an artifact others implement against.
Method: IS-1 v2 (WO60) tagged as spec release v2.0 on the canonical
repo; spec PDF/HTML artifacts attached; citation block (BibTeX) in the
README + site spec page. D1 (PyPI publisher) remains tracked in
WO53-ship-set; do not nag.
Acceptance: spec release tagged; citation renders on the site.
10 changes: 10 additions & 0 deletions TODO.sota/75-d3-catalogue.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# WO75 — D3: catalogue completeness on the site

Status: PLANNED
Goal: every shipped object (293 maps, all index-v12 models, three
runtimes + API status) browsable on interscript.org.
Method: audit the site's map/model catalogue against models.yaml +
maps corpus; fill gaps; add runtime status cards (versions, conformance
level per WO61).
Acceptance: catalogue diff = zero; automated freshness check (site
build pulls the index and fails on drift).
14 changes: 14 additions & 0 deletions TODO.sota/76-e1-paper-formalism.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# WO76 — E1: paper — the ISC formalism, corpus, and interoperability

Status: PLANNED · Depends WO57/WO60
Goal: a complete draft (LREC/COLING-class) presenting ISC as an
executable formalism for authority script conversion; the 293-system
corpus; the differential-testing interoperability methodology; the
conformance regime.
Method: sections — intro (conventions as unverifiable prose), the
formalism, corpus construction + transcription discipline, the
interoperability contract (parity methodology), evaluation (coverage
WO57 + case studies), related work (FST tooling, romanization surveys).
Owner reviews the draft before any external submission (external-
communication law).
Acceptance: full draft in docs/papers/; owner-reviewed.
11 changes: 11 additions & 0 deletions TODO.sota/77-e2-paper-dispatch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# WO77 — E2: paper — technique dispatch for underdetermined mappings

Status: PLANNED · Depends WO62 + plane B results
Goal: a complete draft presenting the dispatch doctrine: ambiguity
class → technique, with the negative-results registry (RL, CTC,
LLM-teachers, correlated distillation) as a first-class contribution
and VRR as an evaluation proposal.
Method: results from B2-B7; backtest table from WO62; the two
structural rules (vocalization-as-composition, abugida lexicon-first)
as the conceptual frame.
Acceptance: full draft with the results matrix; owner-reviewed.
9 changes: 9 additions & 0 deletions TODO.sota/78-e3-benchmark-artifacts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# WO78 — E3: frozen benchmarks as citable artifacts

Status: PLANNED
Goal: eval protocols + conformance suites published as citable,
versioned artifacts.
Method: protocol packages (WO70) + conformance vectors bundled as
tagged releases with per-file sha256 sidecars (data-channel law) and a
citation block per bundle.
Acceptance: bundles tagged; citations render; sidecars verified.
Loading