docs+index: WO39-45 campaign wave (oracle-distill, teacher re-eval, release plan; staged sha-verified model entries)

TODO.sota/00-README.md

@@ -43,32 +43,49 @@ hf jobs run --flavor a100-large -d \
 | 03 | 03-thai-hybrid-latency.md | P1 | — | DONE (py#31, RESULTS #258) |
 | 04 | 04-hebrew-plane-scaleup.md | P2 | #44 | gate CLEARED — artifact DER 8.18 (int8, runtime protocol); release pending owner version |
 | 05 | 05-external-benchmarks.md | P2 | 01, 07 | DONE (RESULTS #259; MILIM awaiting release) |
-| 06 | 06-arabic-r8-noisy-student.md | P2 | #44 | jobs LAUNCHED (label + train, HF queue) |
+| 06 | 06-arabic-r8-noisy-student.md | P2 | #44 | CLOSED — dose-response mapped (RESULTS wave 2); teacher-quality lesson became the WO31 pattern |
 | 07 | 07-nikud-ipa-rule-layer.md | P3 | — | DONE (py#29, py#30) |
-| 08 | 08-thai-small-tier.md | P3 | #44 | job LAUNCHED (run-025, l4x4) |
+| 08 | 08-thai-small-tier.md | P3 | #44 | CLOSED — see 12 (sub-10MB tier negative) |
 | 09 | 09-hebrew-audio-pseudolabel-probe.md | P3 (parked probe) | — | DONE as parked probe (script only) |
 | 10 | 10-heb-plane-2-release.md | P1 | 04 | DONE (release, index-v9, 3 pin bumps, card, golden, HF mirror #28) |
-| 11 | 11-arabic-r8-chain.md | P1 | — | jobs RUNNING (a100) |
+| 11 | 11-arabic-r8-chain.md | P1 | — | CLOSED — r8a/b/c/d verdicts in RESULTS waves 2-4; specialist shipped |
 | 12 | 12-thai-tiny-contract.md | P2 | — | NEGATIVE (caveated) — PER 2038%; sub-10MB closed |
 | 13 | 13-thai-lexicon-release.md | P2 | 03 | DONE (tha-lexicon-kaikki-1.0 + sha, CC BY-SA) |
 | 14 | 14-ts-plane-port.md | P2 | — | DONE (ts#99/#100/#101, CI leg models#265) |
 | 15 | 15-api-edge.md | P2 | 14 | DONE (api#29: edge-first /v1/infer, kind in index) |
 | 16 | 16-parking-lot.md | — | — | parked (not now) |
-| 17 | 17-arabic-register-closure.md | P1 | 11 | dose-response MAPPED (RESULTS wave 2); r8d knee-seeking RUNNING; specialist option staged |
-| 18 | 18-parity-runs.md | P1 | 10 | pending |
+| 17 | 17-arabic-register-closure.md | P1 | 11 | CLOSED — register-specialist doctrine; specialist shipped; numbers superseded by WO35 corrected protocol |
+| 18 | 18-parity-runs.md | P1 | 10 | DONE via WO10+WO23 — golden attached, 3-leg parity GREEN |
 | 19 | 19-lexicon-loader.md | P2 | 13 | DONE (py#33) |
 | 20 | 20-hebrew-large-arm.md | P1 | 04 | NEGATIVE — 8.48 vs base 8.18; base stays shipped |
 | 21 | 21-arabic-plane-transfer.md | P2 | 17 | VERDICT: OOD −3.14 WER, ID +1.66 (Pareto) — specialist candidate staged |
 | 22 | 22-runtime-releases.md | P1 | 14 | PREPARED — owner-gated versions |
 | 23 | 23-ort-alignment.md | P1 | 18 | CLOSED — characterization complete; 3-leg parity GREEN at documented bound |
-| 24 | 24-verdict-wave.md | P1 | 17 | r8a/b/c + run-026 + thai CLOSED; r8d (knee) RUNNING |
+| 24 | 24-verdict-wave.md | P1 | 17 | CLOSED — all verdict waves recorded in RESULTS |
 | 25 | 25-ar-news-specialist.md | P1 | 17 | VERDICT: OOD 10.13/8.98 (crown), ID 5.50; export staged |
 | 27 | 27-ar-large-id-arm.md | P1 | — | NEGATIVE — 2.5765 parity with r7; capacity closed on AR too |
 | 28 | 28-oracle-complementarity.md | P2 | — | CLOSED — ceiling 96.78 real; naive voting catastrophic |
 | 30 | 30-heb-learned-ipa-v0.md | P1 | 09 | NEGATIVE — CER 1.18; ASR teacher is the bottleneck |
-| 31 | 31-heb-phoneme-asr-v1.md | P1 | 30 | SPEC'D — tune the phoneme CTC on Hebrew first (owner block) |
-| 32 | 32-matched-architecture-arms.md | P1 | 17 | BiLSTM DONE (14.30/10.10 — arch-class refuted); run-033 plane RUNNING |
+| 31 | 31-heb-phoneme-asr-v1.md | P1 | 30 | stage 2 CLOSED at 0.546 nCER (capacity negative, metric/conventions fixed); rules 0.2393 stands; ivrit.ai $500 = owner decision |
+| 32 | 32-matched-architecture-arms.md | P1 | 17 | BiLSTM 14.30 refuted; run-033 10.78/9.16 gate-MISS — specialist dominates; successor = word-channel class |
 | 26 | 26-heb-noisy-student.md | P1 | 04 | NEGATIVE — 8.72 vs 8.18; all three text levers closed |
+| 33 | 33-lexicon-disambiguator.md | P1 | 17/32 | NEGATIVE — 18.07/11.44; word-only can't diacritize OOV |
+| 34 | 34-hybrid-word-char.md | P1 | 32/33 | CLOSED — hybrid 13.53, fastText 13.59, vcd ±0, oracle 0.71 (all old-protocol; see 35) |
+| 35 | 35-alternate-protocol.md | P1 | 34 | arm 2 PASS — large-strip 3.5983/1.4715 NEW BEST (0.90 to frontier); 1.1+1.2 staged; 3-way oracle ceiling 2.9484 |
+| 36 | 36-ivrit-scale.md | P1 | 31 | e12 CONVERGED 0.4462 (+8ep bought nothing) — undertraining refuted; residual structural; label-filter arm next, teacher v4 last resort |
+| 37 | 37-portfolio-sota.md | P2 | — | REFERENCE NUMBERS PUBLISHED — fas 1.47 / urd-diac 3.86 / urd-g2p 15.78 CER (n=3000, protocol+artifacts sha'd); urd-g2p = the one future arm |
+| 38 | 38-urd-g2p-base.md | P1 | 37 | GATE PASS 0.1565 < 0.1578 (12.7K full split); export+parity RUNNING; staged urd-g2p-base-staged |
+| 40 | 40-heb-teacher-reeval.md | P1 | 36 | VERDICT 0.1382 TRUE (was 0.87 bug) — student-bottleneck; teacher v4 demoted to last resort |
+| 39 | 39-ar-oracle-distill.md | P1 | 35 | RUNNING — stage-1 relabel on l4x1 (chunk-resume, 9 tests green); stage-2 continue-train queued on a100 |
+| 41 | 41-urd-diac-base.md | P2 | 38 | STAGED — script in bucket; last in a100 queue (after WO39 stage 2) |
+| 42 | 42-heb-label-filter.md | P2 | 40 | STAGED (conditional — fires if e12 misses; self-consistency filter, 10 GPU-h) |
+| 43 | 43-heb-gold-nikud.md | P2 | 36/42 | BLOCKED — dataset has NO license (all-rights-reserved default); owner-to-owner ask to micsell required |
+| 44 | 44-release-plan.md | P1 | owner | PROPOSAL — naming law + publication order + decisions D1-D5 (owner states, I execute) |
+| 45 | 45-large-static-int8.md | P2 | 44 | PARKED — 3 toolchain walls (ORT-static, fp16-refuse, torch-dynamic trace); large tier = measurement; artifact needs external-data-in-zip (runtime-arch) |
+| 46 | 46-int8-artifact-remediation.md | P1 | 44 | ACTIVE — zipcheck under test; 1.0 fp32 re-export queued; release-asset replacement path defined |
+| 47 | 47-fas-diac-release.md | P1 | — | NEW GAP FOUND — fas-diac trained at 0.52% CER, NEVER released; checkpoint on a Modal volume; probe queued |
+| 48b | (48 ext) | P2 | 48 | he-wiki DUD (1,244 pairs); pivot spec'd: he.wikisource CC BY-SA (10-50K gold pairs), queued behind fleet drain |
+| 49 | 49-ur-lexicon-probe.md | P2 | 38 | DEAD — lexicon-oracle 0.2268 WORSE than model 0.1565 on 30% coverage; lever closed at zero GPU |
 
 ## Schedule
 
@@ -77,34 +94,42 @@ Wave 2 (HF training, launched together): WO04 (a100-large), WO06
 (a100-large), WO08 (l4x1) — run concurrently, watch via `hf jobs ls`.
 Wave 3: WO05 tables after 01+07 land; WO09 probe only.
 
-## Path-to-success graph + SOTA matrix (2026-10-07, live)
+## Path-to-success graph + SOTA matrix (2026-10-14, post-WO38/39)
 
 ```
-ARABIC ── in-domain (SadeedDiac-25)        HEBREW ── nikud (nakdimon)
-  r7 2.2864 [BEST dedicated]                 heb-plane-2.0 8.18 [BEST text-only]
-  Claude-3.7 1.39 (API LLM; hallucinates)    next: us (12.48) → 16.4 → 35.6
-    │                                          │
-    └─ r8a (self-labeled arwiki) RUNNING       └─ byt5-large/K=4 (untried)
-    └─ r8b (QCRI silver)       RUNNING       HEBREW ── g2p IPA (their board)
-    └─ gate: hold ID ≤2.2864 AND              rules chain 0.2393 CER (0-train)
-       move OOD 17.38 → ≤10                   ReNikud 0.0244 (audio-supervised)
-                                              └─ learned IPA head: BLOCKED
-ARABIC ── OOD (WikiNews-2024 multiref)          (espeak ceiling 0.4543; needs
-  r7 17.38/11.83 · plane 18.69/11.35            MILIM or audio = 2027)
-  QCRI 2.70 (in-domain-trained silver)
-    └─ register closure = r8a/r8b above      THAI ── g2p (kaikki protocol)
-                                              hybrid 0.1478 PER @0.73ms [BEST]
-ARABIC plane ── 2.7397 (large) ── r8 recipe    tiny student RUNNING (<10MB tier)
-   transfers to plane family post-r8        CLIENT SURFACE — 3-runtime parity ✓
-                                              + preserve mode ✓ + edge /v1/infer ✓
-                                              + index-v9 + 28 sha-verified models
+ARABIC ── WikiNews-2024 multiref (OOD news, WER/DER, first-alt protocol)
+  ladder: shipped 4.10 > base-strip 4.00 > large-strip e6 **3.5607/1.4588**
+  frontier 2.70 → gap 0.86; 3-way oracle ceiling 2.9861 (routing closed <2)
+  → WO39 ORACLE-DISTILL (the capture path): relabel 300K QCRI rows with
+    per-row best arm → continue-train e6 → gate < 3.5607, stretch <= 2.70
+  runtimes return primary choice (py#35 / ts#108 / ruby#805)
+
+ARABIC ── SadeedDiac-25 (ID): r7 2.2864 [BEST dedicated]; specialist
+  5.4814 first-alt rescored — ID was never inflated (artifact OOD-only)
+
+HEBREW ── nikud DER: 8.18 crown (2x runner-up); all text levers closed
+HEBREW ── g2p CER (the open frontier): rules 0.2393 · ReNikud 0.0244
+  audio chain (WO36) now END-TO-END CONSISTENT: v3-teacher labels 200h
+  → student 4ep nCER 0.4415 (first valid number; beat FLEURS-student 0.546)
+  → e12 +8-epoch continuation RUNNING (undertrained hypothesis)
+  → if miss: label filter, then 500h slice (building now) → teacher v4
+URDU ── g2p: run-059 base GATE PASS 0.1565 < 0.1578 (1.0) on full 12.7K
+  split; int8 export + torch/ORT parity in flight; staged
+  urd-g2p-base-staged (owner names release)
+THAI ── PER: hybrid 0.1478 @0.73ms [crown]
+FAS ── g2p 1.47 full-split [anchor beaten 25x]
+CLIENT SURFACE: 3-runtime parity ✓ · preserve ✓ · edge /v1/infer ✓
+  index-v10 · 29+ HF mirrors
 ```
 
 | axis | ours | 2026 frontier | status |
 |---|---|---|---|
-| AR in-domain DER | 2.2864 | 1.39 (LLM API) | #1 dedicated; r8 arms closing |
-| AR OOD multiref WER | 17.38 | 2.70 (their-domain) | register gap; r8a/r8b attacking |
-| HE nikud DER | 8.18 | — (we lead) | crown held; base+K=3 |
-| HE g2p CER | 0.2393 (rules) | 0.0244 (audio) | blocked on supervision (2027) |
-| TH PER | 0.1478 | 26.9 (latency play) | crown held + hybrid latency tier |
+| AR in-domain DER | 2.2864 | 1.39 (LLM API) | #1 dedicated (LLM = no-teacher law) |
+| AR OOD multiref WER | **3.5607** (DER 1.4588) | 2.70 | 0.86 gap → WO39 distill READY |
+| HE nikud DER | **8.18** | — (we lead) | crown held |
+| HE g2p CER | 0.4415 (audio, e12 running) | 0.0244 (ReNikud) | OPEN — undertrained→scale levers queued |
+| UR g2p CER | **15.65** full-split (base) | — | GATE PASS; export+parity in flight |
+| UR diac CER | 3.77 full-split | — | shipped |
+| FAS g2p CER | 1.47 full-split | — | shipped |
+| TH PER | **0.1478** | 26.9 (latency play) | crown held + latency tier |
 | client surface | 3 runtimes + edge | none exists | moat maintained |

TODO.sota/31-heb-phoneme-asr-v1.md

@@ -15,3 +15,16 @@ ReNikud recipe's first stage):
 
 Cost: one fine-tune (~2-4 GPU-hours) + relabel + v1 train. Owner
 green-light for the block.
+
+## Execution (green-light received 2026-10-10)
+
+- run-038-heb-asr-ft RUNNING (stage 1): fine-tune as spec'd; espeak-ng
+  targets (do_phonemize disabled — targets are pre-phonemized; the
+  en-us default would silently mis-label), in-memory corpus, per-phone
+  dev PER gate < 0.395, step-ckpts every 2000, --timeout 4h.
+- run-038 stage-1 VERDICT (r4, 47m): dev PER 0.2306 < 0.395 — GATE
+  PASS (teacher bottleneck broken; 42% over universal).
+- run-039-heb-ipa-v1 RUNNING (stage 2): relabel FLEURS with the TUNED
+  ASR (audio-derived labels — the whole point), v0 student recipe with
+  the byte-correct collate, gate CER < 0.2393 on phonikud
+  heb-g2p-benchmark (gt.tsv from run-032).

TODO.sota/33-lexicon-disambiguator.md

@@ -0,0 +1,41 @@
+# WO33 — the word-identity channel: variant-disambiguation tagger
+
+Decomposition (2026-10-09, local probe, same corpus):
+
+| arm | WikiNews-2024 WER | note |
+|---|---|---|
+| pure most-frequent-reading lookup | 25.15 (DER 13.18) | no model; 482,609-type table; OOV 18.10% |
+| literal char BiLSTM (run-034) | 14.30 (DER 10.10) | char context, no word channel |
+| byt5-base seq2seq specialist (run-029, shipped) | 10.13 (DER 8.98) | implicit lexicon via byte pretraining |
+| byt5-large plane + 900K silver (run-033) | running | their recipe class, stronger backbone |
+
+Lexical memorization is 3/4 of the task; the remaining ~25 points are
+context disambiguation among a word's few observed readings — the word
+channel char-level models approximate only indirectly. QCRI's own
+design (Fadel) is word-level: word-embedding + char-CNN + BiLSTM over
+variant indices.
+
+Arm: run-035 — variant-disambiguation tagger on the same corpus as
+run-033/034. Word-emb (128) + char-CNN (k=2..5, 50 filters each) ->
+BiLSTM 2x512 -> head over {variant idx 0..5, OOV}. Labels from the
+variant table (most-frequent = 0). Decode emits the chosen variant
+string; output preserves the input skeleton letter-exactly. fp32
+(LSTM bf16 = NaN). ~62M params, bs 128, 5 epochs, a100, ~2h.
+
+Gates:
+- WN-2024 multiref < 14.30 (beats the char-only BiLSTM) → the word
+  channel is real at our scale.
+- WN-2024 < 10.13 (beats the shipped specialist) → becomes the
+  ara-diac-news successor; then stack SOTA layers (K-pass, larger
+  backbone, preserve mode).
+- Fails both → the gap is not the word channel either; remaining
+  hypothesis is corpus-internal convention alignment, closed by the
+  register-specialist doctrine (crown = ID specialist + news
+  specialist shipped separately).
+
+Ship rule: same as WO32 — dominant arm on WN-2024 becomes the news
+successor; export via int8 IMF v1 chain only if it beats 10.13.
+
+Complement probe (post-verdict): oracle-min over {run-029, run-035}
+bounds the ensemble ceiling; if >2 points over the best single arm,
+a routing/blending follow-up (WO28 lineage) opens.

TODO.sota/34-hybrid-word-char.md

@@ -0,0 +1,54 @@
+# WO34 — the synthesis arm: char-level BiLSTM + word-embedding feature
+
+run-035 (WO33) verdict, 2026-10-10: word-level variant-disambiguation
+tagger scored **18.07 / 11.44** (WikiNews-2024 multiref) and Sadeed
+Total DER **18.75** — both gates FAILED (<14.30, <10.13). Diagnostic
+value: the word-only design emits bare (undiacritized) words for the
+18.1% OOV types, which alone collapses ID DER; the char-only BiLSTM
+(run-034, 14.30/10.10) composes unseen words but has no word-identity
+channel. Each half in isolation loses to the shipped specialist
+(10.13/8.98).
+
+Convention-style hypothesis CLOSED same day (local probe): marked-letter
+density is 0.8127 (WikiNews-2024 refs, first-alt) / 0.7625
+(wikinews2014 gold) / 0.7432 (news silver) — same full-diacritization
+family; no partial-vs-full style mismatch to harvest.
+
+The arm: run-036 — Fadel's actual joint design at our scale. Char-emb
+128 ⊕ per-char word-embedding (bare-form vocab, isalpha core,
+300K cap) → 3×BiLSTM-512 → per-position plane-combo head. Same corpus
+as run-033/034/035, fp32, bs 64, 6 epochs, a100.
+
+Gates:
+- WN-2024 multiref < 14.30 → word channel validated at char level;
+  hybrid becomes the stacking backbone.
+- WN-2024 multiref < 10.13 → news-successor candidate; stack SOTA
+  layers (K-pass conditioning, larger corpus, preserve mode) and run
+  the IMF v1 export chain.
+- ≥ 14.30 → word-identity is NOT the residual gap at our data scale;
+  remaining hypotheses: tuning maturity (lr/epoch sweep), their
+  evaluator-side conventions (closed), or data scale beyond 900K
+  silver units. Doctrine stands.
+
+Staged follow-ups (armed post-verdict):
+1. Variant-constrained decode — DONE, closed (2026-10-10): ID-only
+   tweak (Sadeed 5.90→5.75), multiref bit-identical to free decode
+   (13.5268/9.9211). Not worth integrating; the OOD residual is not
+   lexical-variant selection.
+2. Oracle-min complement probe — DONE, CLOSED NEGATIVE (2026-10-10):
+   regen of run-029 reproduced 10.032/8.9521 exactly; oracle over
+   {specialist, hybrid} = 9.33/8.77 — 0.71 WER over best solo,
+   below the >2 threshold. Errors overlap; routing opens nothing.
+3. run-037 (STAGED, script uploaded + hardened): run-036's
+   architecture with the scratch word channel replaced by pretrained
+   fastText cc.ar.300 vectors (in-job download, coverage-reported,
+   trainable init; ft_init cache + step-checkpoint resume) — the
+   lexical-density hypothesis: our word channel is undertrained on
+   ~3M words, not wrong; Fadel-class systems ride on billion-word
+   vectors. Same gates. Launches when a GPU slot frees regardless of
+   run-036's outcome (independent axis: scratch vs pretrained).
+4. Tuning-maturity sweep — READY: hybrid trainer now exposes
+   HYB_LR / HYB_HID env knobs; grid {lr 1e-3/5e-4} × {hid 512/768} ×
+   {epochs 6/10} queued after the three primary arms read out.
+5. If run-033 (byt5-large plane) lands <10, race it against run-036/37
+   gates; the dominant arm takes the successor slot.

TODO.sota/35-alternate-protocol.md

@@ -0,0 +1,54 @@
+# WO35 — the alternate-format scoring artifact + the re-opened 1.4-point program
+
+## Discovery (2026-10-10)
+
+Error-typography of the specialist's 1,065 wrong WikiNews words: 74.2%
+length mismatches, 1.4% vowel-class — inverted from expectation. Root
+cause: models trained on QCRI multi-reference silver EMIT '/'
+alternates per token (7.44% of tokens; Sadeed rows too: 12.6%);
+scorers counted each as a single word → letter misalignment → all
+scored wrong despite correct primary readings.
+
+Corrected protocol (first-alternate decode — deterministic,
+shippable, same benchmark):
+
+| arm | prior | first-alt |
+|---|---|---|
+| run-029 specialist (shipped ara-diac-news-1.0) | 10.03/8.95 | **4.10/1.56** |
+| run-036 hybrid | 13.53/9.92 | 8.85/2.84 |
+| symmetric any-vs-any (specialist) | — | 3.18 WER |
+
+Rankings from WO32-34 hold (dominance unchanged); absolute numbers
+shift. models.yaml metrics corrected. Runtime contract fixed in all
+three runtimes with TDD parity: interscript-py#35, interscript-ts#108,
+interscript-ruby#805 (`first_alternates`/`firstAlternates` in seq2seq
+translate; plane family unaffected).
+
+## Answer to "how do we get their silver + conventions"
+
+We already HAVE the conventions — the corpus taught our model to emit
+them (that was the artifact). Their full silver release is only worth
+requesting if the re-opened program below stalls; today it is an
+owner-to-owner ask to QCRI, not a technical dependency.
+
+## Remaining work
+
+1. DONE — Sadeed rescore: 5.4814 (was 5.5008); ID was never inflated;
+   the artifact was OOD-only (−5.9 WER there vs −0.02 DER here).
+2. Re-open the OOD program at 4.10 vs 2.70 (1.4 WER):
+   - run-040 RUNNING: the specialist retrained on first-alt-STRIPPED
+     corpus (identical recipe to run-029 — R8_EPOCHS=4, GOLD=none,
+     MIX=qcri, cap 900K, init r7-best — plus R8_STRIP_ALT=1; targets
+     and inputs both stripped; in-job eval reports raw AND first-alt
+     with project_haraqat on the WikiNews path). Gate: first-alt
+     WER < 4.10. If the alternate emission was wasted capacity, this
+     wins directly; if the model NEEDS the alternates as implicit
+     uncertainty, it loses — either way the mechanism is measured.
+   - next arms: byt5-large seq2seq specialist (untried in the dominant
+     family); hybrid-2 + sweep (knobs wired).
+   - oracle probe re-run under corrected protocol once new arms land.
+3. Card text for ara-diac-news-1.0 mirror (HF repo 29) + release
+   notes: decode-contract change → runtime version bumps are the
+   OWNER's decision (artifact bytes unchanged; sha valid).
+4. Derive the 1.56 DER vs published frontier DER comparisons for the
+   external-benchmark table (WO05) once Sadeed rescore lands.