From 1637a1347dcfffb8fe3a514288822b06e865b4cc Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Fri, 23 Oct 2026 00:57:02 +0800 Subject: [PATCH] =?UTF-8?q?docs(WO64):=20r8=20stage-2=20FINAL=20negative?= =?UTF-8?q?=20=E2=80=94=206.96/7.13=20across=20checkpoints=20kills=20the?= =?UTF-8?q?=20overfit=20hypothesis?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- TODO.sota/64-b3-arabic-sota.md | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/TODO.sota/64-b3-arabic-sota.md b/TODO.sota/64-b3-arabic-sota.md index b53de9c2..8a357e79 100644 --- a/TODO.sota/64-b3-arabic-sota.md +++ b/TODO.sota/64-b3-arabic-sota.md @@ -1,14 +1,20 @@ # WO64 — B3: Arabic to external SOTA (DER < 2.0, WikiNews < 8) -**Stage-2 verdict (2026-10-22, INTERIM — oepoch-0 control pending):** -the r8 noisy-student continuation finished all 36,321 steps (loss -0.001-0.003) but scores **DER-CE 7.13** on the SadeedDiac-25 protocol -(1,200 paragraphs, interscript-ml-tools scoring) — worse than the v1 -lineage (teacher 2.29, rung-2.0 4.82, rung-2.1 4.57) and far off the -< 2.0 target. Suspected cause: the continuation overfit the -oracle-relabeled targets (suspiciously low train loss + eval -regression). oepoch-0 (end of continuation-epoch 1) eval queued as the -control; the r8 student line ships NOTHING at this number. +**Stage-2 verdict (2026-10-23, FINAL — both checkpoints measured): NEGATIVE.** +The r8 noisy-student continuation finished all 36,321 steps (loss +0.001-0.003) but scores **DER-CE 7.13 (oepoch-1) / 6.96 (oepoch-0)** +on the SadeedDiac-25 protocol (1,200 paragraphs, interscript-ml-tools +scoring) — worse than the v1 lineage (teacher 2.29, rung-2.0 4.82, +rung-2.1 4.57) and far off the < 2.0 target. The two checkpoints +scoring the same kills the overfit hypothesis: the continuation +simply did not inherit the teacher's quality — the oracle-relabeled +targets are the bottleneck (same lesson as the Thai arm: low train +loss is not evidence when the targets are). The r8 student line ships +NOTHING; its durable infrastructure (self-contained decode, bucket +sync, chain dispatch) is the salvage. B3's SOTA path: the v1 lineage +numbers + VRR (both published) + a data-quality lever on the oracle +targets if the chain is ever revisited — the r6-morph knowledge- +injection template, not noisy-student relabeling. Status: IN_PROGRESS (r8 stage 2 training 2026-10-22, stage-3 SadeedDiac eval auto-chained; VRR closure = WO81) Goal: SadeedDiac DER < 2.0 (canonical 2.29), WikiNews multiref WER < 8