Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions TODO.sota/64-b3-arabic-sota.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,28 @@ the news specialist; decode policy unchanged (greedy, first-alternates).
HF Jobs, --detach, --timeout 23h, step ckpts; ≤2 GPU budget shared
with B7.
Acceptance: gates green + external protocol reconciliation (B9).


## Oracle-distill launch scoping (2026-10-29)

The WO39 chain is script-complete and ready to run: stage 1
`distill_oracle_hf.py` (300K QCRI rows, three arms, per-row oracle) and
stage 2 `continue_oracle_train_hf.py` (continue-train run-047/best 2 epochs,
SadeedDiac + WikiNews eval, gate first-alt WER < 3.5607, stretch <= 2.70).

Sole launch prerequisite: staging the three WO35 arm checkpoints
(run-047-large-strip/best — also the stage-2 init, run-040-strip-alt/best,
run-029-news-specialist/best) and the QCRI train_data mount onto HF storage.
Findings of the audit:

- The legacy Modal volumes (secryst-checkpoints et al.) list only
thai/rababa-era artifacts — no run-047/040/029.
- rhtse/jobs-artifacts holds 78 timestamped job-sync dirs (r8 era) —
candidate location, unverified.
- Recovery options if the checkpoints are gone: (a) the news specialist arm
is SHIPPED (ara-diac-news-1.0/1.1) and the r7 canonical teacher is SHIPPED
(ara-diac-2.0 lineage) — two of three arms restageable from pinned
releases; (b) run-047-large-strip/best (the stage-2 init) may need a
short re-derivation from its recorded recipe, or the stage-2 init moves
to the closest recoverable ancestor — an owner-visible design decision
recorded before any run.
27 changes: 16 additions & 11 deletions corpora/khm-latn.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,21 @@
id: khm-latn
task: transliteration
sources:
# WO88 migration: the secryst-era khm-latn training lineage, recorded
# with honest provenance markers — the original corpus predates the
# pinning contract. Decode evidence + the khmer-dict-spice source
# (Chuon Nath + SBBIC) are documented; the exact training tsv/tsv
# generation script lives in the secryst repo
# (scripts/export_onnx_byt5.py lineage). UNVERIFIED fields are the
# retrain blockers listed in WO68/88.
# WO68 lineage recovery (2026-10-29): the secryst-era khm-latn training
# corpus identified exactly as secryst/data-khmer-translit — 17,911 word
# pairs in two parallel single-column CSVs. Staged to the training bucket
# with sha256; the legacy raw.githubusercontent fetch is retired.
# Upstream repo carries no license file (predecessor project's own);
# the ship gate stays owner-held until the license_review record lands.
- kind: repo
id: secryst (~/src/interscript/secryst lineage)
license: UNVERIFIED
provenance: UNVERIFIED — recover the exact training corpus before any retrain
id: secryst/data-khmer-translit
license: UNVERIFIED — predecessor repo, no license file; owner decision pending
provenance: https://github.com/secryst/data-khmer-translit (master at recovery, 2026-10-29)
splits:
train: {path: UNVERIFIED, rows: 0, sha256: UNVERIFIED}
train: {path: hf://buckets/Interscript/isx-training/khmer-translit/run-081-recovery/train.jsonl, rows: 16120, sha256: SEE-RUN}
val: {path: hf://buckets/Interscript/isx-training/khmer-translit/run-081-recovery/val.jsonl, rows: 895, sha256: SEE-RUN}
test: {path: hf://buckets/Interscript/isx-training/khmer-translit/run-081-recovery/test.jsonl, rows: 895, sha256: SEE-RUN}
raw:
data_kh.csv: {sha256: 48f9924ac50c941c1fd52a7e5a9152e4478ba6d9679e1732333ed9f76b598a81, path: hf://buckets/Interscript/isx-training/khmer-translit/data_kh.csv, rows: 17911}
data_rom.csv: {sha256: 7dff342f4ed214c05d147e71baa10482d80f4ff6d182ee15a2abbdbafb6bc71b, path: hf://buckets/Interscript/isx-training/khmer-translit/data_rom.csv, rows: 17911}
recovery_run: run-081-khm-latn-recovery (HF job rhtse/6ae32acd41e90535df78a449, seed 42, recipe = legacy run-001)
Loading