From d9c9311c261b35657dda116bfb535a6a8a7c5e44 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Thu, 22 Oct 2026 14:15:46 +0800 Subject: [PATCH 1/2] =?UTF-8?q?docs:=20models-ready=20section=20reflects?= =?UTF-8?q?=20shipped=20state=20=E2=80=94=20runtime=20status,=20index=20pi?= =?UTF-8?q?ns,=20artifact=20notice?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Runtime status table: ts 5.9.0 current (pins index-v12), ruby 3.2.0 decode caveat (predates 512-step cap + UTF-8 scrub — long outputs truncate), python reference loader current (PyPI pending D1). - All three loaders pin index-v12 by default; the env override is no longer required — stale 'archived crystal' guidance removed. - Artifact notice: urd/fas fp16 re-published 2026-10-21 with fp32 graph IO (WO79); stale cached indexes fail loudly and must re-fetch. - Not-yet-ready section: r8/khm/tha training arms; research-only and legacy models flagged. - Fixed stale secryst-era references: reference loader location, Python-runtime row, ruby 'archived crystal' note, index-v5 → index-v12. --- README.md | 39 ++++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 8fa5a3bd..2f4b0dec 100644 --- a/README.md +++ b/README.md @@ -111,14 +111,20 @@ blocked upstream, not on models. Artifacts are plain public downloads; any IMF v1 runtime works (single-row greedy decode: ids = byte + 3 with trailing EOS — `conformance/CONFORMANCE.md`). The reference python loader is -`runtime/src/interscript_ml` (conformance-proven). +`runtime/src/interscript_ml` (conformance-proven, source of the frozen +fixtures). -Point a crystal's index at the current release and resolve by id: +Runtime status: -```sh -export SECRYST_INDEX="https://github.com/interscript/interscript-models/releases/download/index-v12/models-index.yaml" # python crystal -export INTERSCRIPT_ML_INDEX="$SECRYST_INDEX" # ruby gem (archived: override is required) -``` +| runtime | release | state | +|---|---|---| +| Python | reference loader in this repo (`runtime/src/interscript_ml`) | current; PyPI distribution pending publisher registration | +| TypeScript | `interscript` npm **5.9.0** | current — behaviorally identical to main, conformance matrix green; pins `index-v12` (no env override) | +| Ruby | `interscript` gem **3.2.0** | decode caveat — predates the 512-step cap and the UTF-8 scrub: outputs truncate at ~100 tokens and invalid UTF-8 raises instead of U+FFFD. Fine for short strings; for long documents use the reference loader or npm until a patch release | + +All three pin `index-v12` by default — no override needed for the +models above. Python (`SECRYST_INDEX`) and ruby +(`INTERSCRIPT_ML_INDEX`) accept an override to track a newer index. ```python from interscript_ml import Model # reference runtime (this repo) @@ -126,9 +132,24 @@ m = Model.load("urd-diac-base-1.0") # resolves via the index, verifies sh print(m.translate("کلیدهای پرکینزمانند")) ``` -Release pins move per index tag (v11: urd-diac-base; v12: urd-g2p-base, -fas-diac-base). Crystal releases re-pin on their own cadence — until -then the env override above is the supported path. +Artifact notice (2026-10-21): `urd-diac-base-1.0` and +`fas-diac-base-1.0` were re-published with exporter-side fp16→fp32 +graph-IO normalization — same weights, portable graph boundary (node +ONNX Runtime no longer corrupts fp16 bytes; the ruby loader, which +lacks Float16 materialization, decodes them). The re-published digests +are in `index-v12`; an index fetched before that date fails loudly on +sha256 verification and must be re-fetched. + +## Not yet ready + +In training, not shipped: the Arabic r8 noisy-student chain (stage 2), +the Khmer lexicon-conditioned arm, and the Thai lexicon-first arm — +each ships only after its stage-3 external-benchmark verdict clears +the gates. `urd-g2p-base-1.0` ships but is **research-only** +(non-commercial training data). `models.yaml` also carries +legacy/superseded generations (e.g. `ara-diac-1.0`, `layerdrop`, +`ara-diac-small-1.0/2.0`) that resolve via the index but are not +recommended for new use; the table above is the recommended set. ## Repositories From 8167e84acff0e8d211769f470022510ed24289fc Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Thu, 22 Oct 2026 14:21:20 +0800 Subject: [PATCH 2/2] =?UTF-8?q?docs:=20fix=20remaining=20secryst-era=20ref?= =?UTF-8?q?erences=20=E2=80=94=20loader=20location,=20runtime=20row,=20rub?= =?UTF-8?q?y=20note,=20index=20pin?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 2f4b0dec..4c782933 100644 --- a/README.md +++ b/README.md @@ -21,8 +21,8 @@ This repo owns three things and nothing else: model ids against (with per-artifact sha256s and split-part support). 2. **The IMF v1 model-zip format** — the artifact contract: `metadata.yaml` + ONNX graphs + member sha256 manifest. The normative - text is [SPEC.md](SPEC.md); the reference loader now lives in the - [Python crystal](https://github.com/secryst/secryst-py). + text is [SPEC.md](SPEC.md); the reference loader is [`runtime/` in this + repo](https://github.com/interscript/interscript-models/tree/main/runtime). 3. **The model zoo + publish pipeline** — teachers from [secryst-train](https://github.com/secryst/secryst-train) are distilled, gated (parity written into the artifact), and released @@ -34,10 +34,10 @@ This repo owns three things and nothing else: |---|---| | Benchmark | [Misraj/SadeedDiac-25](https://huggingface.co/datasets/Misraj/SadeedDiac-25) (auto-fetched) | | Protocol CLI | `pip install interscript-ml-tools[sadeed]` ([PyPI](https://pypi.org/project/interscript-ml-tools/)) | -| Python runtime | `pip install secryst` ([PyPI](https://pypi.org/project/secryst/)) | +| Python runtime | [`runtime/`](runtime/) in this repo — the reference loader (PyPI distribution pending publisher registration) | | Raw predictions, every frontier run | Release [`frontier-predictions-v1`](https://github.com/interscript/interscript-models/releases/tag/frontier-predictions-v1) | | Teachers + provenance labels | Release [`teachers-arabic-v1`](https://github.com/interscript/interscript-models/releases/tag/teachers-arabic-v1) | -| Student artifacts | `models.yaml` (index-v5) — sha256-verified in every runtime | +| Student artifacts | `models.yaml` (index-v12) — sha256-verified in every runtime | interscript-sadeed-eval score --preds --data Misraj/SadeedDiac-25 @@ -157,7 +157,7 @@ recommended for new use; the table above is the recommended set. |---|---| | [interscript/interscript-models](https://github.com/interscript/interscript-models) | this — contract + zoo + isxml framework | | [interscript/interscript](https://github.com/interscript/interscript) | ISC map DSL (the transliteration contract) | -| [interscript/interscript-ruby](https://github.com/interscript/interscript-ruby) | Ruby ISC gem (maps; the ML-model crystal is archived, see below) | +| [interscript/interscript-ruby](https://github.com/interscript/interscript-ruby) | Ruby ISC gem — maps + `Interscript::ML` model runtime | | [interscript/interscript-py](https://github.com/interscript/interscript-py) | Python crystal — live | | [secryst/secryst](https://github.com/secryst/secryst) · [secryst/secryst-py](https://github.com/secryst/secryst-py) · [secryst/secryst-ts](https://github.com/secryst/secryst-ts) | **archived** original crystals (est. 2020; the ML ruby gem's default index URL is dead — override `INTERSCRIPT_ML_INDEX`) | | [interscript/interscript-train](https://github.com/interscript/interscript-train) · [rababa-farsi](https://github.com/interscript/rababa-farsi) · [rababa-urdu](https://github.com/interscript/rababa-urdu) | archived origins of the train monorepo (full history merged there) |