Repository navigation
WO52 M4: ara/urd-g2p recipes + corpus license provenance - #295
Merged
Merged
Conversation
added 4 commits
October 17, 2026 22:20
…ance Completes the recipe registry across all four shippable lanes: - ara-diac: oracle_relabel (D_*) + sft (O_*) stages routing to the WO39 chain; gate = run-047's 3.5607 first-alt WER line - urd-g2p: sft via train_urd_g2p_hf.py; CER gate 0.17 - StageSpec gains env_prefix (U/D/O legacy conventions); planner mounts hf-dataset sources (hf://datasets/) as well as buckets - corpora registry: qcri-wiki + urd-g2p-dict added; fas-dia verified CC0-1.0 (HF cardData via API) — its license gate now opens; urd-dia/qcri-wiki/urd-g2p-dict stay UNVERIFIED with precise pointers. corpora/PROVENANCE.md records the evidence per corpus (incl. tashkeela GPLv2 note and micsell no-license exclusion) - corpus id grammar split from recipe id grammar (source names are not lang-task ids) The shipped-contracts test pins the exact open-license set, so the gate state cannot drift silently. Suite 229 green.
Verified all four corpora upstream licenses from primary sources: fas CC0-1.0 (gate opens), urd-g2p non-commercial, qcri-wiki unlicensed (Qatara repo has no license file), urd-dia sources unlocated/no license. Schema now distinguishes UNVERIFIED (unread) from LicenseRef-* (read but restricted) — both block; only SPDX-open licenses clear. PROVENANCE.md carries the evidence + the three-item owner decision queue the gate is holding.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Continues the unified framework (M0-M2 merged in #294):
ara-diac(oracle_relabelD_* + sft O_* stages routing to the proven WO39 chain, gate =
run-047's 3.5607 first-alt WER) and
urd-g2p(sft viatrain_urd_g2p_hf.py, CER gate 0.17) join urd-diac + fas-diac.
env_prefixon stages (U/D/O legacy script conventions);planner now also mounts
hf-datasetsources ashf://datasets/.fas-dia verified CC0-1.0 via the HF API — that gate opens.
urd-dia (paper-derived, terms unread), qcri-wiki (QCRI published
dataset, terms unread), urd-g2p-dict (HF "other"/custom) stay
UNVERIFIED with exact pointers in the new
corpora/PROVENANCE.md.Tashkeela's GPLv2 noted for the weights-redistribution review;
micsell (no license) stays excluded.
Test plan
asserts the fas lane stays clear
isxml plan ara-diac --run run-xdry-runs both stages