Repository navigation
feat(r9): YallaMorph/CamelMorph morph aux teacher + RIDE direction probe - #104
Merged
Merged
Conversation
- build_yallamorph_aux: MORPH-prefixed aux line builder (TDD, 12 tests) - gen_yallamorph_pairs: Camel Morph MSA v1.0 paradigm generation, underspecified-spec interface, YallaMorph gold validation (38/52), 300k stratified lines on volume /datasets/yallamorph-aux - train_arabic_r9_yallamorph: r7 recipe + 25% morph aux stream, init run-007-news, resumable + EVAL_DONE idempotent - ride_probe_math + probe_ride_direction: r7-r6 residual direction transferability probe (TDD math; measured TRANSFER, max cos 0.9513)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Teacher-side data lever from the 2026-09-30 arXiv sweep (TODO.sota-2026/01 and /05 in interscript-ml).
run-009-yallamorph (r8-generation teacher; run-008 slot was consumed by the IPA ablation): the r7 recipe plus a MORPH-prefixed aux stream teaching feature-conditional vocalization, built from Camel Morph MSA v1.0 (LREC-COLING 2024, CC BY 4.0) - the resource YallaMorph (arXiv 2609.10153, EMNLP 2026) was constructed from. Deterministic dictionary resource; no LLM labels.
build_yallamorph_aux.py- aux line builder, TDD (12 tests). Input = "MORPH: | ", target = fully diacritized form; the r6-proven ASCII-prefix multitask mechanism.gen_yallamorph_pairs.py- paradigm generation via camel_tools Generator. Key finding: the correct interface is UNDERSPECIFIED generation (pos [+ proclitic variant]); fully-specified requests silently reject cells where morphology leaves a feature unmarked (1st person carries gen 'u'). 13,000 lemmas (YallaMorph sample in-DB + inventory top-up, seed 42) -> 4.7M raw pairs -> 300k stratified lines (60/40 verb/nominal), shipped to volume /datasets/yallamorph-aux. Validation vs YallaMorph few-shot gold: 38/52 exact; residual divergence is proclitic-chain conventions + DB-version lemma gaps, not wrong forms.train_arabic_r9_yallamorph.py- r7 recipe verbatim + aux sampled to exactly 25% of the mix, init run-007-news, gates: SadeedDiac windowed zero-skip <= 2.389 AND WikiNews-2024 multi-ref improves over 17.3794/11.8273; canonical replacement only if ID improves outright. In flight (app ap-f1R8ChDikMKUGyBuJGO6).ride_probe_math.py+probe_ride_direction.py- RIDE-style SFT-residual direction transferability probe (TDD math, 7 tests). Measured: cos(d_classical, d_news) per encoder layer 0.94/0.95/0.92/... decaying to noise by L11 - the r6->r7 delta is domain-general in early/mid layers; the displacement-arm premise holds (max 0.9513 >= 0.5 kill bar). Training-arm spec recorded in TODO.sota-2026/05.