Repository navigation
feat(run-017): plane-factorized diacritization - bidirectional haraqat prediction (Stoicheia WO) - #105
Merged
Conversation
- build_yallamorph_aux: MORPH-prefixed aux line builder (TDD, 12 tests) - gen_yallamorph_pairs: Camel Morph MSA v1.0 paradigm generation, underspecified-spec interface, YallaMorph gold validation (38/52), 300k stratified lines on volume /datasets/yallamorph-aux - train_arabic_r9_yallamorph: r7 recipe + 25% morph aux stream, init run-007-news, resumable + EVAL_DONE idempotent - ride_probe_math + probe_ride_direction: r7-r6 residual direction transferability probe (TDD math; measured TRANSFER, max cos 0.9513)
- haraqat_planes: length-preserving plane decomposition (split/render byte-exact inverse; TDD, 8 tests incl. 500-line benchmark round-trip) - train_arabic_plane: ByT5-small ENCODER + diacritic-plane embedding + per-position classification over the corpus haraqat-combo inventory (15 classes on sample); Mask-Predict self-conditioning at random corruption levels; K=4 parallel passes at inference - corpus: benchmark-convention running text only (r9 lesson applied) - gates: beat 4.5701 (student rung) / 2.2864 (teacher rung); bidirectional conditioning attacks the i'rab component the AR rungs cannot see
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Stoicheia WO (TODO.sota-2026/04 in interscript-ml), entered on owner authorization.
Thesis: diacritization is a length-preserving plane projection - the letters plane is the given skeleton, the haraqat plane is a per-letter classification. An encoder-only plane predictor sees BIDIRECTIONAL sentence context, which the left-to-right seq2seq rungs cannot: i'rab (word-final case endings, 33% of the measured residual) depends on sentence structure ahead. Mask-Predict self-conditioning (trained at random corruption levels, K=4 parallel passes at inference) gives joint consistency without autoregressive decoding.
haraqat_planes.py- plane decomposition (TDD, 8 tests; split/render is a byte-exact inverse, verified round-trip on 500 benchmark lines; canonization only at inventory time so any convention order round-trips)train_arabic_plane.py- ByT5-small encoder (pretrained backbone law) + plane embedding + per-position head; corpus = benchmark-convention running text ONLY (r9 convention-drift lesson applied by design)Pre-registered gates (same windowed zero-skip harness): rung gate beat 4.5701 (on-device class; encoder-only is also smaller than the seq2seq student), SOTA-dedicated gate beat 2.2864. CPU latency benchmark vs ara-diac-small-int8static-2.1 post-hoc.