Skip to content

data(WO67): tha-phonemic-v2 — 122,311 pairs (3.8x) - #343

Merged
ronaldtse merged 1 commit into
mainfrom
data/tha-phonemic-v2
Oct 22, 2026
Merged

ronaldtse merged 1 commit into
mainfrom
data/tha-phonemic-v2

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

The plateau audit's fix, as a pinned corpus:

tier pairs
v1 (ORST only, old engine) 31,869
v2 (orst + thai_words + volubilis, royin 5.3.5) 122,311
  • All three lexicons are Apache-2.0, pythainlp-packaged — same channel as v1, no new licensing.
  • The old engine's cluster/syllable-repeat defect class is fixed at 5.3.5: rejections 6,291 (17%) → 1,167 (0.95%).
  • Splits 109,969 / 1,223 / 11,119 with sha256 pins; builder script included.
  • v1 stays pinned untouched — provenance of the shipped arm is preserved; v2 is a new record.

…-adjacent lexicons x royin 5.3.5)

Verdict of the plateau audit: v1's 31,869-word ceiling was an
extraction artifact. Union of orst + thai_words + volubilis (all
Apache-2.0, pythainlp-packaged), transliterated by the same
deterministic engine at 5.3.5 where the syllable-repeat defect class
is fixed (rejections 17% -> 0.95%). v1 remains pinned as the record
of what trained the shipped arm; v2 is the next arm's corpus.
@ronaldtse
ronaldtse merged commit 57448f1 into main Oct 22, 2026
3 checks passed
@ronaldtse
ronaldtse deleted the data/tha-phonemic-v2 branch October 22, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant