Skip to content

feat: ALA-LC batch 2 - kazakh, kurdish, ladino, judeo-arabic, malay, jawi-pegon, sindhi, uighur, urdu, pushto, ottoman - #266

Merged
ronaldtse merged 1 commit into
mainfrom
feat/alalc-batch2-arabic-script
Nov 8, 2026
Merged

ronaldtse merged 1 commit into
mainfrom
feat/alalc-batch2-arabic-script

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

ALA-LC batch 2 — Arabic-script / Hebrew-script systems (11 maps)

Second batch of the remaining-encodings campaign's ALA-LC work order. Each map transcribes its own source document (read visually, codepoint-verified against the text layer) and carries embedded tests taken from that document's tables and rule examples. Nothing pre-existing is touched; the change only adds files.

Systems encoded

Map Source document Tests
alalc-kaz-Arab-Latn-2012 alalc-kazakh.pdf (2012 version, "No earlier versions") 42
alalc-kur-Arab-Latn-2012 alalc-kurdish.pdf (2012; Arabic column of the Hawar/Cyrillic/Arabic trilingual table) 83
alalc-lad-Hebr-Latn-2025 alalc-ladino.pdf (2025; Hebrew script) 86
alalc-jba-Hebr-Latn-2011 alalc-judeo-arabic.pdf (2011-01-31; Hebrew script) 92
alalc-zsm-Arab-Latn-2011 alalc-malay.pdf (2011; Malay in Jawi-Arabic script) 55
alalc-jav-Arab-Latn-2012 alalc-jawi-pegon.pdf (2012 Jawi/Pegon, successor of the Malay table) 58
alalc-snd-Arab-Latn-2024 alalc-sindhi.pdf (2024) 61
alalc-urd-Arab-Latn-2013 alalc-urdu.pdf (2013) 52
alalc-uig-Arab-Latn-2015 alalc-uighur.pdf (2015) 52
alalc-pus-Arab-Latn-2013 alalc-pushto.pdf (2013) 57
alalc-ota-Arab-Latn-1997 alalc-ottoman.pdf (undated; Birnbaum-based, 1997 system edition) 43

Moghol: source not in the collection (alalc-moghol.pdf does not exist) — not fabricated, per the work order.

Verification

  • Embedded tests: 681/681 pass through the interscript-ts runtime (parser + converter + transliterate), run via a local runner over the 11 new maps.
  • TS parse: npx -y tsx scripts/parse-isc.mjs ../interscript-ts → Parsed 420/420 .isc files.
  • Ruby parse: Interscript::Isc::Parser.parse + DocumentBuilder.build over all 420 files against a fresh origin/main clone of interscript-ruby (parsanol 1.3.135) → Parsed 420/420 .isc files.
  • git status: only the 11 new .isc files; no existing file modified.

Encoding decisions worth review

  • Kurdish h/e (note 4). The hāʼ joining forms carry the H/E distinction. The map renders the heh presentation forms semantically (ﻫ ﻬ → h; ﻪ ﻩ → e; doachashmee forms accordingly, with final ﮫ → e per the ême example) and resolves plain codepoints by context (word-initial he and he before a dual-joining letter = h, else e). Adjacent plain heh pairs (E+H as in lehī) are irreducible from codepoints alone — shaped input romanizes exactly; the tests carry the document's own text-layer encodings.
  • Kazakh he e/h homograph (alphabet rows 8 and 29): defaults to e, exercised by the document's own example; the h reading is lexical and documented.
  • Unwritten short vowels. Urdu note 5, Pushto note 4, Malay and Sindhi tables state vowels must be supplied from a dictionary. The maps encode the letter values, the written marks, and the mechanical rules; examples whose romanization requires lexical vowels are carried as comments with the mechanical rendering noted (e.g. Urdu هم → mechanical hm, document ham).
  • Ottoman rule 2 (conversion). The principle of conversion to modern Turkish orthography is lexical; the map is the letter-by-letter table per rule 9, with the vowel-sign machinery deferred to the Arabic/Persian tables as the document itself directs. The source document carries no edition date; the 1997 attribute follows the ALA-LC 1997 system edition.
  • Pushto ṭ row. The 2013 source's romanization cell for ṭ is not legible in the extraction; the value ṭ follows the earlier ALA-LC Pushto tables and the universal Arabic-script reading (flagged in the map's notes).
  • Undotted/dotted homographs (Kazakh he, Uighur he/ه, Ottoman گ written stroke-less as ك, ڭ written dot-less as ن, Malay ف as shorthand for ڤ): both codepoints are encoded with the table-row values; which reading a given token carries is lexical and noted in each map.
  • Shaped text layers. Kazakh, Ladino and Kurdish tests carry the documents' presentation-form encodings verbatim (in Kurdish and Ladino they are semantically load-bearing); Malay and Uighur presentation forms carry no letter identity, so those maps operate on base codepoints and the tests are the logical transcriptions.
  • Case. Kazakh rule 3, Ottoman/Urdu/Pushto rule on capitalization and the proper-noun capitals in several sources are writer conventions; the maps leave case untouched and the affected examples are commented rather than forced.
  • Moghol is recorded as source-not-in-collection (above) and nothing else.

…jawi-pegon, sindhi, uighur, urdu, pushto, ottoman

Eleven new ALA-LC maps completing the Arabic-script/Hebrew-script
batch of the remaining-encodings campaign, each with embedded tests
taken from its own source document:

- alalc-kaz-Arab-Latn-2012 (Kazakh, Arabic script)
- alalc-kur-Arab-Latn-2012 (Kurdish, Arabic column of the Hawar/Cyrillic/
  Arabic trilingual table; he joining-form h/e semantics)
- alalc-lad-Hebr-Latn-2025 (Ladino)
- alalc-jba-Hebr-Latn-2011 (Judeo-Arabic)
- alalc-zsm-Arab-Latn-2011 (Malay in Jawi-Arabic script)
- alalc-jav-Arab-Latn-2012 (Jawi/Pegon)
- alalc-snd-Arab-Latn-2024 (Sindhi; Urdu rules per its note 2)
- alalc-urd-Arab-Latn-2013 (Urdu)
- alalc-uig-Arab-Latn-2015 (Uighur)
- alalc-pus-Arab-Latn-2013 (Pushto)
- alalc-ota-Arab-Latn-1997 (Ottoman Turkish; Birnbaum-based table,
  1997 system edition)

Verification: 681/681 embedded tests through the interscript-ts
runtime; parse 420/420 .isc in both the TypeScript and Ruby parsers.
@ronaldtse
ronaldtse merged commit b9ea8c4 into main Nov 8, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant