From 7996c98c1b8c35295a49b476946e3e8cb2e55017 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Wed, 28 Oct 2026 17:48:16 +0800 Subject: [PATCH 1/3] docs: coverage and known gaps page States what the 300-map corpus covers, the fifteen held standards not yet encoded (ICS Kurdish and Chinese, five DIN standards, two BSI standards, six BGN/PCGN systems), the five languages whose neural models are deliberately withheld for lack of gold data, and the source documents the project does not hold. Linked from the maps catalogue. --- src/content/docs/Coverage_and_Gaps.adoc | 95 +++++++++++++++++++++++++ src/pages/maps/index.astro | 22 ++++++ 2 files changed, 117 insertions(+) create mode 100644 src/content/docs/Coverage_and_Gaps.adoc diff --git a/src/content/docs/Coverage_and_Gaps.adoc b/src/content/docs/Coverage_and_Gaps.adoc new file mode 100644 index 00000000..87838aa4 --- /dev/null +++ b/src/content/docs/Coverage_and_Gaps.adoc @@ -0,0 +1,95 @@ += Coverage and Known Gaps +Ribose +v1.0, 2026-10-28 + +Interscript encodes authority transliteration standards as executable maps. +Every standard the project holds has been audited against the corpus; this +page states what is covered, what is held but not yet encoded, and what is +deliberately not done. + +The authoritative reference-to-corpus inventory lives in the +https://github.com/interscript/maps/blob/main/docs/coverage/README.adoc[maps repository]. + +== What is covered + +The corpus contains 300 authority-backed systems, including ALA-LC, +BGN/PCGN, ISO, ODNI (the Intelligence Community standard), UN/UNGEGN, +ICAO, DIN, BSI, and national systems. Each map carries its standard's own +example rows and is gated by ISC parse checks, embedded tests, and +cross-runtime parity. + +The neural layer is separate from the map layer. A language can have a map +without a model; most of the corpus does. Shipped models currently cover +Hebrew, Khmer, Thai, Persian, Arabic, and Urdu. + +== Held standards not yet encoded + +Fifteen deterministic tables are held but not yet encoded. Each is a +complete table with its own example rows. None needs training data to +encode; a training corpus is required only to train a model on top of the +map. + +=== ODNI / ICS 630-01 + +* **Kurdish** (Annex G) — the Intelligence Community table for Kurdish + personal names: Perso-Arabic (Sorani) to Latin, using the 26 Latin + letters plus the apostrophe, deliberately without diacritics. It is a + different system from the BGN/PCGN Kurdish 2007 map. +* **Chinese** (Annex D) — not a Han-character romanizer. The standard + applies to names already written in Hanyu Pinyin and normalises that + rendering (surname title case, no umlauted u, spacing, apostrophes). Its + appendices include conversion tables from Wade-Giles and Yale to pinyin. + Han to pinyin is a separate problem that needs a character lexicon; the + corpus already has the BGN/PCGN Hanyu Pinyin map for that. + +=== DIN + +* **DIN 31635** — Arabic, Ottoman Turkish, Persian, Kurdish, and Urdu. +* **DIN 31636** — Hebrew. +* **DIN 32706** — Armenian. +* **DIN 32708** — Japanese. +* **DIN 1460-2** — non-Slavic Cyrillic alphabets. + +DIN 32707 (Georgian), DIN 33903 (Tamil), and DIN 33904 (Devanagari) are +already encoded. + +=== BSI + +* **BS 2979:1958** — transliteration of Cyrillic and Greek characters. +* **PD 6505:1982** — guide to the romanization of Korean. + +=== BGN/PCGN + +* **Burmese**, **Lao**, **Shan**, and **Inuktitut** — complete BGN/PCGN + tables for scripts whose maps are not yet encoded. +* **Hebrew** and **Khmer** — complete BGN/PCGN tables; Hebrew is currently + covered only by the ICAO 9303 map, and Khmer by a neural model, not by a + BGN/PCGN rule map. + +== Not doing: neural models for five languages + +Burmese, Dzongkha, Inuktitut, Lao, and Shan have complete BGN/PCGN tables, +but no gold script-to-romanization data exists to train a neural model. +Interscript withholds a model rather than ship a bad-quality one. This +blocks training, not map coverage: the deterministic maps for these +languages remain in scope. + +== Source documents not held + +Some verified maps have no archived source document in the project: + +* **ALA-LC** — one table document is archived of the 78 tables in the + official catalogue. The 55 encoded ALA-LC maps are verified corpus + entries, but their source tables are not held. +* **ISO** — ISO 9, ISO 9984, and ISO/TR 11941 are held; the other ISO + standards represented in the corpus are not. +* **UN/UNGEGN** — the schedules are not held. + +These are archival gaps, not encoding gaps. + +== How the audit works + +A system is only called missing after checking, in order: the corpus on +main, open pull requests, merged history, and maps under other authority +prefixes. The corpus is the authority for what is encoded; a diff of held +PDFs against `main` alone is not enough. diff --git a/src/pages/maps/index.astro b/src/pages/maps/index.astro index 5b63a5d3..44558302 100644 --- a/src/pages/maps/index.astro +++ b/src/pages/maps/index.astro @@ -103,6 +103,11 @@ const liveDemoable = entries.filter((e) => bundledCodes.has(e.code)).length sourceScripts={sourceScripts} destinationScripts={destinationScripts} /> + +
+

Coverage and known gaps →

+

What is held but not yet encoded, and which neural models are deliberately withheld.

+
@@ -162,4 +167,21 @@ const liveDemoable = entries.filter((e) => bundledCodes.has(e.code)).length margin-top: 3rem; margin-bottom: 6rem; } + .coverage-link { + margin-top: 2.5rem; + padding-top: 1.5rem; + border-top: 1px solid var(--color-rule); + font-size: var(--text-small); + } + .coverage-link p { + margin: 0; + } + .coverage-link p + p { + margin-top: 0.35rem; + color: var(--color-stone); + } + .coverage-link a { + color: var(--color-brand-deep); + font-weight: 600; + } From bf27df17892582cacbd489a4a568d292d7125cde Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Wed, 28 Oct 2026 17:48:16 +0800 Subject: [PATCH 2/3] fix: the ISC page whitespace-collapse regression Four text-to-tag and tag-to-text line breaks made Astro trim rendered spaces. Hand-wrap them at safe boundaries and add the file to .prettierignore so Prettier cannot reintroduce the breaks. site.test.ts: 34/34 pass (the guard was red on main). --- .prettierignore | 1 + src/pages/docs/isc-language.astro | 16 +++++++--------- 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/.prettierignore b/.prettierignore index e770d326..850a4e08 100644 --- a/.prettierignore +++ b/.prettierignore @@ -30,3 +30,4 @@ src/pages/api-docs.astro # hand-wrapped prose (Astro whitespace trap) — see CLAUDE.md src/pages/playground.astro +src/pages/docs/isc-language.astro diff --git a/src/pages/docs/isc-language.astro b/src/pages/docs/isc-language.astro index bc7165c1..2f05c150 100644 --- a/src/pages/docs/isc-language.astro +++ b/src/pages/docs/isc-language.astro @@ -65,10 +65,9 @@ const tests = `tests { "input" -> "expected" }`

Rules and contexts

{rules}

before/after are zero-width - lookbehind/lookahead. boundary is a zero-width word edge. - not_before/not_after negate. - There is no alternation operator — alternatives are - separate rules.

+ lookbehind/lookahead. boundary is a zero-width word + edge. not_before/not_after negate. There is no alternation operator + — alternatives are separate rules.

Aliases

{aliases}
@@ -78,9 +77,9 @@ const tests = `tests { "input" -> "expected" }`

Stages calling stages; stdlib directives

{run}

run takes no other form — stray identifiers are a - parse error (a typo'd map fails loudly). The rababa - directive invokes the ML adapter with mandatory keyword arguments; - an unprovisioned model raises a named error enumerating known ids.

+ parse error (a typo'd map fails loudly). The rababa directive + invokes the ML adapter with mandatory keyword arguments; an + unprovisioned model raises a named error enumerating known ids.

Casing targets

{casing}
@@ -96,7 +95,6 @@ const tests = `tests { "input" -> "expected" }`

Normative reference

The full specification (IS-1 v2) with the 22 frozen rulings and the - cross-runtime guarantee lives in - interscript-models/docs/IS-1-v2.md.

+ cross-runtime guarantee lives in interscript-models/docs/IS-1-v2.md.

From c9068e5cc7468a6b4a7a19b33a4480fcbdbad93f Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Wed, 28 Oct 2026 18:07:32 +0800 Subject: [PATCH 3/3] docs: clarify coverage and gaps page --- src/content/docs/Coverage_and_Gaps.adoc | 134 +++++++++++------------- 1 file changed, 62 insertions(+), 72 deletions(-) diff --git a/src/content/docs/Coverage_and_Gaps.adoc b/src/content/docs/Coverage_and_Gaps.adoc index 87838aa4..6eb3f4db 100644 --- a/src/content/docs/Coverage_and_Gaps.adoc +++ b/src/content/docs/Coverage_and_Gaps.adoc @@ -1,95 +1,85 @@ = Coverage and Known Gaps Ribose -v1.0, 2026-10-28 - -Interscript encodes authority transliteration standards as executable maps. -Every standard the project holds has been audited against the corpus; this -page states what is covered, what is held but not yet encoded, and what is -deliberately not done. - -The authoritative reference-to-corpus inventory lives in the -https://github.com/interscript/maps/blob/main/docs/coverage/README.adoc[maps repository]. +v1.1, 2026-10-28 + +Interscript encodes authority transliteration standards as executable +maps. This page states which systems the corpus covers, which reference +tables in the project's reference collection, the archive of standards +documents used to build and verify the maps, do not yet have a +corresponding map, and which source documents are not in the +collection. Each map carries the example rows of its +standard and is checked by ISC parsing, embedded tests, and +cross-runtime parity. == What is covered -The corpus contains 300 authority-backed systems, including ALA-LC, -BGN/PCGN, ISO, ODNI (the Intelligence Community standard), UN/UNGEGN, -ICAO, DIN, BSI, and national systems. Each map carries its standard's own -example rows and is gated by ISC parse checks, embedded tests, and -cross-runtime parity. +The corpus contains more than 290 authority-backed systems, including +ALA-LC, BGN/PCGN, ISO, ODNI (the Intelligence Community standard), +UN/UNGEGN, ICAO, DIN, BSI, and national systems. -The neural layer is separate from the map layer. A language can have a map -without a model; most of the corpus does. Shipped models currently cover -Hebrew, Khmer, Thai, Persian, Arabic, and Urdu. +The neural layer is separate from the map layer. A language can have a +map without a model, and most languages in the corpus have no model. +Published neural models cover Hebrew, Khmer, Thai, Persian, Arabic, and +Urdu. -== Held standards not yet encoded +== Reference tables not yet encoded -Fifteen deterministic tables are held but not yet encoded. Each is a -complete table with its own example rows. None needs training data to -encode; a training corpus is required only to train a model on top of the -map. +The project's reference collection contains complete reference tables +for the following systems, and the corpus does not yet provide a map +for them. === ODNI / ICS 630-01 -* **Kurdish** (Annex G) — the Intelligence Community table for Kurdish +* **Kurdish** (Annex G) is the Intelligence Community table for Kurdish personal names: Perso-Arabic (Sorani) to Latin, using the 26 Latin - letters plus the apostrophe, deliberately without diacritics. It is a - different system from the BGN/PCGN Kurdish 2007 map. -* **Chinese** (Annex D) — not a Han-character romanizer. The standard + letters and the apostrophe without diacritics. It is a different + system from the BGN/PCGN Kurdish 2007 map. +* **Chinese** (Annex D) is not a Han-character romanizer. The standard applies to names already written in Hanyu Pinyin and normalises that - rendering (surname title case, no umlauted u, spacing, apostrophes). Its - appendices include conversion tables from Wade-Giles and Yale to pinyin. - Han to pinyin is a separate problem that needs a character lexicon; the - corpus already has the BGN/PCGN Hanyu Pinyin map for that. + rendering, including surname capitalization, the omission of the + umlaut on u, spacing, and apostrophes. Its appendices contain + conversion tables from Wade-Giles, Yale, and Mandarin Phonetic to + pinyin. A Han-to-pinyin conversion requires a character lexicon, and + the corpus provides the BGN/PCGN Hanyu Pinyin map for that purpose. === DIN -* **DIN 31635** — Arabic, Ottoman Turkish, Persian, Kurdish, and Urdu. -* **DIN 31636** — Hebrew. -* **DIN 32706** — Armenian. -* **DIN 32708** — Japanese. -* **DIN 1460-2** — non-Slavic Cyrillic alphabets. +* **DIN 31635:2011** covers Arabic, Ottoman Turkish, Persian, Kurdish, + and Urdu. +* **DIN 31636:2018** covers Hebrew. +* **DIN 32706:2010** covers Armenian. +* **DIN 32708:2014** covers Japanese. +* **DIN 1460-2:2011** covers non-Slavic Cyrillic alphabets. -DIN 32707 (Georgian), DIN 33903 (Tamil), and DIN 33904 (Devanagari) are -already encoded. +DIN 32707:2010 (Georgian), DIN 33903:2016 (Tamil), and DIN 33904:2018 +(Devanagari) are already encoded. === BSI -* **BS 2979:1958** — transliteration of Cyrillic and Greek characters. -* **PD 6505:1982** — guide to the romanization of Korean. +* **BS 2979:1958** specifies the transliteration of Cyrillic and Greek + characters. +* **PD 6505:1982** is a guide to the romanization of Korean. === BGN/PCGN -* **Burmese**, **Lao**, **Shan**, and **Inuktitut** — complete BGN/PCGN - tables for scripts whose maps are not yet encoded. -* **Hebrew** and **Khmer** — complete BGN/PCGN tables; Hebrew is currently - covered only by the ICAO 9303 map, and Khmer by a neural model, not by a - BGN/PCGN rule map. - -== Not doing: neural models for five languages - -Burmese, Dzongkha, Inuktitut, Lao, and Shan have complete BGN/PCGN tables, -but no gold script-to-romanization data exists to train a neural model. -Interscript withholds a model rather than ship a bad-quality one. This -blocks training, not map coverage: the deterministic maps for these -languages remain in scope. - -== Source documents not held - -Some verified maps have no archived source document in the project: - -* **ALA-LC** — one table document is archived of the 78 tables in the - official catalogue. The 55 encoded ALA-LC maps are verified corpus - entries, but their source tables are not held. -* **ISO** — ISO 9, ISO 9984, and ISO/TR 11941 are held; the other ISO - standards represented in the corpus are not. -* **UN/UNGEGN** — the schedules are not held. - -These are archival gaps, not encoding gaps. - -== How the audit works - -A system is only called missing after checking, in order: the corpus on -main, open pull requests, merged history, and maps under other authority -prefixes. The corpus is the authority for what is encoded; a diff of held -PDFs against `main` alone is not enough. +The project's reference collection contains complete BGN/PCGN tables +for Adyghe, Avar, Burmese, +Chuvash, Dzongkha, Hebrew, Inuktitut, Kabardian, Karachay-Balkar, +Khmer, Lao, Modern Syriac, Ossetian, Shan, Udmurt, and Yakut. The +corpus does not yet provide a map for these systems. Hebrew is +represented by the ICAO 9303 map and Khmer by a neural model; neither +substitutes for a BGN/PCGN rule map. + +== Source documents not in the collection + +The corpus contains verified maps for several authorities whose source +table documents are not in the project's reference collection: + +* **ALA-LC** — the collection contains one table document of the 78 + tables in the official catalogue. The corpus contains 55 ALA-LC maps. +* **ISO** — ISO 9, ISO 9984, and ISO/TR 11941 are in the collection. + The corpus's other ISO maps, which implement ISO 233, ISO 843, ISO + 3602, ISO 11940, and ISO 15919, have no source document in the + collection. +* **UN/UNGEGN** — the corpus contains 32 UN maps, and the schedules are + not in the collection.