Skip to content

Latest commit

 

History

History
188 lines (151 loc) · 7.99 KB

README.adoc

File metadata and controls

188 lines (151 loc) · 7.99 KB

Reference-to-corpus inventory

This document records, by authority, the reference documents in the Interscript project’s reference collection, which is the archive of standards documents against which the maps are built and verified, and the maps in the corpus that implement them. A reference document is listed whether or not the corpus provides a map for it, and each map is named with the reference standard from which it was encoded when that standard is in the collection. The corpus is the authority for what is encoded.

ICS 630-01

ICS 630-01 is the Intelligence Community standard for foreign-language transliteration. The standard establishes methods for rendering personal names written in non-Roman scripts into the Roman alphabet and delegates the language-specific tables to annexes administered by the Senior Language Authority of each IC element. The project’s reference collection contains 29 documents for the standard: the standard itself, 23 language annexes lettered A to W, three Chinese appendices, and revisions of Annexes J and M.

Annex Language Maps in the corpus

A

Arabic

odni-ara-Arab-Latn-2004, odni-ara-Arab-Latn-2015

B

Belarusian

odni-bel-Cyrl-Latn-2015

C

Chechen

odni-che-Cyrl-Latn-2015

D

Chinese

None

E

Georgian

odni-kat-Geor-Latn-2015

F

Hindi and Urdu

odni-hin-Deva-Latn-2004, odni-hin-Deva-Latn-2015, odni-urd-Arab-Latn-2015

G

Kurdish

None

H

Macedonian

odni-mkd-Cyrl-Latn-2005, odni-mkd-Cyrl-Latn-2015

I

Pashto

odni-pus-Arab-Latn-2011

J

Persian and Dari

odni-fas-Arab-Latn-2004, odni-fas-Arab-Latn-2015, odni-prs-Arab-Latn-2004, odni-prs-Arab-Latn-2015

K

Russian

odni-rus-Cyrl-Latn-2015

L

Serbian

odni-srp-Cyrl-Latn-2005, odni-srp-Cyrl-Latn-2015

M

Ukrainian

odni-ukr-Cyrl-Latn-2015

N

Tajik

odni-tgk-Cyrl-Latn-2015

O

Bulgarian

odni-bul-Cyrl-Latn-2005, odni-bul-Cyrl-Latn-2015

P

Azeri

odni-aze-Cyrl-Latn-2015

Q

Kazakh

odni-kaz-Cyrl-Latn-2015

R

Kyrgyz

odni-kir-Cyrl-Latn-2015

S

Tatar

odni-tat-Cyrl-Latn-2015

T

Turkmen

odni-tuk-Cyrl-Latn-2015

U

Uyghur

odni-uig-Cyrl-Latn-2015

V

Uzbek

odni-uzb-Cyrl-Latn-2015

W

Korean

odni-kor-Hang-Latn-2015

Twenty-one of the 23 language annexes have maps in the corpus. The Chinese annex (D) and the Kurdish annex (G) do not.

Chinese annex

Annex D applies to personal names already rendered in Hanyu Pinyin and standardises that rendering: surname capitalization, omission of the umlaut on u, spacing, and apostrophes. The annex assumes the pinyin form and does not supply a lexicon for the conversion of Han characters to pinyin. Its appendices are Appendix A, the IC transliteration table; Appendix B, conversion tables from Wade-Giles, Yale, and Mandarin Phonetic to pinyin; and Appendix C, a list of common Chinese surnames. A pinyin normalisation map and Latn-to-Latn conversion maps can be encoded from the annex. A Han-to-pinyin map requires a character lexicon, and the corpus provides bgnpcgn-zho-Hans-Latn-1979 for that conversion.

Kurdish annex

Annex G is a Perso-Arabic (Sorani) to Latin table for personal names. The table uses the 26 Latin letters and the apostrophe, without diacritics. The corpus’s bgnpcgn-kur-Arab-Latn-2007 implements a different authority and does not substitute for the annex.

ODNI reference folder

The folder contains documents for Arabic, Belarusian, Bulgarian, Farsi/Persian and Dari, Hindi/Urdu, Korean, Kurdish, Macedonian, Pashto, Serbian, and Ukrainian. The corpus provides ODNI maps for ten of the eleven languages. The Kurdish document is the same table as Annex G and has no map in the corpus. The Chinese annex is in the ICS 630-01 collection.

DIN

Standard Title Maps in the corpus

DIN 1460-2:2011

Romanization of Cyrillic alphabets, Part 2: non-Slavic languages

None

DIN 31635:2011

Romanization of the Arabic alphabet for Arabic, Ottoman Turkish, Persian, Kurdish, and Urdu

None

DIN 31636:2018

Romanization of the Hebrew alphabet

None

DIN 32706:2010

Romanization of the Armenian alphabet

None

DIN 32707:2010

Romanization of the Georgian alphabet

din-kat-Geor-Latn-32707-2010

DIN 32708:2014

Romanization of Japanese

None

DIN 33903:2016

Romanization of Tamil

din-tam-Taml-Latn-33903-2016

DIN 33904:2018

Transliteration of Devanagari

din-hin-Deva-Latn-33904-2018, din-mar-Deva-Latn-33904-2018, din-nep-Deva-Latn-33904-2018, din-pli-Deva-Latn-33904-2018, din-pra-Deva-Latn-33904-2018, din-san-Deva-Latn-33904-2018

Five of the eight DIN standards in the collection have no map in the corpus. The corpus also contains din-grc-Grek-Latn-31634-2011-t1, which implements DIN 31634 for Greek; that standard is not in the collection.

ISO

Standard Scope Maps in the corpus

ISO 9:1995

Cyrillic to Latin

iso-rus-Cyrl-Latn-9-1995

ISO 9984:1996

Georgian to Latin

iso-kat-Geor-Latn-9984-1996

ISO/TR 11941:1996

Korean to Latin, two methods

iso-kor-Hang-Latn-1996-method1, iso-kor-Hang-Latn-1996-method2

All three ISO documents in the collection have maps in the corpus. The corpus’s other ISO maps, which implement ISO 233, ISO 843, ISO 3602, ISO 11940, and ISO 15919, have no source document in the collection.

BSI

Standard Title Maps in the corpus

BS 2979:1958

Transliteration of Cyrillic and Greek characters

None

PD 6505:1982

Guide to the romanization of Korean

None

BGN/PCGN

The BGN/PCGN systems represented in the corpus and the reference tables that have no map are listed in bgnpcgn-status.adoc in this directory.

Historical editions

The collection contains 16 documents in the 1964 BGN romanization guide and 30 documents in the 1994 USBGN romanization guide. These documents record earlier editions of systems whose current BGN/PCGN sheets are also in the collection, including Hebrew (1962), Burmese (1970), Khmer (1972), and Lao (1966).

ALA-LC

The official catalogue of 78 ALA-LC romanization tables is captured in alalc-roman.html, and differences against the corpus are recorded in alalc-diff.txt. The collection contains one table document, alalc/1997/mongolia.pdf. The corpus contains 55 ALA-LC maps, whose source tables are not in the collection. Encoding the 47 catalog systems that are not in the corpus requires acquisition of their table documents. Several of those systems overlap languages represented under other authorities, including Burmese, Khmer, Lao, Shan, Hebrew, Pashto, Thai, Urdu, Uyghur, Uzbek, and Kurdish.

ICAO

The collection contains ICAO 9303 Part 3 (Amendment 1) and the 9303 Arabic helper. The corpus contains eight ICAO maps. The Part 3 in the collection does not contain the language transliteration tables, which are published in another part or edition of 9303.

BAS, UN, and other authorities

  • The two BAS documents have maps: bas-rus-Cyrl-Latn-2017-bss and bas-rus-Cyrl-Latn-2017-oss.

  • The corpus contains 32 UN maps, and the UNGEGN schedules are not in the collection.

  • The remaining single-system authorities, including ELOT, MVD, GOST, MEXT, MOCT, LSHK, ACADSIN, SAC, HK, and KP, have verified maps in the corpus and no source document in the collection.

Reference tables not yet encoded

Twenty-five reference tables in the collection have no map in the corpus: two ICS annexes, five DIN standards, two BSI standards, and sixteen BGN/PCGN systems. The BGN/PCGN systems are listed in bgnpcgn-status.adoc; the others are listed in their authority sections above.

Neural model coverage

The deterministic map and the neural model are independent products. The corpus can provide a map for a language without a published model, and most languages in the corpus have no model. Neural models are published for Hebrew, Khmer, Thai, Persian, Arabic, and Urdu. No neural model is published for Burmese, Dzongkha, Inuktitut, Lao, or Shan, because no parallel corpus of source-script names with romanized equivalents is available for training in those languages.