diff --git a/.changeset/30618-script-subtag-read-aliases.md b/.changeset/30618-script-subtag-read-aliases.md new file mode 100644 index 00000000000..0a59760d39d --- /dev/null +++ b/.changeset/30618-script-subtag-read-aliases.md @@ -0,0 +1,14 @@ +--- +'@objectstack/spec': minor +--- + +fix(spec/system): the platform locale table reads the script-subtag Chinese spellings, so a `zh-Hant-TW` / `zh-Hant-HK` reader gets a registered `zh-Hant` bundle instead of Simplified Chinese + +Clause-②: yes (widening) +Metadata-protocol: semantic + +`PLATFORM_LOCALE_READ_ALIASES` (`@objectstack/spec/system`) gains five rows, one for the script-subtag spelling of each regional Chinese alias: `zh-Hans-CN` and `zh-Hans-SG` read as `zh-Hans`; `zh-Hant-TW`, `zh-Hant-HK` and `zh-Hant-MO` read as `zh-Hant`. `platformLocaleTagOf` answers them (case-insensitively, as for every row), and `PlatformLocaleReadAlias` gains the five literal keys. + +What changes for a reader: browsers and operating systems send these spellings. `resolveBundleLocale` and `resolveLocaleAmong` (and so `resolveLocale` and `resolveOfferedLocale` in `@objectstack/core`) now map them through the table before the base-language step, so over a bundle that holds both scripts a `zh-Hant-TW`, `zh-Hant-HK` or `zh-Hant-MO` request reaches the `zh-Hant` key and a `zh-Hans-CN` or `zh-Hans-SG` request reaches the Simplified key (`zh-CN`, `zh-Hans` or a bare `zh`), whatever order the keys were registered in. Before, the first registered `zh-*` key answered them, and over `{ zh, zh-Hant }` a `zh-Hant-TW` request got the bare `zh` (Simplified) key. The table step maps each registered key through the table too, so a bundle key or offered code spelled `zh-Hant-TW` / `zh-Hans-CN` now counts as its script for every request the table reads as that script: `zh-TW` or `zh-Hant` over `[en, zh-CN, zh-Hant-TW]` gets `zh-Hant-TW` (before: `zh-CN`), `zh-CN` over `[en, zh-Hant, zh-Hans-CN]` gets `zh-Hans-CN` (before: `zh-Hant`), and `zh-Hant-TW` over `[en, zh, zh-Hant-HK]` gets `zh-Hant-HK` (before: `zh`). Setup's default-language options (`localization.locale`) name the five codes from the table (`zh-Hant-TW` is 繁體中文, `zh-Hans-CN` is 简体中文) where they used the runtime's display name (中文（繁體，台灣）), and a stored `zh-Hant-TW` default now shows selected on the 繁體中文 option: where the deployment listed `zh-CN` before `zh-Hant`, it used to take the place of the 简体中文 option. The values Setup accepts are unchanged. + +Unchanged: the list stays closed. Only these five spellings are added; a script-subtag spelling without a row (`zh-Hant-US`) and every other code the table does not name (`pt-PT`, `en-GB`, `es-419`) answer `undefined` and fall back through base language and variant expansion exactly as before. With no registered key of the requested script, a script-subtag request still reaches the other script's key, as before. No call that matched before stops matching. diff --git a/content/docs/protocol/kernel/i18n-standard.mdx b/content/docs/protocol/kernel/i18n-standard.mdx index 63be6a86b07..2cf41593f66 100644 --- a/content/docs/protocol/kernel/i18n-standard.mdx +++ b/content/docs/protocol/kernel/i18n-standard.mdx @@ -163,33 +163,48 @@ The rule: - **Adding or removing a language** is a change to the table, followed by one backfill of every surface. Nothing else declares the list. -**Read aliases.** Codes that stored values already carry keep resolving: +**Read aliases.** Codes that stored values already carry, and the +script-subtag spellings browsers and operating systems send, keep resolving: `PLATFORM_LOCALE_READ_ALIASES` maps each to a table tag, and -`platformLocaleTagOf(code)` answers the table tag a stored code means. +`platformLocaleTagOf(code)` answers the table tag a code means. -| Stored code | Reads as | +| Code | Reads as | |:--|:--| -| `zh`, `zh-CN`, `zh-SG` | `zh-Hans` | -| `zh-TW`, `zh-HK`, `zh-MO` | `zh-Hant` | +| `zh`, `zh-CN`, `zh-SG`, `zh-Hans-CN`, `zh-Hans-SG` | `zh-Hans` | +| `zh-TW`, `zh-HK`, `zh-MO`, `zh-Hant-TW`, `zh-Hant-HK`, `zh-Hant-MO` | `zh-Hant` | | `ja-JP` | `ja` | | `es-ES` | `es` | | `pt` | `pt-BR` | | `en-US` | `en` | A table tag answers itself, the match ignores case (`zh-cn` reads as -`zh-Hans`), and any other code answers `undefined`: no region fallback is -guessed, so `pt-PT` is not `pt-BR`. New writes use the table's tags. The +`zh-Hans`), and any other code answers `undefined`. The list is closed and is +read row by row, never as a pattern, so no region or script fallback is +guessed: `pt-PT` is not `pt-BR`, and a script-subtag spelling without a row +(`zh-Hant-US`) answers `undefined` too. New writes use the table's tags. The examples on this page that spell `zh-CN` stay valid, because `zh-CN` is a read alias of `zh-Hans`. `platformLocaleTagOf` answers which table language a code *means*; the -fallback ladder below answers which key of a bundle *serves* a request. A -reader that uses both maps the code first and then resolves the table tag -against the bundle, so they do not contradict each other: `pt` means `pt-BR`, -and a request for `pt-BR` may still be served by a bundle's `pt` key. The -ladder alone cannot tell the two Chinese scripts apart: once a bundle holds -both `zh-Hans` and `zh-Hant`, a request for `zh` or `zh-CN` is answered by key -order (`zh-Hant` when that key comes first), where the table says `zh-Hans`. +fallback ladder answers which key of a bundle *serves* a request. The ladder is +one function, `resolveLocaleAmong`: `resolveBundleLocale` runs it over a +bundle's keys, and core's `resolveLocale` is the same function. Its steps are +exact match, case-insensitive match, then the **same table language**, then +base language, then the first registered variant. The table-language step maps +the request through `platformLocaleTagOf` and takes the key spelled as that +table tag, else the first key the table maps to it, so the two answers never +contradict each other: `pt` means `pt-BR`, and a request for `pt-BR` is served +by a bundle's `pt` key. A bundle holding both scripts answers `zh`, `zh-CN` and +`zh-Hans-CN` with its Simplified key and `zh-TW` and `zh-Hant-TW` with its +Traditional one, whatever order the keys were registered in. + +Two kinds of request still go to the base-language and first-variant steps, +where registration order decides. A code the table does not name skips the +table step: over keys registered as `en`, `zh-CN`, `zh-Hant` a `zh-Hant-US` +request gets `zh-CN`, and with `zh-Hant` registered before `zh-CN` it gets +`zh-Hant`. A request whose table language has no key falls through the same +way: `zh-TW` over `en`, `zh-CN` gets `zh-CN`, because a related language beats +none. ### Locale Fallback diff --git a/packages/core/src/fallbacks/fallbacks.test.ts b/packages/core/src/fallbacks/fallbacks.test.ts index bc025a78eea..96e858887c8 100644 --- a/packages/core/src/fallbacks/fallbacks.test.ts +++ b/packages/core/src/fallbacks/fallbacks.test.ts @@ -315,6 +315,30 @@ describe('resolveLocale and resolveBundleLocale — one shared case list', () => ['zh-SG', ['en', 'zh-Hant', 'zh-CN'], 'zh-CN'], ['zh-Hans', ['en', 'zh-Hant', 'zh-CN'], 'zh-CN'], ['zh-TW', ['en', 'zh', 'zh-Hant'], 'zh-Hant'], + // The script-subtag spellings a browser or OS sends are read aliases too. + ['zh-Hant-TW', ['en', 'zh-CN', 'zh-Hant'], 'zh-Hant'], + ['zh-Hant-TW', ['en', 'zh-Hant', 'zh-CN'], 'zh-Hant'], + ['zh-Hant-HK', ['en', 'zh-CN', 'zh-Hant'], 'zh-Hant'], + ['zh-Hant-HK', ['en', 'zh-Hant', 'zh-CN'], 'zh-Hant'], + ['zh-Hant-MO', ['en', 'zh-CN', 'zh-Hant'], 'zh-Hant'], + ['zh-Hant-MO', ['en', 'zh-Hant', 'zh-CN'], 'zh-Hant'], + ['zh-Hans-CN', ['en', 'zh-CN', 'zh-Hant'], 'zh-CN'], + ['zh-Hans-CN', ['en', 'zh-Hant', 'zh-CN'], 'zh-CN'], + ['zh-Hans-SG', ['en', 'zh-CN', 'zh-Hant'], 'zh-CN'], + ['zh-Hans-SG', ['en', 'zh-Hant', 'zh-CN'], 'zh-CN'], + ['ZH-HANT-tw', ['en', 'zh-CN', 'zh-Hant'], 'zh-Hant'], + ['zh-Hant-TW', ['en', 'zh', 'zh-Hant'], 'zh-Hant'], + // ...and so are keys spelled that way: a key counts as the script it names. + ['zh-TW', ['en', 'zh-CN', 'zh-Hant-TW'], 'zh-Hant-TW'], + ['zh-TW', ['en', 'zh-Hant-TW', 'zh-CN'], 'zh-Hant-TW'], + ['zh-Hant', ['en', 'zh-CN', 'zh-Hant-TW'], 'zh-Hant-TW'], + ['zh-Hant', ['en', 'zh-Hant-TW', 'zh-CN'], 'zh-Hant-TW'], + ['zh-CN', ['en', 'zh-Hant', 'zh-Hans-CN'], 'zh-Hans-CN'], + ['zh-CN', ['en', 'zh-Hans-CN', 'zh-Hant'], 'zh-Hans-CN'], + ['zh-Hant-TW', ['en', 'zh', 'zh-Hant-HK'], 'zh-Hant-HK'], + ['zh-Hant-TW', ['en', 'zh-Hant-HK', 'zh'], 'zh-Hant-HK'], + // One script registered: a related language still beats none. + ['zh-Hant-TW', ['en', 'zh-CN'], 'zh-CN'], // Today's alias-spelled bundles keep answering. ['ja', ['en', 'zh-CN', 'ja-JP', 'es-ES'], 'ja-JP'], ['es', ['en', 'zh-CN', 'ja-JP', 'es-ES'], 'es-ES'], @@ -344,6 +368,19 @@ describe('resolveLocale and resolveBundleLocale — one shared case list', () => expect(i18n.t('settings', 'zh-HK')).toBe('設定'); expect(i18n.t('settings', 'zh')).toBe('设置'); }); + + it.each<[order: string, loadOrder: string[]]>([ + ['Simplified loaded first', ['zh-CN', 'zh-Hant']], + ['Traditional loaded first', ['zh-Hant', 'zh-CN']], + ])('a script-subtag reader of the in-memory service gets its own script (%s)', (_order, loadOrder) => { + const text: Record = { 'zh-CN': '设置', 'zh-Hant': '設定' }; + const i18n = createMemoryI18n(); + i18n.loadTranslations('en', { settings: 'Settings' }); + for (const locale of loadOrder) i18n.loadTranslations(locale, { settings: text[locale] }); + expect(i18n.t('settings', 'zh-Hant-TW')).toBe('設定'); + expect(i18n.t('settings', 'zh-Hant-HK')).toBe('設定'); + expect(i18n.t('settings', 'zh-Hans-CN')).toBe('设置'); + }); }); describe('createMemoryI18n locale fallback', () => { diff --git a/packages/spec/src/system/i18n-resolver.test.ts b/packages/spec/src/system/i18n-resolver.test.ts index 8c4bc7d9ba0..1661dd9ee2f 100644 --- a/packages/spec/src/system/i18n-resolver.test.ts +++ b/packages/spec/src/system/i18n-resolver.test.ts @@ -1048,10 +1048,11 @@ describe('locale fallback resolution (BCP-47)', () => { // The platform locale table's read aliases decide which key serves a request // before any base-language or first-variant guess, so a Traditional reader -// (`zh-TW`, `zh-HK`, `zh-MO`) reaches a `zh-Hant` key and a Simplified one -// reaches `zh-CN` whatever order the bundle registered them in. Core's -// `resolveLocale` is `resolveLocaleAmong`; its own suite runs one case list -// against both resolvers. +// (`zh-TW`, `zh-HK`, `zh-MO`, or the script-subtag `zh-Hant-TW` a browser +// sends) reaches a `zh-Hant` key and a Simplified one reaches `zh-CN` whatever +// order the bundle registered them in. Core's `resolveLocale` is +// `resolveLocaleAmong`; its own suite runs one case list against both +// resolvers. describe('resolveBundleLocale — the platform table language step', () => { const orders: Array<[string, Record]> = [ ['Simplified first', { en: {}, 'zh-CN': {}, 'zh-Hant': {} }], @@ -1060,11 +1061,17 @@ describe('resolveBundleLocale — the platform table language step', () => { for (const [name, bundle] of orders) { describe(name, () => { - it.each(['zh-TW', 'zh-HK', 'zh-MO', 'zh-tw', 'zh-Hant'])('%s reaches zh-Hant', (requested) => { + it.each([ + 'zh-TW', 'zh-HK', 'zh-MO', 'zh-tw', 'zh-Hant', + 'zh-Hant-TW', 'zh-Hant-HK', 'zh-Hant-MO', 'ZH-HANT-tw', + ])('%s reaches zh-Hant', (requested) => { expect(resolveBundleLocale(bundle, requested)).toBe('zh-Hant'); }); - it.each(['zh-CN', 'zh', 'zh-SG', 'zh-Hans', 'zh-cn'])('%s reaches zh-CN', (requested) => { + it.each([ + 'zh-CN', 'zh', 'zh-SG', 'zh-Hans', 'zh-cn', + 'zh-Hans-CN', 'zh-Hans-SG', 'zh-hans-cn', + ])('%s reaches zh-CN', (requested) => { expect(resolveBundleLocale(bundle, requested)).toBe('zh-CN'); }); }); @@ -1079,6 +1086,24 @@ describe('resolveBundleLocale — the platform table language step', () => { it('runs before the base-language step: a bare `zh` key is Simplified', () => { expect(resolveBundleLocale({ zh: {}, 'zh-Hant': {} }, 'zh-TW')).toBe('zh-Hant'); expect(resolveBundleLocale({ 'zh-Hant': {}, zh: {} }, 'zh-CN')).toBe('zh'); + expect(resolveBundleLocale({ zh: {}, 'zh-Hant': {} }, 'zh-Hant-TW')).toBe('zh-Hant'); + expect(resolveBundleLocale({ 'zh-Hant': {}, zh: {} }, 'zh-Hans-CN')).toBe('zh'); + expect(resolveBundleLocale({ en: {}, zh: {}, 'zh-Hant-HK': {} }, 'zh-Hant-TW')).toBe('zh-Hant-HK'); + expect(resolveBundleLocale({ en: {}, 'zh-Hant-HK': {}, zh: {} }, 'zh-Hant-TW')).toBe('zh-Hant-HK'); + }); + + it('reads a key spelled with a script subtag as the script it names, in either registration order', () => { + for (const keys of [['en', 'zh-CN', 'zh-Hant-TW'], ['en', 'zh-Hant-TW', 'zh-CN']]) { + const bundle = Object.fromEntries(keys.map((code) => [code, {}])); + expect(resolveBundleLocale(bundle, 'zh-TW')).toBe('zh-Hant-TW'); + expect(resolveBundleLocale(bundle, 'zh-Hant')).toBe('zh-Hant-TW'); + } + for (const keys of [['en', 'zh-Hant', 'zh-Hans-CN'], ['en', 'zh-Hans-CN', 'zh-Hant']]) { + const bundle = Object.fromEntries(keys.map((code) => [code, {}])); + expect(resolveBundleLocale(bundle, 'zh-CN')).toBe('zh-Hans-CN'); + } + // One script registered: a related language still beats none. + expect(resolveBundleLocale({ en: {}, 'zh-CN': {} }, 'zh-Hant-TW')).toBe('zh-CN'); }); it('keeps the alias-spelled bundles of today: ja → ja-JP, es → es-ES, en → en-US', () => { diff --git a/packages/spec/src/system/platform-locales.test.ts b/packages/spec/src/system/platform-locales.test.ts index 9b049851220..410cf43ef57 100644 --- a/packages/spec/src/system/platform-locales.test.ts +++ b/packages/spec/src/system/platform-locales.test.ts @@ -61,6 +61,12 @@ describe('PLATFORM_LOCALE_READ_ALIASES: stored codes keep resolving', () => { ['zh-TW', 'zh-Hant'], ['zh-HK', 'zh-Hant'], ['zh-MO', 'zh-Hant'], + // The script-subtag spellings of the five regional aliases (#30618). + ['zh-Hans-CN', 'zh-Hans'], + ['zh-Hans-SG', 'zh-Hans'], + ['zh-Hant-TW', 'zh-Hant'], + ['zh-Hant-HK', 'zh-Hant'], + ['zh-Hant-MO', 'zh-Hant'], ['ja-JP', 'ja'], ['es-ES', 'es'], ['pt', 'pt-BR'], @@ -72,13 +78,15 @@ describe('PLATFORM_LOCALE_READ_ALIASES: stored codes keep resolving', () => { expect(platformLocaleTagOf(code)).toBe(tag); }); - it('the map is exactly those ten aliases', () => { + it('the map is exactly those fifteen aliases', () => { expect(PLATFORM_LOCALE_READ_ALIASES).toEqual(Object.fromEntries(ALIASES)); }); - it('the map is typed on its ten literal keys: an unlisted code does not index it as defined', () => { + it('the map is typed on its fifteen literal keys: an unlisted code does not index it as defined', () => { const zhCN: 'zh-Hans' = PLATFORM_LOCALE_READ_ALIASES['zh-CN']; expect(zhCN).toBe('zh-Hans'); + const zhHantTW: 'zh-Hant' = PLATFORM_LOCALE_READ_ALIASES['zh-Hant-TW']; + expect(zhHantTW).toBe('zh-Hant'); // @ts-expect-error `pt-PT` is not a read alias, so the index is not typed as a table tag. expect(PLATFORM_LOCALE_READ_ALIASES['pt-PT']).toBeUndefined(); }); @@ -113,12 +121,19 @@ describe('platformLocaleTagOf', () => { ['EN', 'en'], ['en-us', 'en'], ['ja-jp', 'ja'], + ['ZH-HANS-cn', 'zh-Hans'], + ['zh-hans-sg', 'zh-Hans'], + ['ZH-HANT-tw', 'zh-Hant'], + ['zh-hant-hk', 'zh-Hant'], + ['Zh-Hant-Mo', 'zh-Hant'], ])('ignores case: %s reads as %s', (code, tag) => { expect(platformLocaleTagOf(code)).toBe(tag); }); - // The list is closed: no region or script fallback is guessed. - it.each(['pt-PT', 'de-AT', 'en-GB', 'zh-Hans-CN', 'es-419', 'fr-CA', 'it', 'he', 'zh-Hant-TW'])( + // The list is closed: no region or script fallback is guessed. A + // script-subtag spelling without its own row (`zh-Hant-US`, `zh-Hans-US`) + // answers undefined too: the rows are read one by one, never as a pattern. + it.each(['pt-PT', 'de-AT', 'en-GB', 'es-419', 'fr-CA', 'it', 'he', 'zh-Hant-US', 'zh-Hans-US'])( 'a code the table and the aliases do not list, %s, answers undefined', (code) => { expect(platformLocaleTagOf(code)).toBeUndefined(); diff --git a/packages/spec/src/system/platform-locales.ts b/packages/spec/src/system/platform-locales.ts index ef49c0c854d..baefd8712ce 100644 --- a/packages/spec/src/system/platform-locales.ts +++ b/packages/spec/src/system/platform-locales.ts @@ -59,11 +59,18 @@ export const PLATFORM_LOCALE_TAGS: readonly PlatformLocaleTag[] = Object.freeze( ); /** - * Read aliases: a code that stored values already carry, mapped to the table - * tag it means. Reads only: a new write uses the table tag itself. + * Read aliases: a code that stored values already carry, or that a browser or + * operating system sends, mapped to the table tag it means. Reads only: a new + * write uses the table tag itself. + * + * Each regional Chinese alias also has a row for its script-subtag spelling, + * which names the table tag outright: `zh-Hans-CN` and `zh-Hans-SG` read as + * `zh-Hans`, `zh-Hant-TW`, `zh-Hant-HK` and `zh-Hant-MO` as `zh-Hant`. * * The list is closed. A code it does not name is not guessed at, so there is - * no region fallback here (`pt-PT` is not `pt-BR`, `zh-Hans-CN` is not listed). + * no region or script fallback here: `pt-PT` is not `pt-BR`, and a + * script-subtag spelling without a row (`zh-Hant-US`) answers nothing, because + * the rows are read one by one, never as a pattern. * Frozen. */ export const PLATFORM_LOCALE_READ_ALIASES = Object.freeze({ @@ -73,6 +80,12 @@ export const PLATFORM_LOCALE_READ_ALIASES = Object.freeze({ 'zh-TW': 'zh-Hant', 'zh-HK': 'zh-Hant', 'zh-MO': 'zh-Hant', + // The script-subtag spellings of the five regional aliases above (#30618). + 'zh-Hans-CN': 'zh-Hans', + 'zh-Hans-SG': 'zh-Hans', + 'zh-Hant-TW': 'zh-Hant', + 'zh-Hant-HK': 'zh-Hant', + 'zh-Hant-MO': 'zh-Hant', 'ja-JP': 'ja', 'es-ES': 'es', pt: 'pt-BR', @@ -109,8 +122,9 @@ const TAG_BY_FOLDED_CODE: ReadonlyMap = new Map