Repository navigation
Conversation
Signed-off-by: Sai Asish Y <[email protected]>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reading the same source messages from several PO catalogs currently allocates a separate message-ID string for each catalog. Share normalized singular IDs and each plural component through
sys.intern()when the parser retains a message. Translations, contexts and mutable message metadata remain independent; ignored obsolete IDs are not interned.Enable this only on conventional CPython builds. Free-threaded CPython keeps interned strings until shutdown even when its GIL is enabled, so those builds and other Python implementations keep the existing behavior.
Refs #571.
For eight catalogs with 128 long IDs each, fresh-process
tracemallocmeasurements on conventional CPython 3.13 gave:CPython 3.14 reproduced the same differences. Shared IDs went from 1,024 distinct string objects to 128. This favors overlapping catalogs: the unique-ID case adds 415,088 bytes of interning bookkeeping, and that difference remained after releasing catalogs. In 100 successive batches of 128 unique IDs, the retained difference plateaued at the same value; this is an observation, not a bound for all workloads. These are traced Python allocations, not total RSS or a speedup claim.
Inputs were prepared before tracing, with one warm-up parse and
locale=None. Separate cold-import measurements included the build guard: the median retained-allocation difference was 709 bytes on 3.13 and 763 bytes on 3.14 over three pairs. The baseline already initializessysconfigthroughzoneinfoon those runtimes; this does not establish the same cost on every supported runtime.Validation: the original sharing assertions fail on upstream and pass with sharing enabled; the free-threaded/other-implementation guard assertions fail before the guard and pass afterward. All 129 targeted parser/catalog cases pass on CPython 3.13 and 3.14, and 33 writer/MO compatibility cases pass on 3.13, using the complete imported CLDR 48.2 data. The configured Ruff 0.14.10 check passes. Runtime branch tests simulate build configuration; no real free-threaded or PyPy runtime was available locally.
Shared/unique/Unicode memory reproducer
Run this in a fresh process for each case (
shared,unique, orunicode) on upstream and the PR branch. The source checkout must be importable asbabel.