Skip to content

Warn and skip byte string literals in the Python extractor (fixes #1190) - #1368

Open
simpleqt wants to merge 1 commit into
python-babel:masterfrom
simpleqt:sq/warn-and-skip-byte-literals
Open

simpleqt wants to merge 1 commit into
python-babel:masterfrom
simpleqt:sq/warn-and-skip-byte-literals

Conversation

@simpleqt

@simpleqt simpleqt commented Oct 6, 2026

Copy link
Copy Markdown

Problem

pybabel extract aborts the whole run when a keyword call receives a byte string literal, e.g. a source file containing _(b'foo') (as produced by some tools converting legacy code):

  File "babel/messages/extract.py", line 619, in extract_python
    messages.append(''.join(buf))
TypeError: sequence item 0: expected str instance, bytes found

Root cause: _parse_python_string() is annotated -> str | None, but for a byte string literal the ast.Constant branch returns the raw bytes value, which later reaches ''.join(buf).

As agreed in #1190, byte string literals are not translatable messages, so they should be skipped with a warning rather than crashing the extraction.

Fix

In _parse_python_string(), check the constant's type: a bytes value now emits a UserWarning naming the literal and returns None, which the extractor already handles as "nothing extracted" (the same way non-constant arguments such as _(hello.there) are skipped).

>>> from io import BytesIO
>>> from babel.messages import extract
>>> buf = BytesIO(b"msg1 = _(b'foo')\nmsg2 = _('bar')\n")
>>> list(extract.extract('python', buf, extract.DEFAULT_KEYWORDS, []))
...UserWarning: Byte string literal b'foo' is not a translatable message and was not extracted.
[(2, 'bar', [], None)]

Tests

test_extract_byte_string_literal_warns_and_skips — red on master (reproduces the exact TypeError from the issue), green with this change. Full suite: 31 failed, 6277 passed with the fix vs 32 failed, 6276 passed on master — the only delta is the new test; the remaining failures are the pre-existing CLDR-data-drift cases (this checkout runs against release locale data).

Fixes #1190.


Disclosure: this pull request was prepared with the assistance of an AI coding agent (analysis, patch and tests), and was reviewed and submitted by a human contributor.

`_(b'foo')` crashed `pybabel extract` with
`TypeError: sequence item 0: expected str instance, bytes found`
because `_parse_python_string` returned the raw `bytes` value of an
`ast.Constant`, violating its `str | None` annotation, and the value
was later fed to `''.join(buf)`.

Byte string literals are not translatable messages, so following the
behavior agreed in python-babel#1190 they are now skipped with a UserWarning instead
of aborting the whole extraction run.

Fixes python-babel#1190.
Copilot AI balanced review requested due to automatic review settings October 6, 2026 10:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pybabel extract crashes when it encounters a byte string

2 participants