shlex.push_source leaks buffered parent punctuation into pushed source

[augusto-rehfeldt]

Bug report

Bug description:

Documented behaviour: The shlex documentation, under shlex.push_source(), states: "Push an input source stream onto the input stack." The source attribute documentation describes stacked input processing: "When the end of that file is reached, files are popped off the stack until the original input stream is resumed." Together these require the pushed source to be processed before resuming its parent.

Expected: The next token after push_source('b ') is 'b'; the parent's ';' is returned only after the pushed source ends.

Actual: The next token is ';', punctuation from the suspended parent stream.

import shlex

p, c = 'a', 'b'
if not all(s and s.isascii() and s.isalpha() for s in (p, c)):
    print('REFUTATION REJECTED: invalid input')
else:
    L = shlex.shlex(p + ';tail', posix=True, punctuation_chars=';')
    if L.get_token() != p:
        print('REFUTATION REJECTED: parent token was not consumed as required')
    else:
        L.push_source(c + ' ')
        actual, expected = L.get_token(), c
        if actual != expected:
            print('REFUTATION CONFIRMED:', (p, c),
                  'actual:', repr(actual), 'expected:', repr(expected))
        else:
            print('REFUTATION REJECTED: pushed source was processed first')

Output on Python 3.14.6 (Windows-11-10.0.26220-SP0), standard library shlex:

REFUTATION CONFIRMED: ('a', 'b') actual: ';' expected: 'b'

This report was found and written by an automated property-testing tool I run (bugforge). The reproducer above was executed and its output is pasted unedited; no person reviewed the report before it was filed. The search script is in https://github.com/augusto-rehfeldt/bugforge-results/tree/main/shlex-20261003-090557-c3

CPython versions tested on:

3.14

Operating systems tested on:

Windows

[ZeroIntensity]

See #158748 (comment).