Repository navigation
zlib.Decompress.unconsumed_tail not cleared at end of stream (regression from gh-158169) #158398
Closed as not planned
Description
The test test_decompress_unconsumed_tail_after_eof (added in gh-158169 / #158177) fails on main:
FAIL: test_decompress_unconsumed_tail_after_eof
AssertionError: b'0123456789' != b''
Reproducer:
import zlib
source = b'abcdefghijklmnopqrstuvwxyz'
remainder = b'0123456789'
dco = zlib.decompressobj()
data = dco.decompress(zlib.compress(source) + remainder, 1)
data += dco.decompress(dco.unconsumed_tail)
assert dco.eof
assert data == source
assert dco.unconsumed_tail == b''
assert dco.unused_data == remainderRoot cause:
Commit 8122ff4 (gh-158169) added this line to save_unconsumed_input() in Modules/zlibmodule.c:
self->zst.next_in += left_size;This advances next_in past the leftover input. The following block then computes:
Py_ssize_t left_size = (Byte *)data->buf + data->len - self->zst.next_in;Since next_in was already advanced, left_size is computed incorrectly, so unconsumed_tail is not cleared at end of stream.
Activity
picnixz commentedon Sep 29, 2026
cc @StanFromIreland
picnixz commentedon Sep 29, 2026
3.15 is not affected because backport was not yet merged.
hugovk commentedon Sep 29, 2026
The reproducer and test both pass with no error for me on macOS with latest
main, and the CI and buildbots are green.@Tony123-tech How did you run this? On what OS? How did you build?
Tony123-tech commentedon Sep 29, 2026
Thanks for checking. I'll re-test on latest main tomorrow and report back with my exact commit, OS, and build steps.
picnixz commentedon Sep 29, 2026
I guess the agent was at the wrong main? or something was wrong? I closed the PR because it was likely automated and there was no answer to Hugo's question. We need this answer before your PR. And please, don't use LLMs without checking what they tell you back. They can easily be get out-of-sync (forgot to pull main for instance)
StanFromIreland commentedon Sep 29, 2026
I'm closing this for now, I don't see how this could occur (on any platform), I presume the build didn't include the fix.
Please let us know if that's incorrect, and I will re-open.
Tony123-tech commentedon Sep 30, 2026
You're right — I was testing on an older local main (421c411).
It passes on upstream/main (77c0675). Sorry for the noise.
hosu3das20s05s commentedon Sep 30, 2026
zzffeggs commentedon Sep 30, 2026