Repository navigation
Conversation
When a changeset type deserializes from zero bytes (e.g. `()`), `EntryIter::next` never advanced the file cursor, causing `Store::load` to loop forever. Check that the stream position actually moved after a successful deserialize; if it did not, terminate the iterator. Closes bitcoindevkit#2307
…oindevkit#2258 Clarify that this fix for the zero-width changeset hang is independent of the bincode -> postcard migration tracked in bitcoindevkit#2258. If bitcoindevkit#2258 lands first this check becomes moot, but the regression test is still worth keeping.
|
Nice fix. The position check is a good choice over 1. A related hang isn't covered: collections of zero-sized elements. The check runs only after let mut bytes = MAGIC.to_vec();
bytes.push(0xfd); // varint tag: u64 follows
bytes.extend_from_slice(&u64::MAX.to_le_bytes());
std::fs::write(&path, &bytes).unwrap();
Store::<Vec<()>>::load(MAGIC, &path); // still hangs on this branchbincode deserializes about 2^64 zero-byte elements inside a single call, and its size limit doesn't count them. None of the 2. The hang doesn't need trailing bytes. On 3. Minor: with this change, a |
|
Saw nymius's NACK on #2333 about the bincode → postcard migration in #2258. What this PR does: it fixes the immediate hang under the current bincode-based store — it's not trying to solve or block the postcard migration. If #2258 lands first, this specific check becomes unnecessary, but the regression test in test_zero_width.rs is still worth keeping as a guard against the same bug resurfacing. Moreover bincode isn't going to fix this for us, and #2258 hasn't landed yet, so this is the patch for now. |
`()` decodes at end of file as well as before trailing bytes, so a store holding no entries at all hung on the same offset check. Cover it, and assert the loaded changeset rather than only that `load` returned. Also correct the comment above the check. It claimed the check becomes unnecessary once the postcard migration in bitcoindevkit#2258 lands. It does not: a collection of zero-sized elements with a corrupt length prefix loops inside the decoder and never reaches this point, and postcard has that same hole, so neither this check nor a byte-counting size limit catches it. Say what the check actually covers instead.
|
@Manuel1234477 Thanks, this is a careful review. I checked all three and you're right on each one, including the two you flagged as unverified. 1. Confirmed, Your other hunch is right too, and it goes further than you suggested. I built a standalone postcard test, encoding
2. Confirmed and added as 3. Also accurate. Worth noting #2258 says trailing bytes will be rejected outright, so that divergence goes away on its own, but I agree it's undocumented today. |
Summary
()),EntryIter::nextnever advanced the file cursor, causingStore::loadto loop forever on any trailing bytes after the magic header.deserialize_from, check that the stream position actually moved. If it did not (zero bytes consumed), setfinished = trueand returnOk(None)to terminate the iterator.Store::<()>::loadwith trailing bytes, asserting termination within 5 seconds.Closes #2307