Skip to content

fix(file_store): prevent infinite loop on zero-width changeset types - #2352

Open
luhrhenz wants to merge 3 commits into
bitcoindevkit:masterfrom
luhrhenz:fix/file-store-zero-width-infinite-loop
Open

luhrhenz wants to merge 3 commits into
bitcoindevkit:masterfrom
luhrhenz:fix/file-store-zero-width-infinite-loop

Conversation

@luhrhenz

@luhrhenz luhrhenz commented Oct 8, 2026

Copy link
Copy Markdown

Summary

  • When a changeset type deserializes from zero bytes (e.g. ()), EntryIter::next never advanced the file cursor, causing Store::load to loop forever on any trailing bytes after the magic header.
  • After a successful deserialize_from, check that the stream position actually moved. If it did not (zero bytes consumed), set finished = true and return Ok(None) to terminate the iterator.
  • Add integration test using Store::<()>::load with trailing bytes, asserting termination within 5 seconds.

Closes #2307

When a changeset type deserializes from zero bytes (e.g. `()`),
`EntryIter::next` never advanced the file cursor, causing `Store::load`
to loop forever. Check that the stream position actually moved after a
successful deserialize; if it did not, terminate the iterator.

Closes bitcoindevkit#2307
…oindevkit#2258

Clarify that this fix for the zero-width changeset hang is independent
of the bincode -> postcard migration tracked in bitcoindevkit#2258. If bitcoindevkit#2258 lands
first this check becomes moot, but the regression test is still worth
keeping.
@luhrhenz
luhrhenz requested a review from a team as a code owner October 9, 2026 13:37
@Manuel1234477

Copy link
Copy Markdown

Nice fix. The position check is a good choice over size_of::<T>() == 0. A few notes from testing locally:

1. A related hang isn't covered: collections of zero-sized elements. The check runs only after deserialize_from returns. For example, a Vec<()> (or BTreeSet<()>) entry whose length prefix is corrupted never returns:

let mut bytes = MAGIC.to_vec();
bytes.push(0xfd);                              // varint tag: u64 follows
bytes.extend_from_slice(&u64::MAX.to_le_bytes());
std::fs::write(&path, &bytes).unwrap();
Store::<Vec<()>>::load(MAGIC, &path);          // still hangs on this branch

bincode deserializes about 2^64 zero-byte elements inside a single call, and its size limit doesn't count them. None of the bdk_chain changesets have this shape, so maybe it's out of scope. If so, it'd be good to narrow the comment. I also think a length prefix (#2258) limits an entry's bytes, not its number of elements, so the check might not become moot after all. I haven't verified that.

2. The hang doesn't need trailing bytes. On master, a valid, empty Store::<()> (magic bytes only) also hangs, because () deserializes even at end-of-file. This PR fixes that too. A test for it would be worth adding, and the test could assert the result (Ok with a None changeset), not just that load finishes.

3. Minor: with this change, a () store with trailing garbage loads as Ok(None), while other types return Err(Bincode). That seems fine, but it may be worth a sentence in the docs.

@luhrhenz

luhrhenz commented Oct 9, 2026

Copy link
Copy Markdown
Author

Saw nymius's NACK on #2333 about the bincode → postcard migration in #2258. What this PR does: it fixes the immediate hang under the current bincode-based store — it's not trying to solve or block the postcard migration. If #2258 lands first, this specific check becomes unnecessary, but the regression test in test_zero_width.rs is still worth keeping as a guard against the same bug resurfacing. Moreover bincode isn't going to fix this for us, and #2258 hasn't landed yet, so this is the patch for now.

`()` decodes at end of file as well as before trailing bytes, so a store
holding no entries at all hung on the same offset check. Cover it, and
assert the loaded changeset rather than only that `load` returned.

Also correct the comment above the check. It claimed the check becomes
unnecessary once the postcard migration in bitcoindevkit#2258 lands. It does not: a
collection of zero-sized elements with a corrupt length prefix loops
inside the decoder and never reaches this point, and postcard has that
same hole, so neither this check nor a byte-counting size limit catches
it. Say what the check actually covers instead.
@luhrhenz

luhrhenz commented Oct 10, 2026 •

Copy link
Copy Markdown
Author

@Manuel1234477 Thanks, this is a careful review. I checked all three and you're right on each one, including the two you flagged as unverified.

1. Confirmed, Store::<Vec<()>>::load with your bytes still hangs on this branch. The hang is inside deserialize_from, and my check runs after it returns, so it never gets a turn. I also tested your hunch about the size limit: I applied #2351's with_limit(256 MiB) to this branch and it still hangs, because the limit counts bytes read and zero-sized elements read none.

Your other hunch is right too, and it goes further than you suggested. I built a standalone postcard test, encoding u64::MAX as a sequence-length varint and deserializing Vec<()>, and postcard hangs the same way. So #2258 does not close this, and it does not make this check moot either. That means the comment above the check was simply wrong, and I've corrected it to say what it actually covers rather than claiming the migration supersedes it.

Vec<()> is also reachable rather than hypothetical, since impl<T> Merge for Vec<T> in bdk_core is unconditional. I don't think it can be fixed cleanly here though: stopping that loop means interrupting the decoder partway through its own work, bincode 1.x exposes no sequence-length cap, and capping element count against remaining file bytes cannot work when the elements consume no bytes. Since it survives the migration it wants its own issue, so I've filed it as #2364 and credited you there.

2. Confirmed and added as load_terminates_on_empty_zero_width_store. Magic bytes only, no trailing garbage, and it asserts the changeset is None rather than just that load finished. I checked it hangs without the offset check, so it's a genuinely separate trigger from the trailing-byte case.

3. Also accurate. Worth noting #2258 says trailing bytes will be rejected outright, so that divergence goes away on its own, but I agree it's undocumented today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

file_store: Store::load never terminates when a changeset entry decodes to zero bytes

2 participants