Repository navigation
feat: detect reorgs through the multidownloader in l1 bridge and claim syncers - #1994
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9e3880f775
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| rd.subscriptionsLock.Lock() | ||
| delete(rd.subscriptions, id) | ||
| rd.subscriptionsLock.Unlock() |
There was a problem hiding this comment.
Unblock pending reorg notifications when unsubscribing
When migrating an L1 syncer from legacy to multidownloader, the detector can already be notifying its restored subscription before Unsubscribe runs: runReorgDetectorL1IfNeeded starts polling before the L1 constructors and migration cleanup execute. If a polling tick detects a reorg during that startup window, notifySubscriber captures the subscription and blocks sending on its unbuffered ReorgedBlock channel. Deleting the map entry here does not unblock that send, and the migrated syncer never reads the channel. Consequently, detectReorgInTrackedList remains stuck in errGroup.Wait, preventing further reorg checks for every remaining legacy subscriber. Remove migrated subscriptions before polling starts, or make pending notifications cancellable by Unsubscribe.
Useful? React with 👍 / 👎.
cb4400c to
c90a8b8
Compare
…d claim syncers When the L1 bridge syncer or the L1 claim syncer select the multidownloader they now run on the native multidownloader sync driver instead of the legacy driver and the legacy reorg detector. The multidownloader is the single source of truth for logs, headers, reorgs and finality. - multidownloader/sync: logs hook, logs enricher and contextual appenders in the downloader; lazy mode and SyncNextBlock bootstrap in the driver; BlockByBlockProcessor adapter that keeps the post-commit subscriptions - multidownloader: GetLastReorgData, used for the last reorg event - IsBlockFinalized reads the finality flag stored by the multidownloader - reorgdetector: Unsubscribe, so a syncer migrated from the legacy downloader leaves no tracked blocks that would block the detector - mdrsynctest: in-memory multidownloader for end-to-end syncer tests Co-Authored-By: Claude Fable 5.2 <[email protected]>
… claim syncers BridgeL1Sync.Downloader and ClaimL1Sync.Downloader default to "multidownloader". The L2 sections keep "legacy", the only value they support. A configuration that disables L1Multidownloader must now select Downloader = "legacy" explicitly in both L1 sections: there is no silent fallback. Co-Authored-By: Claude Fable 5.2 <[email protected]>
3d85067 to
86c71d8
Compare
The H5 optimizations were written for the leftover legacy EVMDriver. After #1994 the L1 bridge/claim syncers run on the native multidownloader driver, so install the claim-trace enricher and the atomic 100-block processor batch on that path and keep the legacy/L2 constructors one-block-at-a-time. Refs #1224
Resuming a multidownloader DB that already has logs for a downloaded range (Safe/Step re-querying a covered window) hit UNIQUE on logs(address, block_number, log_index) and stalled the whole downloader. Mirror the block-header path: INSERT, and on a primary-key collision keep the stored row when the payload matches. A contradictory payload is ErrLogIntegrityConflict and never overwrites. Refs #1224
The H5 optimizations were written for the leftover legacy EVMDriver. After #1994 the L1 bridge/claim syncers run on the native multidownloader driver, so install the claim-trace enricher and the atomic 100-block processor batch on that path and keep the legacy/L2 constructors one-block-at-a-time. Refs #1224
A log is canonical only within the block identified by (block number, block hash). Add one shared guard, CheckLogsMatchBlockHeaders / CheckLogsMatchBlockHeaderMap, and apply it before any state or storage mutation, replacing the old TODO in saveLogsAndBlocks: - StepSafe fails closed: a downloaded batch whose log does not match the block header retrieved for its block returns the typed ErrLogBlockHeaderMismatch before the in-memory state is advanced, so an invalid batch never reaches storage. - SaveEthLogsWithHeaders rejects, before saveLogsAndBlocks, a log whose block number has no header (including a nil header entry), a log whose hash contradicts the header, and contradictory duplicate headers for the same block number. - SaveEthLogs validates the logs against the block rows derived from the logs themselves, catching a later log in the same block that carries a different hash instead of letting the first one silently define the block. The guard runs outside the storage lock and before any write, so a contradictory batch performs no write at all. Empty batches remain valid. The same guard backs both the step logic and the storage APIs so the rule cannot drift. Refs #1224
Query only addresses pending over the whole range; split at join and leave boundaries and ignore empty segments. Refs #1224
Keep conversion consistent with the shared integrity guard, cover mismatched duplicate replays, and document pending-query boundaries. Refs #1224
The H5 optimizations were written for the leftover legacy EVMDriver. After #1994 the L1 bridge/claim syncers run on the native multidownloader driver, so install the claim-trace enricher and the atomic 100-block processor batch on that path and keep the legacy/L2 constructors one-block-at-a-time. Refs #1224
|
Validation at 52ab74c + H5 head 32ee7fe: replayed a consistent snapshot of the original Mainnet H5 DBs. DEBUG confirms the first safe query is [26414042,26414124] with only the two info-tree contracts; the next query starts at 26414125 and includes Bridge. The previously conflicting Bridge log 26414064/index37 remains one row with identical payload and header hash. No UNIQUE, panic, fatal or log/header mismatch errors. Bridge/Claim reached 26455471, Info reached 26455404, MD reached 26455471. Bounded functional validation lasted 131.396 seconds, then intentional SIGINT with exit 0; the planned five-minute observation window was NOT completed. Scoped race tests and lint pass; historical reorg E2E failures are separate and no full-CI-green claim is made. |
52ab74c
into
feat/l1-bridge-claim-multidownloader
The H5 optimizations were written for the leftover legacy EVMDriver. After #1994 the L1 bridge/claim syncers run on the native multidownloader driver, so install the claim-trace enricher and the atomic 100-block processor batch on that path and keep the legacy/L2 constructors one-block-at-a-time. Refs #1224
|
Review follow-up: head eca4aed adds a per-subscription done channel (including DB-restored subscriptions) closed by Unsubscribe, releasing notifications blocked on ReorgedBlock or ReorgProcessed, with regression tests. E2E now accounts for sparse stored-header rollback ranges on L1 and waits for the L2 claim syncer to reach the paused head before reorging in B2/C1. Scoped reorgdetector/cmd race tests, lint, and the focused range test pass; live E2E results will be verified in CI. |
🔄 Update — 20 November 2026
52ab74c8. Adds pending-boundary planning: the sync plans up to thepending boundary and does not over-query past it, and empty segments are ignored
instead of being treated as work.
StepSafe): a missing or conflicting log/headerentry is rejected with the typed
ErrLogBlockHeaderMismatch; the guard is nil-safe.The original idempotency is retained.
-racetests over the multidownloader, claimsync,bridgesync, cmd and config trees passed — one isolated note: the claim run saw an
initial
EOFand was retried; the retry passed (no assertion or data failure).linton the multidownloader tree: 0 issues.are unchanged and still refer to their original heads; they are not re-measured on
52ab74c8.🔄 Changes Summary
Stacked on #1952. When the L1 bridge syncer or the L1 claim syncer select
Downloader = "multidownloader", they now run on the native multidownloader sync driver (multidownloader/sync, the onel1infotreesyncalready uses) instead of the legacysync.EVMDriver+ legacy reorg detector. The multidownloader becomes the single source of truth for logs, headers, reorgs and finality.The goal is that, once this path is tested on real networks, the legacy driver and the legacy reorg detector can be removed for these syncers.
IsBlockFinalizedreads the finality flag the multidownloader stores for that block number and hash.BridgeSync.GetLastReorgEventreports the last reorg the multidownloader detected (newGetLastReorgDatain the multidownloader and its storage).multidownloader/sync), needed to keep the behaviour fix: integrate l1 bridge and claim multidownloader 1224 #1952 preserved:EVMDownloader: optional logs hook, logs enricher and context-aware appenders (bridge transaction resolution through the shared provider; claim "prefer detailed event" hook).EVMDriver: lazy mode (Sync(ctx, nil)waits for a starting point) andSyncNextBlockbootstrap, for the claim syncer withAutoStart = false. Stale logs (ErrTransactionInclusionMismatch) are retried after a backoff instead of counting as failures. The error counter is reset after a successful step.BlockByBlockProcessor: adapts the existing one-block-per-transaction processors to the driver and keeps the bridge post-commit subscriptions.legacytomultidownloaderis removed from the L1 reorg detector on start (newReorgDetector.Unsubscribe). Otherwise its leftover tracked blocks would be loaded as a subscription nobody reads, and a reorg on one of them would block the detector for every other subscriber.NewL1WithDownloaderandNewStandaloneClaimSyncWithDownloaderno longer take a reorg detector. The multidownloader code paths live in their own files (bridgesync_multidownloader.go,claimsync_multidownloader.go).[BridgeL1Sync].Downloaderand[ClaimL1Sync].Downloadernow default to"multidownloader"(it was"legacy"). Nodes that do not set the field switch to the shared multidownloader on upgrade, and download the bridge contract history through it once. A configuration with[L1Multidownloader].Enabled = falsefails at startup unless it setsDownloader = "legacy"in both L1 sections.Downloader = "legacy", and every L2 syncer keeps using it.📋 Config Updates
🔌 API Updates
🔌 Bridge service API
GET /last-reorg-event?network_id=0: no interface change. When the L1 bridge syncer runs on the multidownloader, the event now comes from the multidownloader's reorg table instead of the legacy reorg detector.🔌 Proxy API
🔌 Others API
✅ Testing
bridgesync:TestMultidownloaderSyncer_SyncsAndHandlesReorgruns the real syncer over an in-memory multidownloader with no reorg detector: sync, subscriptions, finality, reorg rollback, re-sync and last reorg event.claimsync:TestMDLazyIntegration_BootstrapsOnDemand(lazy bootstrap) andTestMDIntegration_ReorgAndFinality.multidownloader/sync: hooks, contextual appenders, lazy mode,SyncNextBlock,BlockByBlockProcessor,IsBlockFinalized.multidownloader,multidownloader/storage:GetLastReorgData.reorgdetector:Unsubscribe.make lint: 0 issues.🐞 Issues
🔗 Related PRs
📝 Notes
l1infotreesync, issue bug: updating aggkit sometimes produce a stuck situation 'error downloading blocks' #1638): the last block processed by the legacy downloader may be unknown to the multidownloader. At or below the finalized block it is validated against L1 by hash. Above it, the syncer retries until the multidownloader stores the block or it becomes final. If the multidownloader stores a different block at that height, it needs manual intervention. Documented indocs/l1_shared_downloader.md.sync.EVMDownloaderare no longer used by production code, andl1infotreesyncstill has its own copy of the finality check thatmdrsync.IsBlockFinalizednow provides.multidownloader/sync/mdrsynctest(in-memory multidownloader).🤖 Generated with Claude Code