Skip to content

feat: detect reorgs through the multidownloader in l1 bridge and claim syncers - #1994

Merged
joanestebanr merged 8 commits into
feat/l1-bridge-claim-multidownloaderfrom
feat/l1-bridge-claim-native-md-driver
Nov 20, 2026
Merged

joanestebanr merged 8 commits into
feat/l1-bridge-claim-multidownloaderfrom
feat/l1-bridge-claim-native-md-driver

Conversation

@joanestebanr

@joanestebanr joanestebanr commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

🔄 Update — 20 November 2026

  • New head 52ab74c8. Adds pending-boundary planning: the sync plans up to the
    pending boundary and does not over-query past it, and empty segments are ignored
    instead of being treated as work.
  • Log/header guard (shared storage / StepSafe): a missing or conflicting log/header
    entry is rejected with the typed ErrLogBlockHeaderMismatch; the guard is nil-safe.
    The original idempotency is retained.
  • Testing (scoped, this revision): -race tests over the multidownloader, claimsync,
    bridgesync, cmd and config
    trees passed — one isolated note: the claim run saw an
    initial EOF and was retried; the retry passed (no assertion or data failure).
    lint on the multidownloader tree: 0 issues.
  • Live replay: IN PROGRESS — no result claimed yet. The historical figures in this body
    are unchanged and still refer to their original heads; they are not re-measured on
    52ab74c8.

🔄 Changes Summary

Stacked on #1952. When the L1 bridge syncer or the L1 claim syncer select Downloader = "multidownloader", they now run on the native multidownloader sync driver (multidownloader/sync, the one l1infotreesync already uses) instead of the legacy sync.EVMDriver + legacy reorg detector. The multidownloader becomes the single source of truth for logs, headers, reorgs and finality.

The goal is that, once this path is tested on real networks, the legacy driver and the legacy reorg detector can be removed for these syncers.

  • Reorg detection: the driver asks the multidownloader whether the last processed block (number + hash) is still valid; on a reorg it rolls the processor back from the first reorged block and re-syncs.
  • Finality: IsBlockFinalized reads the finality flag the multidownloader stores for that block number and hash.
  • Last reorg: BridgeSync.GetLastReorgEvent reports the last reorg the multidownloader detected (new GetLastReorgData in the multidownloader and its storage).
  • Native driver extensions (multidownloader/sync), needed to keep the behaviour fix: integrate l1 bridge and claim multidownloader 1224 #1952 preserved:
    • EVMDownloader: optional logs hook, logs enricher and context-aware appenders (bridge transaction resolution through the shared provider; claim "prefer detailed event" hook).
    • EVMDriver: lazy mode (Sync(ctx, nil) waits for a starting point) and SyncNextBlock bootstrap, for the claim syncer with AutoStart = false. Stale logs (ErrTransactionInclusionMismatch) are retried after a backoff instead of counting as failures. The error counter is reset after a successful step.
    • BlockByBlockProcessor: adapts the existing one-block-per-transaction processors to the driver and keeps the bridge post-commit subscriptions.
  • Migration safety: a syncer switched from legacy to multidownloader is removed from the L1 reorg detector on start (new ReorgDetector.Unsubscribe). Otherwise its leftover tracked blocks would be loaded as a subscription nobody reads, and a reorg on one of them would block the detector for every other subscriber.
  • NewL1WithDownloader and NewStandaloneClaimSyncWithDownloader no longer take a reorg detector. The multidownloader code paths live in their own files (bridgesync_multidownloader.go, claimsync_multidownloader.go).

⚠️ Breaking Changes

  • 🛠️ Config: [BridgeL1Sync].Downloader and [ClaimL1Sync].Downloader now default to "multidownloader" (it was "legacy"). Nodes that do not set the field switch to the shared multidownloader on upgrade, and download the bridge contract history through it once. A configuration with [L1Multidownloader].Enabled = false fails at startup unless it sets Downloader = "legacy" in both L1 sections.
  • 🔌 API/CLI: none.
  • 🗑️ Deprecated Features: none. The legacy path is still available with Downloader = "legacy", and every L2 syncer keeps using it.

📋 Config Updates

  • 🧾 Diff/Config snippet: new defaults. To keep the previous behaviour:
[BridgeL1Sync]
Downloader = "legacy"

[ClaimL1Sync]
Downloader = "legacy"

🔌 API Updates

🔌 Bridge service API

  • GET /last-reorg-event?network_id=0: no interface change. When the L1 bridge syncer runs on the multidownloader, the event now comes from the multidownloader's reorg table instead of the legacy reorg detector.

🔌 Proxy API

  • None.

🔌 Others API

  • None.

✅ Testing

  • 🤖 Automatic:
    • bridgesync: TestMultidownloaderSyncer_SyncsAndHandlesReorg runs the real syncer over an in-memory multidownloader with no reorg detector: sync, subscriptions, finality, reorg rollback, re-sync and last reorg event.
    • claimsync: TestMDLazyIntegration_BootstrapsOnDemand (lazy bootstrap) and TestMDIntegration_ReorgAndFinality.
    • multidownloader/sync: hooks, contextual appenders, lazy mode, SyncNextBlock, BlockByBlockProcessor, IsBlockFinalized.
    • multidownloader, multidownloader/storage: GetLastReorgData. reorgdetector: Unsubscribe.
    • make lint: 0 issues.
  • 🖱️ Manual: not run. This has not been validated against a real node or kurtosis yet.

🐞 Issues

🔗 Related PRs

📝 Notes

  • Upgrade edge case (same as l1infotreesync, issue bug: updating aggkit sometimes produce a stuck situation 'error downloading blocks' #1638): the last block processed by the legacy downloader may be unknown to the multidownloader. At or below the finalized block it is validated against L1 by hash. Above it, the syncer retries until the multidownloader stores the block or it becomes final. If the multidownloader stores a different block at that height, it needs manual intervention. Documented in docs/l1_shared_downloader.md.
  • Halted processor: with the native driver a halted bridge processor is reset to its last committed block and the range is downloaded again, instead of waiting for a reorg as the legacy driver does.
  • Left for the legacy removal: the contextual appender / logs enricher hooks fix: integrate l1 bridge and claim multidownloader 1224 #1952 added to sync.EVMDownloader are no longer used by production code, and l1infotreesync still has its own copy of the finality check that mdrsync.IsBlockFinalized now provides.
  • New test helper package multidownloader/sync/mdrsynctest (in-memory multidownloader).

🤖 Generated with Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T09:35:51.149638Z 9e3880f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9e3880f775

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +45 to +47
rd.subscriptionsLock.Lock()
delete(rd.subscriptions, id)
rd.subscriptionsLock.Unlock()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Unblock pending reorg notifications when unsubscribing

When migrating an L1 syncer from legacy to multidownloader, the detector can already be notifying its restored subscription before Unsubscribe runs: runReorgDetectorL1IfNeeded starts polling before the L1 constructors and migration cleanup execute. If a polling tick detects a reorg during that startup window, notifySubscriber captures the subscription and blocks sending on its unbuffered ReorgedBlock channel. Deleting the map entry here does not unblock that send, and the migrated syncer never reads the channel. Consequently, detectReorgInTrackedList remains stuck in errGroup.Wait, preventing further reorg checks for every remaining legacy subscriber. Remove migrated subscriptions before polling starts, or make pending notifications cancellable by Unsubscribe.

Useful? React with 👍 / 👎.

@joanestebanr
joanestebanr force-pushed the feat/l1-bridge-claim-multidownloader branch 2 times, most recently from cb4400c to c90a8b8 Compare November 19, 2026 11:37
@joanestebanr joanestebanr changed the title feat(sync): detect reorgs through the multidownloader in l1 bridge and claim syncers feat: detect reorgs through the multidownloader in l1 bridge and claim syncers Nov 19, 2026
joanestebanr and others added 2 commits November 19, 2026 14:56
…d claim syncers

When the L1 bridge syncer or the L1 claim syncer select the multidownloader
they now run on the native multidownloader sync driver instead of the legacy
driver and the legacy reorg detector. The multidownloader is the single source
of truth for logs, headers, reorgs and finality.

- multidownloader/sync: logs hook, logs enricher and contextual appenders in
  the downloader; lazy mode and SyncNextBlock bootstrap in the driver;
  BlockByBlockProcessor adapter that keeps the post-commit subscriptions
- multidownloader: GetLastReorgData, used for the last reorg event
- IsBlockFinalized reads the finality flag stored by the multidownloader
- reorgdetector: Unsubscribe, so a syncer migrated from the legacy downloader
  leaves no tracked blocks that would block the detector
- mdrsynctest: in-memory multidownloader for end-to-end syncer tests

Co-Authored-By: Claude Fable 5.2 <[email protected]>
… claim syncers

BridgeL1Sync.Downloader and ClaimL1Sync.Downloader default to
"multidownloader". The L2 sections keep "legacy", the only value they support.

A configuration that disables L1Multidownloader must now select
Downloader = "legacy" explicitly in both L1 sections: there is no silent
fallback.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
@joanestebanr
joanestebanr force-pushed the feat/l1-bridge-claim-native-md-driver branch from 3d85067 to 86c71d8 Compare November 19, 2026 14:14
joanestebanr added a commit that referenced this pull request Nov 19, 2026
The H5 optimizations were written for the leftover legacy EVMDriver. After
#1994 the L1 bridge/claim syncers run on the native multidownloader driver,
so install the claim-trace enricher and the atomic 100-block processor
batch on that path and keep the legacy/L2 constructors one-block-at-a-time.

Refs #1224
Resuming a multidownloader DB that already has logs for a downloaded
range (Safe/Step re-querying a covered window) hit UNIQUE on
logs(address, block_number, log_index) and stalled the whole downloader.

Mirror the block-header path: INSERT, and on a primary-key collision
keep the stored row when the payload matches. A contradictory payload
is ErrLogIntegrityConflict and never overwrites.

Refs #1224
joanestebanr added a commit that referenced this pull request Nov 20, 2026
The H5 optimizations were written for the leftover legacy EVMDriver. After
#1994 the L1 bridge/claim syncers run on the native multidownloader driver,
so install the claim-trace enricher and the atomic 100-block processor
batch on that path and keep the legacy/L2 constructors one-block-at-a-time.

Refs #1224
A log is canonical only within the block identified by (block number,
block hash). Add one shared guard, CheckLogsMatchBlockHeaders /
CheckLogsMatchBlockHeaderMap, and apply it before any state or storage
mutation, replacing the old TODO in saveLogsAndBlocks:

- StepSafe fails closed: a downloaded batch whose log does not match the
  block header retrieved for its block returns the typed
  ErrLogBlockHeaderMismatch before the in-memory state is advanced, so an
  invalid batch never reaches storage.
- SaveEthLogsWithHeaders rejects, before saveLogsAndBlocks, a log whose
  block number has no header (including a nil header entry), a log whose
  hash contradicts the header, and contradictory duplicate headers for the
  same block number.
- SaveEthLogs validates the logs against the block rows derived from the
  logs themselves, catching a later log in the same block that carries a
  different hash instead of letting the first one silently define the
  block.

The guard runs outside the storage lock and before any write, so a
contradictory batch performs no write at all. Empty batches remain valid.
The same guard backs both the step logic and the storage APIs so the rule
cannot drift.

Refs #1224
Query only addresses pending over the whole range; split at join and leave boundaries and ignore empty segments. Refs #1224
Keep conversion consistent with the shared integrity guard, cover mismatched duplicate replays, and document pending-query boundaries. Refs #1224
joanestebanr added a commit that referenced this pull request Nov 20, 2026
The H5 optimizations were written for the leftover legacy EVMDriver. After
#1994 the L1 bridge/claim syncers run on the native multidownloader driver,
so install the claim-trace enricher and the atomic 100-block processor
batch on that path and keep the legacy/L2 constructors one-block-at-a-time.

Refs #1224
@joanestebanr

Copy link
Copy Markdown
Collaborator Author

Validation at 52ab74c + H5 head 32ee7fe: replayed a consistent snapshot of the original Mainnet H5 DBs. DEBUG confirms the first safe query is [26414042,26414124] with only the two info-tree contracts; the next query starts at 26414125 and includes Bridge. The previously conflicting Bridge log 26414064/index37 remains one row with identical payload and header hash. No UNIQUE, panic, fatal or log/header mismatch errors. Bridge/Claim reached 26455471, Info reached 26455404, MD reached 26455471. Bounded functional validation lasted 131.396 seconds, then intentional SIGINT with exit 0; the planned five-minute observation window was NOT completed. Scoped race tests and lint pass; historical reorg E2E failures are separate and no full-CI-green claim is made.

@joanestebanr
joanestebanr merged commit 52ab74c into feat/l1-bridge-claim-multidownloader Nov 20, 2026
68 of 74 checks passed
joanestebanr added a commit that referenced this pull request Nov 20, 2026
The H5 optimizations were written for the leftover legacy EVMDriver. After
#1994 the L1 bridge/claim syncers run on the native multidownloader driver,
so install the claim-trace enricher and the atomic 100-block processor
batch on that path and keep the legacy/L2 constructors one-block-at-a-time.

Refs #1224
@joanestebanr

Copy link
Copy Markdown
Collaborator Author

Review follow-up: head eca4aed adds a per-subscription done channel (including DB-restored subscriptions) closed by Unsubscribe, releasing notifications blocked on ReorgedBlock or ReorgProcessed, with regression tests. E2E now accounts for sparse stored-header rollback ranges on L1 and waits for the L2 claim syncer to reach the paused head before reorging in B2/C1. Scoped reorgdetector/cmd race tests, lint, and the focused range test pass; live E2E results will be verified in CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant