Skip to content

fix: integrate l1 bridge and claim multidownloader 1224 - #1952

Open
joanestebanr wants to merge 10 commits into
developfrom
feat/l1-bridge-claim-multidownloader
Open

joanestebanr wants to merge 10 commits into
developfrom
feat/l1-bridge-claim-multidownloader

Conversation

@joanestebanr

@joanestebanr joanestebanr commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

🔄 Changes Summary

Initial integration scope for issue #1224: the L1 synchronizers can share a single
L1 multidownloader instead of each running its own dedicated EVM downloader.

  • Shared downloader (L1 only): the L1 bridge syncer and the L1 claim syncer select
    their backend independently via the new per-syncer Downloader option
    ("legacy" | "multidownloader"). The zero value resolves to legacy, so the
    existing L1InfoTreeSync multidownloader / default downloader behaviour remains
    unchanged
    . Startup-mode selection logging is not claimed to be byte-for-byte
    identical.
  • L2 remains legacy-only: the L2 bridge/claim syncers reject
    downloader = "multidownloader" at startup — the shared instance is L1-scoped.
  • Startup ordering/validation: every L1 syncer is built and registered with the
    shared multidownloader first, then the multidownloader is initialized and started,
    and only then the L1 consumers and the txn-sender backfill start. Misuse
    (multidownloader requested but not available, or an unsupported selection) fails
    fast with a clear error.
  • Canonical inclusion (header fix): transaction-inclusion storage validates every
    write against the canonical (block_number, block_hash) and only serves inclusions
    whose hash still matches the canonical header, so a reorg cannot serve a stale
    inclusion.
  • REST adaptation + cleanup fix: the public/admin REST server bind is synchronous;
    a failure now returns an error instead of log.Fatalf, so the deferred
    context.CancelFunc runs on abort instead of os.Exit. No REST routing change —
    endpoints and semantics are identical.
  • Storage/docs/tests: new transaction storage (migration 0003), idempotency and
    lifecycle/reorg regression tests, config validation, and
    docs/l1_shared_downloader.md.

Out of scope: the H5 trace-prefetch performance work (post-canonical validation of
the trace window) is tracked separately and is not part of this PR.

🔄 Review follow-up — 16 Nov 2026

The PR head moved from ac08f230 to e0820f79 to address the review threads.
Both new commits are signed (SSH ED25519) and GitHub reports them verified: valid:

  • 7990ca1f — fix(sync): address shared downloader review and test lifecycle
    (7990ca1)
  • e0820f79 — test(bridgeservicefinder): isolate unreachable service fixtures
    (e0820f7)

Key changes:

  • Backfill retry: backfillTable is now a cancellation-aware, bounded-backoff
    scheduler; records that are not covered yet are re-selected on the next pass instead of
    stopping the backfill permanently, while partial success and the legacy transient path
    stay bounded.
  • Scheduler rescans: reorg-aware, per-demand progress with a bounded failed-page queue
    and stale-generation protection, so a tick no longer re-reads all completed history
    (no new performance number claimed).
  • SQLite read lock: dh.mutex is an RWMutex; log reads take the read lock while
    coverage+storage atomicity is preserved under the write lock.
  • Migration floor: the initial LOG floor stays at cfg.InitialBlockNum on purpose
    (reorg replay / historic backfill below L); only the transaction range uses a separate
    L+1 future floor. The migration re-download cost is not claimed to be eliminated.
  • Context: an optional per-call contextual appender passes the active generation ctx
    and keeps the legacy signature; no mutable ctx state is stored.
  • Deep copy: the duplicated clone helper was removed; CopySyncerConfig is the single
    nil-safe deep copy.
  • Test-only APIs: unused derive* / Storage methods were removed and the tests moved
    to the canonical production primitives.

Testing — local whole-repo CI at the new head e0820f79:

MallocNanoZone=0 go test -count=1 -short -race -p 1 -covermode=atomic -timeout 15m ./...

→ exit 0, 88 packages ok, 0 failures (sequential, isolated worktree).

Benchmark scope (unchanged): the Cardona and Mainnet figures below were measured on
the former heads (ac08f230 for H0, 03f4c143 for H5) before this review
follow-up. They are not re-measured on e0820f79 / ccbcea69 and are not a
guarantee for the current revision; the fixed ranges, dates, percentages and proof counts
are unchanged.

⚠️ Breaking Changes

  • 🛠️ Config: New optional key Downloader (legacy | multidownloader) on the
    L1/L2 bridge and claim syncer config sections. Default legacy is backwards
    compatible.
  • 🔌 API/CLI: None. log.Fatalf → returned error on REST bind failure only changes
    the failure path (process aborts with a non-zero exit and a wrapped error), not the
    API surface.
  • 🗑️ Deprecated Features: None.

📋 Config Updates

  • 🧾 Diff/Config snippet:

    [BridgeL1Sync]
    Downloader = "legacy" # or "multidownloader"
    
    [ClaimL1Sync]
    Downloader = "legacy" # or "multidownloader"

    MultiDownloader must be enabled when any L1 consumer selects it.

🔌 API Updates

🔌 Bridge service API

  • 🔌 Bridge service API Update: No endpoint changes. The only behaviour change is
    that a failure to bind the public/admin REST listener aborts startup with a returned
    error (deferred cancellation runs) instead of log.Fatalf/os.Exit. Not a breaking
    interface change.

🔌 Proxy API

  • 🔌 AProxy API Update: N/A.

🔌 Others API

  • 🔌 Others API Update: N/A.

✅ Testing

  • 🤖 Automatic — scoped, over the 7 changed package trees
    (cmd, bridgesync, claimsync, multidownloader, sync, config, types). This is
    not a full-repo run and not E2E (GOWORK = external integration.go.work,
    toolchain go1.27.3):
    • go test -count=1 -short -race over those 7 trees → all ok
      (log: logs/integration-scoped-test.log).
    • golangci-lint run --timeout 5m over the same 7 trees → 0 issues
      (log: logs/pr1-lint.log).
    • make build → exit 0 (log: logs/pr1-build.log; binaries written to the
      git-ignored integration/target/, nothing committed).
    • New coverage: shared-downloader integration (*_md_integration_test.go), lazy
      construction, constructor validation, storage transaction idempotency/regression,
      and cmd startup tests.
  • 🖱️ Manual / E2E: not run in this PR — real-node E2E validation is handled
    separately and is not claimed here.

📊 Performance — legacy vs integrated L1 multidownloader (2026-11-13)

Same fixed window and same historical dataset on both sides: initialBlock = 4,789,186;
fixed LatestBlock targets L1InfoTreeSync = 12,119,079 and Bridge/Claim =
12,119,148
(inclusive spans: 7,329,894 and 7,329,963 blocks). Stock and
integrated runs share the same source-config digest (04e82d69…c924, chainId 11155111)
and produced identical business data inside the window — L1 info leaves 229,510,
bridges 218,283, claims 63,067 (post-target rows are extra and excluded). One
trial per variant, same Docker/RPC host; real RPC load varies run to run.

syncer legacy stock v0.12.0-rc4 integrated L1 MD (H0) Δ
L1InfoTreeSync 2,212.7389 s (36 min 53 s) 2,305.0785 s (38 min 25 s) +4.17 %
Bridge 9,327.2030 s (2 h 35 min 27 s) 2,947.9874 s (49 min 08 s) −68.39 %
Claim 5,256.7631 s (1 h 27 min 37 s) 4,394.8026 s (1 h 13 min 15 s) −16.40 %
Wall-clock (parallel max, not sum) 2 h 35 min 27 s 1 h 13 min 15 s −52.88 %
  • L1InfoTreeSync is not a legacy-vs-MD comparison: it already used the default
    multidownloader in both runs, so its +4.17 % is run-to-run timing variation,
    not an integration effect.
  • Legacy bridge/claim ran independent direct-RPC downloaders (each with its own
    header/log fetching, nothing shared); L1 info stayed on MD. The integrated H0 build
    runs one shared L1 multidownloader for bridge, claim and transaction inclusion,
    with H0 idempotency — and without the H5 trace-prefetch/batching work (separate,
    not in this PR).
  • The three syncers run in parallel: the wall-clock row is the max, not the sum.

Evidence (read-only, no writes): the legacy stock DBs re-checked read-only reproduce the
exact window counts (229,510 / 218,283 / 63,067, block ≤ fixed target); a full
stock-vs-reference comparison over the frozen reference run (…c10cde59) passed every
business/proof/root/history/header check — bridgeStrictEqual, claimsAllMatch,
claimFieldValidityOk, tablesAllEqual, schemaAllMatch, rootHistoryMatch,
headerEventOk, expectFromAddressNulls=0 — and the H0 integrated validation is a
fresh PASS. One trial per variant: absolute seconds are load-sensitive and these Cardona
figures
are not a guarantee for the current PR head (latest develop + REST adaptation),
which was not re-benchmarked end-to-end on Cardona (the mainnet run below is a separate
measurement and does use the current signed revision). Comparison made 2026-11-13 against the
fixed 2026-11-12 targets, not against the live tip.

📊 Performance — Mainnet — 13–14 November 2026

Same fixed window and same historical dataset for every variant: initialBlock = 16,896,718;
frozen LatestBlock targets L1InfoTreeSync = 26,407,018 and Bridge/Claim = 26,407,089
(inclusive spans: 9,510,301 and 9,510,372 blocks). The target snapshot was captured
2026-11-13T15:58:41Z on chainId 1 (latest 26,407,089, finalized 26,407,018); these
results are from 2026-11-14. This is a frozen snapshot, not the live chain head today.
All three variants replay the same public Katana source (identical for every variant; the
source URL/token is never recorded) — shared source-config digest
786fcb8660dee2c015b70aefcdf627a2a6cad6c505c987d9f79158de77d69043.

  • STOCK = stock v0.12.0-rc4 (commit 5453f356…, binary sha256 5b072065…): legacy
    bridge/claim independent direct-RPC downloaders, L1 info on the default multidownloader.
  • H0 = the PR head at measurement time (commit ac08f230…, binary sha256 256c753b…),
    before the 16 Nov review follow-up (current head e0820f79…): one shared L1
    multidownloader for bridge, claim and transaction inclusion, with the canonical-header
    idempotency fix and without the H5 trace-prefetch/batching work.
  • The two new binaries (H0, H5) were built fresh from the then-current signed commits with
    -buildvcs=true and vcsModified=false (embedded vcs.revision matches the signed HEAD);
    the stock binary is the immutable signed v0.12.0-rc4 artifact (hard-linked, sha256 unchanged).
syncer legacy stock v0.12.0-rc4 integrated L1 MD (H0) Change in elapsed time
L1InfoTreeSync 46 min 27 s 55 min 08 s +18.70 %
Bridge 3 h 17 min 30 s 55 min 10 s −72.07 %
Claim 10 h 49 min 31 s 9 h 22 min 52 s −13.34 %
Wall-clock (parallel max, not sum) 10 h 49 min 31 s 9 h 22 min 52 s −13.34 %
  • "Change in elapsed time" = (H0 − stock) / stock × 100, computed from the unrounded times;
    negative = faster.

  • L1InfoTreeSync is not a legacy-vs-MD comparison: it already ran on the default
    multidownloader in all three variants, so its +18.70 % is run-to-run timing variation
    and must not be attributed to the integration feature — the observed timing includes the
    shared workload and provider conditions, not only the feature. The stock Info run is therefore
    never labelled "legacy".

  • The wall-clock row is the max of the three syncers running in parallel (here the Claim
    row), i.e. the single-controller wall clock until all three complete including the final
    validation — not the sum of the three times.

  • One trial per variant, same provider, the existing Docker container kept active, variable RPC
    load and no control run: the differences are not statistically significant and are not a
    guarantee for other hosts/conditions.

  • Compared data (identical in all three comparisons) — from the strict compare
    src_count / loc_count / datasets_equal fields, 0 differences: 124,918 L1InfoTree leaves
    (l1info_leaf), bridges 265,822 (bridge), claims 114,300 (claim).
    These are the business rows written inside the fixed window, not scanned block heights and
    not overshoot rows.

  • Comparison strictness: schema match, integrity, claim proof-field formats (32 comma-separated
    32-byte hashes), metadata/address fields, root history, event hashes, keys and all columns —
    0 diff across H0-vs-STOCK, H5-vs-STOCK and H5-vs-H0, all ok=true with every component
    verdict true. This is a data/format comparison: it does not claim cryptographic on-chain
    proof validation or internal RHT payload equality (tree-node tables are not compared;
    validated=false).

  • 3-variant reference (same frozen inputs; H5 = H0 + prefetch/batching):

    syncer stock 5453f356 H0 ac08f230 H5 03f4c143
    L1InfoTreeSync 46 min 27 s 55 min 08 s 52 min 00 s
    Bridge 3 h 17 min 30 s 55 min 10 s 52 min 02 s
    Claim 10 h 49 min 31 s 9 h 22 min 52 s 3 h 14 min 08 s
    Wall-clock (parallel max) 10 h 49 min 31 s 9 h 22 min 52 s 3 h 14 min 08 s

    H5 total vs legacy stock: −70.11 %. (The H5 numbers are the subject of the successor PR2.)

  • These mainnet percentages are kept separate and are not merged with the Cardona
    percentages above.

  • Evidence (read-only comparisons, no writes): summary.json, reference-integrity.json,
    stock-report.json, h0-report.json, h5-report.json, H0-vs-STOCK.json,
    H5-vs-STOCK.json, H5-vs-H0.json. All three variants: status=completed,
    nodeExit code=0 / SIGINT unexpected=false, harness/reader/fatal = none.

  • Disk / retention (optional note): the mainnet data was produced fresh with no reused cache;
    the only reclaim was an authorized Cardona-only cleanup (case P1, 15 files, ~6.517 GiB)
    plus removal of the H0 intermediate DBs after all three comparisons — the stock + H5 DBs and all
    reports were retained, and the source Docker/config was untouched.

🐞 Issues

🔗 Related PRs

📝 Notes

  • Commits: ac08f2304c2ec70f239ce4551438a67d80c7c1c1
    (feat(sync): share multidownloader between l1 bridge and claim syncers),
    51 files changed, +11788 / −269, plus the 16 Nov review follow-up 7990ca1f and
    e0820f79. Current head e0820f79d17c2545f072e8890fad21e5ff71806b. All signed
    (SSH ED25519).

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T09:09:21.891893Z 8ceb2ab Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac08f2304c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bridgesync/backfill_tx_sender.go Resolved
@joanestebanr
joanestebanr marked this pull request as draft November 13, 2026 14:30
@joanestebanr joanestebanr self-assigned this Nov 13, 2026

@joanestebanr joanestebanr left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review with Claude Code (/code-review, high effort). Findings are deduplicated but were not individually verified, so please double-check before acting on them. A manual security pass over the diff found no exploitable issues (SQL uses placeholders/sqlx.In; RPC transactions are validated against the log before caching; no new endpoints or secret logging).

The four most relevant findings are the first four inline comments (backfill stopping permanently, unbounded tx worker rescans, dh.mutex held across GetEthLogs, and full re-download when switching an existing node to the multidownloader).

Comment thread bridgesync/backfill_tx_sender.go Resolved
Comment thread bridgesync/bridgesync.go Resolved
Comment thread bridgesync/downloader.go Resolved
Comment thread multidownloader/evm_multidownloader.go Resolved
Comment thread multidownloader/types/syncer_config.go Resolved
@joanestebanr
joanestebanr marked this pull request as ready for review November 16, 2026 14:38

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e0820f79d1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread multidownloader/evm_multidownloader.go Outdated
// the cursor are never skipped. Rewinding the cursor (not the
// cached rows) is what preserves correctness: unchanged blocks
// stay a cache hit and only the reorged region is re-examined.
dh.rewindTransactionSchedulerFrom(reorgErr.OffendingBlockNumber)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Rewind transaction scans to the actual first affected block

When a reorg starts before the block where it is detected, this rewinds the transaction scheduler too little. ReorgProcessor.ProcessReorg searches backward for the common ancestor and deletes transactions starting at firstUnaffectedBlock + 1, which can be lower than OffendingBlockNumber. For example, if detection occurs at block 100 but blocks 90–100 are replaced, an already-advanced scheduler resumes at 100 and never scans the replacement logs in blocks 90–99. Those transactions remain missing from the background cache unless a foreground consumer requests them or the service restarts. Rewind using the actual rollback boundary, or reset the schedules after a successful reorg.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid, fixed in c15a55ee (branch rebased onto develop, new head c15a55ee).

  • ReorgProcessor.ProcessReorg now returns the first reorged block (firstUnaffectedBlock + 1, i.e. the actual rollback boundary used by MoveReorgedBlocks) together with the error (multidownloader/types/reorg_processor.go, multidownloader/reorg_processor.go; mock regenerated with make generate-mocks).
  • The run loop rewinds the transaction scheduler from min(firstReorgedBlock, OffendingBlockNumber) instead of OffendingBlockNumber, so replacement logs between the common ancestor and the detection block are re-scanned.
  • Tests: TestEVMMultidownloader_Start/reorg_rewinds_the_transaction_scheduler_to_the_rollback_boundary (schedule at frontier 200, reorg detected at 100 with rollback boundary 90 -> frontier must be 90; it fails with the previous code, 100 != 90), and ProcessReorg tests now assert the returned boundary (including the missing-block case where it is lower than the detection block).

@joanestebanr
joanestebanr force-pushed the feat/l1-bridge-claim-multidownloader branch from e0820f7 to c15a55e Compare November 19, 2026 08:53
@joanestebanr

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c15a55ee38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sync/evmdriver.go
cancel()
// Wait for the whole generation (downloader + block processing) to
// exit cleanly before rolling back and restarting.
d.joinGeneration(downloaderDone, blockProcessingDone)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Make all downloader sends cancelable before joining generations

When the L2 GER downloader is blocked sending to a full download buffer during a reorg, this new join never completes. l2gersync/l2_ger_syncer.go also uses EVMDriver, but both downloaderSovereign.Download (evm_downloader_sovereign.go:138) and downloaderLegacy.Download (evm_downloader_legacy.go:199) still use unconditional downloadedCh <- *block sends. Canceling the generation stops the block-processing consumer without unblocking those sends, so the driver waits forever before rolling back or acknowledging the reorg, also stalling the shared L2 reorg detector. Make those implementations' sends cancellation-aware as well before requiring every downloader generation to be joined.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid, fixed in 8ceb2ab8. The generation join (joinGeneration / waiting on downloaderDone, also in SyncNextBlock) was introduced by this PR; on develop the driver only waits for the block-processing goroutine, so the hang was new here.

  • l2gersync/evm_downloader_sovereign.go and l2gersync/evm_downloader_legacy.go: the block send is now select { case <-ctx.Done(): close(downloadedCh); return; case downloadedCh <- *block: }, matching sync.EVMDownloader.reportBlocks / reportEmptyBlock. These two and sync.EVMDownloader are the only sync.Downloader implementations.
  • Tests: l2gersync/evm_downloader_cancel_test.go (TestDownloaderSovereign_Download_CancelWhileBlockedOnSend, TestDownloaderLegacy_Download_CancelWhileBlockedOnSend) block the downloader on an unread channel, cancel, and require Download to return and close the channel. Both fail (timeout) with the previous unconditional sends.

@joanestebanr

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8ceb2ab8a5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sync/evmdriver.go
cancel()
// Wait for the whole generation (downloader + block processing) to
// exit cleanly before rolling back and restarting.
d.joinGeneration(downloaderDone, blockProcessingDone)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Propagate generation cancellation into the sovereign GER appender

On a sovereign L2, if an injected GER is not yet indexed by the L1 info tree and an RPC stalls, this join can still prevent reorg processing indefinitely. The block sends are now cancelable, but downloaderSovereign.buildAppender creates context.Background() before calling logGERMissingFromL1InfoTree and isGERRemovedFromL2; those functions perform L1/L2 RPCs that cannot observe the canceled generation. Consequently, the downloader cannot exit, and the driver never reaches rollback or acknowledges the reorg to the shared L2 detector. Install a contextual appender for this downloader and pass the active generation context through its RPC calls, including the GlobalExitRootMap call that currently omits CallOpts.Context.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid, fixed in f4e25f79 (branch also rebased onto current develop 5f03b13). It is reachable only because of the generation join this PR adds, so I fixed every appender that runs on an EVMDriver downloader, not only the sovereign one:

  • l2gersync (sovereign): the GER appenders are now installed as contextual appenders (InstallContextualLogAppenders), the context.Background() in the insert handler is gone, and the download context reaches logGERMissingFromL1InfoTree (L1 GER map read, IsUpToDate) and isGERRemovedFromL2 (removal scan and GlobalExitRootMap with CallOpts{Context: ctx}).
  • l2gersync (legacy): findLatestInjectedGER passes CallOpts{Context: ctx} to GlobalExitRootMap.
  • bridgesync: eth_getTransactionByHash and debug_traceTransaction use CallContext with the caller's context, and the contextual BridgeEvent appender is installed on every path, not only the multidownloader one.
  • claimsync: the ClaimEvent and pre-Etrog ClaimEvent handlers (the ones that trace the claim tx) are contextual appenders and trace with CallContext.

Tests (each stalls an RPC until its context is canceled; they hang with the previous code):

  • TestDownloaderSovereign_Download_CancelWhileAppenderRPCStalls: Download returns after cancel.
  • TestBridgeEventAppender_CancelWhileRPCStalls.
  • TestBuildClaimEventHandlerPreEtrog_CancelWhileTraceStalls.

joanestebanr and others added 7 commits November 19, 2026 12:29
Introduce a single shared L1 multidownloader instance consumed by both the
bridge and claim syncers, selected through the new per-syncer `Downloader`
option ("legacy" default, or "multidownloader"). Startup builds and validates
the shared downloader, starts the L1 consumers/backfill only after it is
initialized, and keeps L2 syncers on the legacy downloader.

Also adapt startup to the REST API rework: a failure to bind the public or
admin listener now returns an error instead of log.Fatalf, so the existing
deferred context cancellation runs on abort rather than calling os.Exit.

Includes multidownloader transaction storage (migration 0003), idempotency and
lifecycle regression tests, config validation, and docs.

Refs #1224
Address review feedback on the shared L1 multidownloader integration
(refs #1224):

- backfill: make the transaction requirement retry while coverage
  advances and cancellation-aware, keeping the log frontier at the
  historical InitialBlockNum floor so reorg inspection and old-record
  resolution below the committed checkpoint L stay covered; document
  why an L+1 shortcut is wrong.
- multidownloader: track worker generations on incremental reorgs and
  failed pages, guard atomic coverage/storage reads with an RWMutex,
  factor a single copy helper, and drop unused production APIs while
  retaining their tests.
- bridgesync: wire the contextual per-download bridge appender and fail
  closed (ErrContextualAppendersUnsupported / ErrLogsEnricherUnsupported)
  when the downloader does not expose the hooks, instead of silently
  falling back to a constructor-time appender or lazy resolution.
- test helpers: fix the lifecycle so workers are cancelled and joined
  and the backend is closed before the test ends.

Refs #1224
Make the two RequireAllHealthyOnStart unreachable-service matrix tests
(6a/6b) inject a HealthChecker that reports every probed URL unhealthy,
instead of relying on a closed httptest server.

A resolved trusted-sequencer URL has its port rewritten to
DefaultBridgeServicePort before probing, so when a service happens to be
listening on that fixed port locally the default HTTP probe succeeds and
the tests flake (Start returns nil and the entry is recorded healthy).
Injecting the unhealthy response makes the health-matrix outcome
deterministic and independent of real HTTP reachability; these are unit
tests of the health gating, not of live unreachable HTTP.

The bridge URL default-port assertion and the RequireAllHealthyOnStart
behaviour are retained.

Refs #1224
ProcessReorg searches backwards for the common ancestor and removes data
from firstUnaffectedBlock+1, which can be lower than the block where the
reorg was detected. The background transaction scheduler was rewound
only to OffendingBlockNumber, so replacement logs between the common
ancestor and the detection block were never scanned.

ReorgProcessor.ProcessReorg now returns the first reorged block and the
run loop rewinds the scheduler from min(firstReorgedBlock, offending).

Refs #1224

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The EVMDriver now joins the downloader goroutine of a generation before
rolling back on a reorg or restarting. The legacy and sovereign L2 GER
downloaders sent blocks with an unconditional channel send, so a
downloader blocked on a full buffer after the consumer stopped would
never return, stalling the driver (and the shared L2 reorg detector).

Both sends now select on ctx.Done() and close the channel on
cancellation, like sync.EVMDownloader already does.

Refs #1224

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The EVMDriver joins the downloader of a generation before rolling back on
a reorg, so an appender blocked on an RPC that ignores the generation
context stalls the reorg (and the shared reorg detector) indefinitely.

- l2gersync: the sovereign GER appenders are installed as contextual
  appenders and the L1/L2 GER map reads, the removal scan and the L1 info
  tree status check use the download context (the context.Background()
  in the insert handler is gone). The legacy downloader passes ctx to the
  L2 GlobalExitRootMap read.
- bridgesync: eth_getTransactionByHash and debug_traceTransaction use
  CallContext with the caller context, and the contextual BridgeEvent
  appender is installed on every path, not only the multidownloader one.
- claimsync: the ClaimEvent and pre-Etrog ClaimEvent handlers, which trace
  the claim transaction, are contextual appenders and trace with
  CallContext.

Refs #1224

Co-Authored-By: Claude Opus 5.5 <[email protected]>
ExtractTxnAddresses now fetches the transaction with CallContext bound to
the caller context, so the activity RPC scanner tests must stub
CallContext instead of Call.

Refs #1224

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@joanestebanr
joanestebanr force-pushed the feat/l1-bridge-claim-multidownloader branch from cb4400c to c90a8b8 Compare November 19, 2026 11:37
…ULL from_address

With syncFromInBridges=true the backfill selection requires from_address, but an
asset event whose trace has no bridgeAsset call legitimately resolves a NULL
from_address. The successful update fills txn_sender but leaves from_address
NULL, so the record still matches the selection and backfillTable re-selects and
re-resolves it forever (a no-progress busy loop).

Track the records processed in a run whose from_address legitimately stays NULL
in an in-memory completed set keyed by (block_num, block_pos) and exclude them
from the selection. Only successfully persisted rows are marked; a row whose
extraction failed is never marked, so it stays retryable. No schema change.

Refs #1224
… commit

A page already queued for retry must not advance the incremental transaction
cursor. Below the cursor it is already accounted for; above the cursor it only
appears after a retry-queue overflow rewound the cursor to the oldest queued
page. Committing that overflow page advanced the cursor to pageEnd+1, skipping
every gap between the rewind and the page (the queue that recorded them was just
cleared), silently losing them.

Only a brand new page may advance the cursor now. Add a regression over more
than one maxNewPagesPerScan window with sparse failures that proves the rewound
region is re-covered with no holes.

Refs #1224
Use bounded keyset pagination over (block_num, block_pos) instead of a growing SQL exclusion list. Refs #1224
@joanestebanr
joanestebanr force-pushed the feat/l1-bridge-claim-multidownloader branch from eca4aed to f67517a Compare November 20, 2026 11:51
@joanestebanr

Copy link
Copy Markdown
Collaborator Author

Fresh code review at the previous head found two real issues, fixed by f67517a and its parent 351776a: (1) successful backfill rows whose from_address legitimately remains NULL no longer cause a no-progress loop; selection now uses bounded keyset pagination over (block_num, block_pos), and failed rows remain retryable; (2) retry commits no longer advance the transaction scheduler frontier and skip unscheduled pages after overflow. Regression tests were shown to fail before the fixes and pass afterward; bridgesync/multidownloader -race and lint pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant