Skip to content

docs(validators): add node state-recovery and rollup-outage runbooks - #435

Open
MuncleUscles wants to merge 3 commits into
mainfrom
docs/node-runbooks
Open

MuncleUscles wants to merge 3 commits into
mainfrom
docs/node-runbooks

Conversation

@MuncleUscles

@MuncleUscles MuncleUscles commented Jul 3, 2026 •

Copy link
Copy Markdown
Member

Description

Adds two runbooks for Bradbury Phase 2 validator operators and links to them from the monitoring and upgrade guides.

  • Rollup and sequencer outages. Compare the L2 head across independent providers and inspect the public chain.rpc and chain.websocket health conditions. Legacy strings in the WebSocket log examples are quoted from the released node. The guide distinguishes a provider outage from a chain-wide stall and explains how to verify recovery.
  • State recovery. Verify the source deployment before seeding from a snapshot. Node v0.6.0-rc.5 provides a portable GenVM-only snapshot and a restore command that validates it in an empty data directory. The runbook covers both snapshot recovery and replay from genesis, including preservation of the target validator's private operation records and keystore.

Implementation references

Validation

  • Production build, protocol and Consensus v0.6 documentation checks, generated Node documentation, Markdown exports and code-example lint passed.
  • Verified 19 internal page and section links across both new runbooks. The docs browser suite passed all 173 tests.
  • Ran the eight top-level snapshot restore tests against the node v0.6.0-rc.5 release commit. These cover portable restore, manifest validation, invalid destinations and malformed archives.

Add two operator runbook pages under pages/validators/:

- state-recovery.mdx: recovering a node that has fallen behind or lost
  state. Documents the two real paths — seeding a new node from a
  POST /snapshot archive (create-only endpoint; restore is a manual
  filesystem drop-in, no first-class restore command), and a trustless
  full re-sync from genesis that re-executes history through GenVM.
  Honestly flags where restore tooling is thin.
- rollup-outage.mdx: incident response when the underlying rollup /
  sequencer is unreachable. Detection (doctor + /health zksync-connectivity
  + transport circuit-breaker metrics), why validators are not falsely
  slashed (slashing/quarantine are on-chain; the node skips duties rather
  than mis-voting), status-comms, and automatic resume. Grounded in the
  Bradbury outage of 2026-07-01 as precedent.

Wire both into pages/validators/_meta.json and cross-link setup-guide,
monitoring, upgrade, staking, and slashing.
@netlify

netlify Bot commented Jul 3, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for genlayer-docs ready!

Name Link
🔨 Latest commit 0c6a70d
🔍 Latest deploy log https://app.netlify.com/projects/genlayer-docs/deploys/6a47939fddad5c0008866650
😎 Deploy Preview https://deploy-preview-435--genlayer-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 58 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d0bd127e-c062-4d56-8a89-1e318a6e8fec
Commits

Reviewing files that changed from the base of the PR and between 942e5cb and aa4b6a8.

Files selected for processing: 5

  • pages/validators/_meta.json
  • pages/validators/monitoring.mdx
  • pages/validators/rollup-outage.mdx
  • pages/validators/state-recovery.mdx
  • pages/validators/upgrade.mdx
Finishing Touches

🧪 Generate unit tests (beta)

  • Create PR with unit tests
  • Commit unit tests in branch docs/node-runbooks
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@netlify

netlify Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for genlayer-docs ready!

Name Link
🔨 Latest commit aa4b6a8
🔍 Latest deploy log https://app.netlify.com/projects/genlayer-docs/deploys/6afedb091813b34130c33a00
😎 Deploy Preview https://deploy-preview-435--genlayer-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant