feat(beacon): follow gloas live

CLAUDE.md

@@ -45,7 +45,7 @@ crates/
     └─ test-fixtures/       # Spec-fixture loading (prod dep of rpc's Hive test driver)
   net/
     ├─ api/                 # Actor protocol traits wiring BlockChain ↔ P2P
-    ├─ p2p/                 # libp2p: gossipsub + req-resp (Status, BlocksByRoot, BlocksByRange)
+    ├─ p2p/                 # libp2p: gossipsub + req-resp (Status, BlocksByRoot, BlocksByRange, gloas envelopes)
     │   ├─ src/gossipsub/   # Topic encoding, message handling
     │   ├─ src/req_resp/    # Request/response codec and handlers
     │   └─ src/metrics.rs   # Peer connection/disconnection tracking
@@ -334,12 +334,27 @@ actual_slot = finalized_slot + 1 + relative_index
   - Beacon wire: `validate_messages()` is on, so every beacon message waits for a verdict (~4.2s before gossipsub's cache evicts it). Rules in `state_transition::beacon::gossip` (cheap half inline, stateful half on a bounded `spawn_blocking` task); plumbing in `p2p/src/beacon/verdict.rs`. Lean gossip still auto-forwards
   - Data columns: every check runs in p2p. A column gossip did not accept (`Queue`/`Overloaded`), every fetched column, and parked columns replayed after their parent imports go through `column::chain_checks` in `p2p/src/beacon/column_checks.rs`. The chain actor stores what it gets unchecked; only debug builds re-run `chain_checks` there
   - Beacon subscribes seven global topics plus two node-id-derived subnet families: custody
-    columns and backbone attestation subnets. `beacon_aggregate_and_proof` and
+    columns and backbone attestation subnets. Gloas digests add two more topics,
+    `execution_payload` and `payload_attestation_message` (`BeaconTopics::for_fork`; earlier
+    digests never carry them). Gloas has its own rules for `beacon_block`,
+    `data_column_sidecar` (fork enum `DataColumnSidecar`, fork from the topic's digest),
+    aggregates (`SignedAggregateAndProof::Gloas`, aggregation-bits length bounded before
+    expansion) and attestations (`verify_attestation_payload_status`). Deliberate
+    departures: with no bad-block cache a block seen without a post-state is queued or
+    ignored where the spec rejects; an envelope whose block state is not cached is queued and
+    forwarded to the actor (which verifies every envelope) rather than dropped; payload votes
+    are applied without a second signature check. `beacon_aggregate_and_proof` and
     `beacon_attestation_{subnet_id}` validate in p2p like blocks/columns, with their own
     permit pool. The actor applies only accepted aggregates, attesting indices already
     gossip-verified, held one slot per `validate_on_attestation`'s `current_slot >= data.slot
     + 1`; subnet attestations are verified and relayed but never applied. See
     [`docs/beacon_wire.md`](docs/beacon_wire.md)
+  - Fork-digest rollover happens at runtime (`ForkSchedule`, `beacon::transition`): the next
+    digest's topics are joined `SUBSCRIBE_LEAD_EPOCHS` before its boundary and the previous
+    one's left `UNSUBSCRIBE_LAG_EPOCHS` after, blob-parameter (BPO) boundaries included.
+    **Known gap**: ethrex's `DiscoveryServer` cannot replace the served ENR at runtime, so its
+    `eth2` entry keeps the startup digest until a restart; peers discovering the node after a
+    boundary reject the record
 - **Req/Resp**: Status, BlocksByRoot, BlocksByRange (snappy frame compression + varint length)
   - Beacon adds `beacon_blocks_by_{range,root}/2` alongside its Status/Ping/MetaData/Goodbye set.
     Both serve from the checkpoint-anchored store, and `build_status` advertises it
@@ -349,6 +364,11 @@ actual_slot = finalized_slot + 1 + relative_index
     the chain actor, since the store's head trails a delivered batch by the whole actor
     mailbox) and the store's head (which gossip imports move and the watermark never sees).
     Fetched blocks reach the actor as `BlockSource::Sync`.
+  - Gloas adds `execution_payload_envelopes_by_{range,root}/1`. Server: canonical FULL
+    envelopes only, and the head's envelope only when the head node is FULL. Client: by root
+    on `needs_envelope` (same dedup and retry ladder as block fetches), by range after a
+    gloas span's blocks were handed to the actor, with range sync holding the next batch
+    until the envelope answer is forwarded (the actor keeps unmatched envelopes only briefly)
     Their chunks carry four `<context-bytes>` (the block's own epoch's fork digest), which is why
     `BeaconWire` and the codec carry `genesis_validators_root`. See [`docs/beacon_wire.md`](docs/beacon_wire.md)
 
@@ -451,20 +471,23 @@ against them. `beacon::tests::the_built_in_network_derives_what_it_always_did`
 pins the parsed mainnet config to `Config::mainnet()`, so the file and the
 Rust constant cannot drift apart. Sepolia's config schedules gloas: its
 `GLOAS_*` keys are claimed (`ForkName::Gloas` exists, and `Config` carries the
-fork's schedule, timing and churn fields), and its state transition and fork
-choice exist, but the live follower refuses gloas blocks
-(`process_or_pend_block`) and a gloas anchor (`refuse_unfollowable_fork` in
-`main.rs`), since nothing delivers payload envelopes or payload attestations to
-it, so a Sepolia follower stops tracking the chain at `GLOAS_FORK_EPOCH`
-regardless. `network::warn_if_unfollowed_fork_scheduled` (shared by
-`BuiltInNetwork::resolve` and a loaded `NetworkDir`) warns about this
-explicitly at startup, once per scheduled fork for which `ForkName::is_followed`
-answers no (the one place that says which forks the node follows), separately
-from the ignored-keys warning, which now only ever names heze's keys and
+fork's schedule, timing and churn fields), and the live follower follows it
+(`ForkName::Gloas.is_followed()`): the chain actor imports gloas blocks and
+their payload envelopes, the p2p layer delivers both and the payload votes, and
+the follower crosses `GLOAS_FORK_EPOCH` without a restart (fork-digest
+rollover, see Networking). `refuse_unfollowable_fork` in `main.rs` and
+`network::warn_if_unfollowed_fork_scheduled` (shared by
+`BuiltInNetwork::resolve` and a loaded `NetworkDir`) are now inert guards for
+the next fork the node cannot follow: the first refuses such an anchor with
+`UnsupportedFork`, the second warns once per scheduled fork for which
+`ForkName::is_followed` answers no (the one place that says which forks the
+node follows). The ignored-keys warning only ever names heze's keys and
 `GAS_LIMIT_SCHEDULE`/`INCLUSION_LIST_DUE_BPS`.
 Platåberget (the Glamsterdam testnet) is already past its `GLOAS_FORK_EPOCH`,
-so its checkpoint anchor is a gloas state that `refuse_unfollowable_fork`
-refuses; its entry is there for when the live follower takes gloas blocks. Its
+so its checkpoint anchor is a gloas state: it is accepted, starts with no
+payload known (head EMPTY), and its first FULL child is held until the anchor's
+envelope arrives (best-effort from the checkpoint URL's
+`execution_payload_envelopes` endpoint, else the by-root request). Its
 file says `CONFIG_NAME: 'testnet'` (a placeholder for Prysm), so
 `BuiltInNetwork::resolve` sets every built-in's `CONFIG_NAME` to its
 `--network` name.
@@ -662,6 +685,11 @@ transitions are in `ethlambda-types`, per the section above. Nothing above
   tests inside the module, are `#[cfg_attr(not(feature = ...), ignore)]` so they
   report as ignored rather than silently absent. Turning the feature on is what
   `make test-beacon` does, and it runs `--lib` too so those 15 are not missed.
+  `ethlambda-blockchain` has its own `beacon-spec-tests` feature gating the
+  fixture-driven chain-actor tests (the gloas envelope cases in its `--lib`
+  target, which read `consensus-spec-tests`); `make test-beacon-mainnet` runs
+  them, and without the feature they report as ignored, so a plain `cargo test`
+  needs no download.
 - Every fixture case is its own test, named `<runner>/<fork>/<handler>/<suite>/<case>`,
   so a failure names the case and not the suite around it. The spec binary
   therefore supplies its own harness (`harness = false`), since a case is only
@@ -725,9 +753,33 @@ transitions are in `ethlambda-types`, per the section above. Nothing above
   suites.
 - **Gloas** lives in `stf/gloas.rs`, `stf/epoch/gloas.rs`, `helpers/gloas.rs`,
   the gloas section of `fork_choice.rs`, and `containers/gloas.rs` in
-  `ethlambda-types`. **The live follower does not follow gloas**: it refuses
-  gloas blocks and a gloas anchor until envelopes and payload attestations are
-  delivered to it. `get_head_node` is one bottom-up walk for every fork, with
+  `ethlambda-types`. **The live follower follows gloas.** The chain actor's
+  envelope path is `beacon_envelope.rs`: an envelope waits for its block's
+  post-state, then for every sampled column when the bid has commitments
+  (data availability moves from the block to the envelope), then goes to
+  `on_execution_payload_envelope`; a block whose parent is FULL with an
+  unverified payload is held until that envelope verifies. All queues are
+  capped, aged out and swept at finality, a missing parent envelope is re-asked
+  once a slot (`FetchRequest.needs_envelope`, asked at once when the block is
+  held, whatever its source, since `BlockSource::Sync` also covers by-root
+  answers), an envelope for a block that is stored but not imported is checked
+  against that block's bid and held outside the per-arrival-slot cap
+  (`awaiting_import`, which range catch-up needs), and `beacon_columns.rs` bounds
+  the parking of gloas column sidecars (they carry no signature). Fork-choice
+  events are timed at arrival, not at the slot tick, and the head's payload
+  status is kept with its root (`Store::head_payload_status`, recomputed after
+  a restart). The node stays a follower: the validator-client endpoints still
+  refuse a gloas epoch (`refuse_validator_duties_from_gloas`).
+  With `--execution-endpoint`, an envelope that passes the consensus checks is
+  judged by `engine_newPayloadV5` (INVALID with a null `latestValidHash` is a
+  hash mismatch that refuses only that envelope; INVALID with one condemns the
+  payload via the execution-chain walk; an engine error keeps the envelope and
+  retries once a slot, probing with the oldest), and a gloas head sends
+  `engine_forkchoiceUpdatedV4` with the custody columns (head hash by the head
+  node's FULL or EMPTY status, finalized and safe from the checkpoint blocks'
+  `bid.parent_block_hash`, safe being a fallback for fast confirmation). See
+  [`docs/beacon_engine.md`](docs/beacon_engine.md).
+  `get_head_node` is one bottom-up walk for every fork, with
   the current slot's fork selecting the payload rules and the boost gate, so a
   follower on any network that schedules gloas (Sepolia does) pays work
   proportional to its votes plus blocks every slot and decodes no block in the
@@ -822,11 +874,20 @@ snapshot (`States`) + diff (`StateDiffs`) pairs; `BlockRoots` and `LiveChain`
 index by slot for range serving and fork choice. Attestations and gossip
 signatures are not persisted; they live in in-memory `Store` buffers consumed
 during the tick pipeline. See [`docs/data_storage.md`](docs/data_storage.md)
-for the full reference: what each of the ten tables holds and how it's
+for the full reference: what each of the twelve tables holds and how it's
 keyed, the snapshot/diff reconstruction algorithm, the block-import write
 sequence, pruning rules, what never changes at runtime, and startup/restore
 behavior.
 
+- `ExecutionPayloadEnvelopes` (verified gloas envelopes, keyed by slot and
+  root) and `BlockTimeliness` (the gloas part of `block_timeliness`) are
+  reloaded on resume for the unfinalized window; the PTC vote vectors are
+  reseeded empty and each payload gets a verdict seeded for the execution
+  client to promote. Adding them did not bump `DB_VERSION`.
+- A restart of a beacon data directory currently hits
+  `SpecAssert("block_root in store.unrealized_justifications")` in head
+  computation until lambdaclass/ethlambda#628 (persists unrealized
+  justifications) lands; that is not gloas-specific.
 - `BlockProof` is the only pruned block table (below the finalized
   boundary); `get_signed_block` returns `None` for a pruned finalized block.
 - A `StateDiff` omits `config` and `validators`, trusting they never mutate;

Makefile

@@ -55,6 +55,7 @@ test-beacon: test-beacon-mainnet test-beacon-minimal ## 🧪 Run the Beacon Chai
 
 test-beacon-mainnet: consensus-spec-tests cryptography-specs ## 🧪 Run the Beacon Chain spec tests, mainnet preset
 	$(BEACON_TEST) --features beacon-spec-tests
+	cargo test -p ethlambda-blockchain --lib --profile release-fast --features beacon-spec-tests
 
 test-beacon-minimal: consensus-spec-tests cryptography-specs ## 🧪 Run the Beacon Chain spec tests, minimal preset
 	$(BEACON_TEST) --features beacon-spec-tests,preset-minimal

README.md

@@ -132,9 +132,11 @@ provider. See
 [`ethlambda beacon`](#ethlambda-beacon--the-ethereum-beacon-chain) below for
 the remaining flags.
 
-`--network plataberget` (the Glamsterdam testnet) is built in as well, but
-Platåberget has run gloas since its `GLOAS_FORK_EPOCH` and the follower does
-not take gloas blocks or a gloas anchor yet, so the node cannot follow it yet.
+`--network plataberget` (the Glamsterdam testnet) is built in as well. It has
+run gloas since its `GLOAS_FORK_EPOCH`, which the follower follows: a gloas
+checkpoint anchor is accepted, and payload envelopes are fetched and imported.
+It stays a follower, so the validator-client endpoints still refuse a gloas
+epoch.
 
 ### Lean consensus devnet
 

bin/ethlambda/Cargo.toml

@@ -64,7 +64,7 @@ libc.workspace = true
 # `test-util` for `#[tokio::test(start_paused = true)]`: the checkpoint-sync
 # tests would otherwise wait out the real retry backoff. Dev-only, so the
 # feature never reaches the shipped binary.
-tokio = { workspace = true, features = ["test-util"] }
+tokio = { workspace = true, features = ["test-util", "net", "io-util", "macros", "rt"] }
 # Keygen writes a directory of key files and the benchmark corpus writes a
 # key set, so their tests need a throwaway directory.
 tempfile = "3"

bin/ethlambda/src/beacon.rs

@@ -24,11 +24,10 @@
 //! all. `node`'s checkpoint sync is untouched: a lean node fetches a
 //! *finalized* anchor, which genuinely has no local source.
 
+use ethlambda_p2p::beacon::fork_schedule::ForkSchedule;
 use ethlambda_p2p::beacon::swarm::BeaconWireConfig;
 use ethlambda_types::beacon::config::Config;
-use ethlambda_types::beacon::constants::FAR_FUTURE_EPOCH;
 use ethlambda_types::beacon::containers::BeaconState;
-use ethlambda_types::beacon::fork_digest::{compute_fork_digest, next_fork_boundary};
 use ethlambda_types::beacon::preset;
 use ethlambda_types::beacon::primitives::{Epoch, Root};
 use ethlambda_types::enr::EnrForkId;
@@ -47,22 +46,11 @@ pub fn time_at_epoch(config: &Config, genesis_time: u64, epoch: Epoch) -> u64 {
     genesis_time + epoch * config.seconds_per_slot * preset::SLOTS_PER_EPOCH
 }
 
-/// The `eth2` ENR entry for this chain at this epoch.
-///
-/// `next_fork_*` point at the next boundary that moves the digest, which
-/// includes blob-parameter-only forks. Peers tolerate a difference here by
-/// design: only `fork_digest` has to match.
+/// The `eth2` ENR entry for this chain at this epoch, from the same schedule
+/// the running node follows, so the record it starts with and the one it would
+/// switch to name the same boundaries.
 pub fn enr_fork_id(config: &Config, genesis_validators_root: Root, epoch: Epoch) -> EnrForkId {
-    let fork_digest = compute_fork_digest(config, genesis_validators_root, epoch);
-    // With no boundary ahead, the spec says to repeat the current fork's own
-    // version and name `FAR_FUTURE_EPOCH` as the epoch it activates at.
-    let boundary = next_fork_boundary(config, epoch);
-    let named_epoch = boundary.unwrap_or(epoch);
-    EnrForkId {
-        fork_digest,
-        next_fork_version: config.fork_version(config.fork_at_epoch(named_epoch)),
-        next_fork_epoch: boundary.unwrap_or(FAR_FUTURE_EPOCH),
-    }
+    ForkSchedule::new(config, genesis_validators_root).enr_fork_id(epoch)
 }
 
 /// Ethereum mainnet's genesis `BeaconState`, SSZ-encoded: a test fixture.
@@ -187,28 +175,32 @@ pub fn wire_params(
     );
     ethlambda_p2p::metrics::set_beacon_fork_digest(&digest_hex);
 
-    // The digest is computed once. Crossing a boundary while running strands
-    // this node on topic names nobody publishes to, so say when that is.
+    // The digest moves at every fork and blob-schedule boundary, and the p2p
+    // actor follows it without a restart (`ethlambda_p2p::beacon::transition`):
+    // the next digest's topics are joined ahead of the boundary, the switch
+    // happens at it, and the old topics are left after it, per
+    // `SUBSCRIBE_LEAD_EPOCHS` and `UNSUBSCRIBE_LAG_EPOCHS`. Say what is coming.
     //
-    // The gloas boundary gets its own message rather than the generic one: a
-    // restart there recomputes the digest and resubscribes to the right
-    // topics, the same as any other boundary, but it does not make this
-    // build any more able to import a gloas block or cross a gloas epoch
-    // boundary (the chain actor refuses every gloas block, since nothing
-    // delivers payload envelopes or payload attestations to it yet).
-    // Suggesting a restart "crosses" it would read as a stall a restart
-    // clears, which this is not.
-    match next_fork_boundary(&chain, epoch) {
-        Some(boundary) if boundary == chain.gloas_fork_epoch => warn!(
-            boundary_epoch = boundary,
-            boundary_unix_time = time_at_epoch(&chain, genesis.genesis_time, boundary),
-            "The fork digest changes at this boundary, but this build cannot follow past it: \
-             a restart resubscribes to the right topics, not past the gloas gap itself"
+    // A fork the node does not follow keeps its own warning: crossing its
+    // digest works like any other, but the node stops tracking the chain
+    // there regardless. No fork is in that state today, since gloas is
+    // followed, so the warning is the guard for the next one.
+    let schedule = ForkSchedule::new(&chain, genesis.genesis_validators_root);
+    match schedule.next_boundary_after(epoch) {
+        Some(next) if !next.fork.is_followed() => warn!(
+            boundary_epoch = next.activation_epoch,
+            boundary_unix_time = time_at_epoch(&chain, genesis.genesis_time, next.activation_epoch),
+            fork = next.fork.as_str(),
+            fork_digest = %hex::encode(next.digest),
+            "The fork digest changes at this boundary and the node will switch topics, but this \
+             build cannot follow the fork itself"
         ),
-        Some(boundary) => info!(
-            boundary_epoch = boundary,
-            boundary_unix_time = time_at_epoch(&chain, genesis.genesis_time, boundary),
-            "The fork digest changes at this boundary; restart the node to cross it"
+        Some(next) => info!(
+            boundary_epoch = next.activation_epoch,
+            boundary_unix_time = time_at_epoch(&chain, genesis.genesis_time, next.activation_epoch),
+            fork = next.fork.as_str(),
+            fork_digest = %hex::encode(next.digest),
+            "The fork digest changes at this boundary; the node will cross it without a restart"
         ),
         None => info!("No fork or blob-schedule boundary is scheduled"),
     }
@@ -293,7 +285,7 @@ pub fn wire_params(
 #[cfg(test)]
 mod tests {
     use super::*;
-    use ethlambda_types::beacon::constants::CUSTODY_REQUIREMENT;
+    use ethlambda_types::beacon::constants::{CUSTODY_REQUIREMENT, FAR_FUTURE_EPOCH};
     use ethlambda_types::beacon::fork::ForkName;
 
     /// Mainnet's genesis, 2020-12-01 12:00:23 UTC.