diff --git a/CLAUDE.md b/CLAUDE.md index 61083878..0a8186d0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -767,8 +767,24 @@ transitions are in `ethlambda-types`, per the section above. Nothing above the parking of gloas column sidecars (they carry no signature). Fork-choice events are timed at arrival, not at the slot tick, and the head's payload status is kept with its root (`Store::head_payload_status`, recomputed after - a restart). The node stays a follower: the validator-client endpoints still - refuse a gloas epoch (`refuse_validator_duties_from_gloas`). + a restart). The node also serves gloas validator duties, self-build only (no + bids, no proposer preferences; see `docs/spec_deviations.md`). + `state_transition/src/beacon/gloas_block_production.rs` assembles the block and + its envelope (`gloas_payload_inputs`, `parse_gloas_execution_requests`, + `pack_gloas_attestations`, `pack_payload_attestations`, `assemble_gloas_block`, + `gloas_data_column_sidecars`); `rpc/src/beacon/gloas_proposal.rs` serves + `produceBlockV4` and the envelope endpoints (cache of the current and previous + slot's builds; the envelope POST waits up to 4 s for its block, then gossips + the envelope and all column sidecars through `RpcToP2P`, relying on gossipsub + fanout for subnets the node does not subscribe to) and `rpc/src/beacon/ptc.rs` + the PTC duties, payload attestation data and pool. One + `SharedPayloadAttestationPool` is shared by p2p (filled on accepted gossip) and + rpc. The store keeps an in-memory envelope-arrival record + (`insert_beacon_envelope_seen`, keyed by root, pruned with the el-hash cache, + not persisted) that `payload_present` reads against + `fork_choice::get_payload_due_ms`; `helpers::gloas::get_ptc_assignments` backs + the duties. `attestation_data.index` at a gloas slot comes from + `Store::head_payload_status`. With `--execution-endpoint`, an envelope that passes the consensus checks is judged by `engine_newPayloadV5` (INVALID with a null `latestValidHash` is a hash mismatch that refuses only that envelope; INVALID with one condemns the @@ -776,8 +792,21 @@ transitions are in `ethlambda-types`, per the section above. Nothing above retries once a slot, probing with the oldest), and a gloas head sends `engine_forkchoiceUpdatedV4` with the custody columns (head hash by the head node's FULL or EMPTY status, finalized and safe from the checkpoint blocks' - `bid.parent_block_hash`, safe being a fallback for fast confirmation). See + `bid.parent_block_hash`, safe being a fallback for fast confirmation). Building + a gloas payload for `produceBlockV4` uses `PayloadAttributesV4` (V3's fields + plus `slot_number` and `target_gas_limit`, the parent bid's gas limit) through + `forkchoice_updated_v4_with_attributes` and `get_payload_v6` + (`BuiltGloasPayload`, `ExecutionPayloadV4`). See [`docs/beacon_engine.md`](docs/beacon_engine.md). + The validator client (`crates/validator`) at a gloas slot runs: propose at the + boundary (block published bare, then the proposer-signed envelope with + `Eth-Blob-Data-Included: true`, both inside the proposal's attester-offset + budget; an envelope failure is logged and counted, not a failed proposal), + attest at `ATTESTATION_DUE_BPS_GLOAS`, aggregate at `AGGREGATE_DUE_BPS_GLOAS`, + then the PTC vote at `PAYLOAD_ATTESTATION_DUE_BPS`. `SlotClock` picks the + offsets by each slot's own fork, and `/eth/v1/config/spec` supplies them. It + still keeps no slashing-protection record; PTC votes have an in-memory + `(validator, slot)` dedup only, and envelopes are unguarded. `get_head_node` is one bottom-up walk for every fork, with the current slot's fork selecting the payload rules and the boost gate, so a follower on any network that schedules gloas (Sepolia does) pays work diff --git a/bin/ethlambda/src/main.rs b/bin/ethlambda/src/main.rs index d8f32c6c..f5037732 100644 --- a/bin/ethlambda/src/main.rs +++ b/bin/ethlambda/src/main.rs @@ -717,12 +717,17 @@ async fn run_node(options: Options) -> eyre::Result<()> { // read by the aggregate endpoint and block production; unused on lean. let attestation_pool = ethlambda_state_transition::beacon::attestation_pool::SharedAttestationPool::default(); + // Filled by gossip and the Beacon API's payload attestation endpoint, read + // by block production and `GET .../pool/payload_attestations`. + let payload_attestation_pool = + ethlambda_state_transition::beacon::payload_attestation_pool::SharedPayloadAttestationPool::default(); let p2p = P2P::spawn( built, setup.store.clone(), setup.node_names, discovery, attestation_pool.clone(), + payload_attestation_pool.clone(), ) .await .wrap_err("failed to start discv5 discovery")?; @@ -734,6 +739,12 @@ async fn run_node(options: Options) -> eyre::Result<()> { let rpc_sync_status = sync_status.clone(); let rpc_events = events.clone(); let rpc_p2p = p2p.actor_ref().to_rpc_to_p2p_ref(); + let rpc_custody_columns = match &setup.chain { + ChainActor::Beacon { + custody_columns, .. + } => ethlambda_rpc::CustodyColumns(custody_columns.clone()), + ChainActor::Lean(..) => ethlambda_rpc::CustodyColumns::default(), + }; // Block production builds its payloads with the same execution client the // chain actor validates them with. let rpc_engine = match &setup.chain { @@ -757,6 +768,8 @@ async fn run_node(options: Options) -> eyre::Result<()> { ethlambda_rpc::BeaconApiHandles { p2p: rpc_p2p, attestation_pool: attestation_pool.clone(), + payload_attestation_pool: payload_attestation_pool.clone(), + custody_columns: rpc_custody_columns, engine: rpc_engine, }, local_peer_id, diff --git a/crates/blockchain/src/beacon_envelope.rs b/crates/blockchain/src/beacon_envelope.rs index ca204601..a893a651 100644 --- a/crates/blockchain/src/beacon_envelope.rs +++ b/crates/blockchain/src/beacon_envelope.rs @@ -562,6 +562,11 @@ impl BlockChainServer { ); self.store .insert_beacon_block_payload_status(root, slot, status); + // When it arrived, not when it finished verifying: a held envelope is + // verified late, and the committee's `payload_present` asks whether it + // was seen on time. + let seen_ms = held.event_ms.saturating_sub(config.genesis_time_ms()); + self.store.insert_beacon_envelope_seen(root, slot, seen_ms); // Lets `latestValidHash` find this FULL node. self.store.insert_beacon_el_block_hash( root, diff --git a/crates/blockchain/state_transition/src/beacon/fork_choice.rs b/crates/blockchain/state_transition/src/beacon/fork_choice.rs index 59718c41..42296d1c 100644 --- a/crates/blockchain/state_transition/src/beacon/fork_choice.rs +++ b/crates/blockchain/state_transition/src/beacon/fork_choice.rs @@ -3530,6 +3530,13 @@ pub fn get_payload_attestation_due_ms(config: &Config) -> u64 { get_slot_component_duration_ms(config.payload_attestation_due_bps, config) } +/// `get_payload_due_ms` (gloas `fork-choice.md`): how far into a slot, in +/// milliseconds, an execution payload envelope must have been seen for the +/// payload timeliness committee to vote `payload_present`. +pub fn get_payload_due_ms(config: &Config) -> u64 { + get_slot_component_duration_ms(config.payload_due_bps, config) +} + // --------------------------------------------------------------------------- // Merge transition helpers (bellatrix) // --------------------------------------------------------------------------- diff --git a/crates/blockchain/state_transition/src/beacon/gloas_block_production.rs b/crates/blockchain/state_transition/src/beacon/gloas_block_production.rs new file mode 100644 index 00000000..a9800cba --- /dev/null +++ b/crates/blockchain/state_transition/src/beacon/gloas_block_production.rs @@ -0,0 +1,1079 @@ +//! Assembling a self-built gloas block and the payload envelope that fulfills +//! it, per gloas `validator.md` ("Constructing the `BeaconBlockBody`") and +//! `builder.md` ("Constructing the `SignedExecutionPayloadBid`" and +//! "Constructing the `SignedExecutionPayloadEnvelope`"). +//! +//! Gloas splits what fulu's [`super::block_production`] assembles in one piece: +//! the block commits to a bid, and the payload travels afterward in an +//! envelope. This node only ever builds for itself (`BUILDER_INDEX_SELF_BUILD`), +//! so the bid is its own: zero value, the G2 point at infinity for a +//! signature, and every field read off the payload its execution client built. +//! The proposer then owes the network the envelope (signed with its own key, +//! see `process_execution_payload_bid`'s self-build branch) and the data +//! columns of the blobs the bid commits to; [`assemble_gloas_block`] returns +//! the unsigned envelope next to the block. +//! +//! Everything the execution client supplies arrives already built. What this +//! module decides is what the consensus layer decides: which parent payload +//! the build extends ([`gloas_payload_inputs`]), which attestations and +//! payload attestations ride in the body, and the state root, which comes from +//! running the block through gloas's `process_block` on a copy of the state. + +use std::collections::BTreeMap; + +use super::block_production::{empty_sync_aggregate, pack_attestations}; +use super::bls; +use super::config::Config; +use super::error::{Error, Result, verify}; +use super::helpers::accessors::{ + CommitteeCache, get_beacon_proposer_index, get_current_epoch, get_randao_mix, +}; +use super::helpers::gloas::{ + get_indexed_payload_attestation, get_ptc, gloas_state_ref, is_attestation_same_slot, + is_valid_indexed_payload_attestation, +}; +use super::stf; +use ethlambda_types::beacon::{ + constants, + containers::{ + BeaconState, + capella::Withdrawal, + electra, + gloas::{ + self, BeaconBlock, BeaconBlockBody, ExecutionPayload, ExecutionPayloadBid, + ExecutionPayloadEnvelope, ExecutionRequests, PayloadAttestation, + PayloadAttestationData, PayloadAttestationMessage, SignedExecutionPayloadBid, + }, + }, + preset, + primitives::{ + BlsSignature, Bytes32, ExecutionBlockHash, HashTreeRoot as _, KzgCommitment, Root, Slot, + ValidatorIndex, + }, +}; + +/// What the execution client needs to build the payload for the slot `state` +/// has been advanced to: `PayloadAttributesV4`, less the fee recipient, plus +/// the execution block the build extends. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct GloasPayloadInputs { + pub timestamp: u64, + pub prev_randao: Bytes32, + pub withdrawals: Vec, + /// The execution block the payload must extend: the head of the + /// `forkchoiceUpdated` that starts the build. + pub head_block_hash: ExecutionBlockHash, + /// `hash_tree_root(state.latest_block_header)`. + pub parent_beacon_block_root: Root, + pub slot_number: u64, + pub target_gas_limit: u64, +} + +/// The payload inputs for the block at `state`'s slot, from `prepare_execution_payload` +/// (gloas `validator.md`). +/// +/// `build_on_full` is `should_build_on_full(store, head, current_slot)`: the +/// proposer builds on the parent's revealed payload, so the parent's requests +/// (`parent_requests`, empty when the parent is pre-gloas) are applied to a +/// copy of the state first, and the withdrawals are those of that copy. Else +/// the parent's payload is skipped and the withdrawals are the ones the state +/// already expects. The target gas limit is the parent bid's own (contract +/// decision: this node expresses no preference of its own). +pub fn gloas_payload_inputs( + state: &BeaconState, + build_on_full: bool, + parent_requests: &ExecutionRequests, + config: &Config, +) -> Result { + let inner = gloas_state_ref(state, "gloas_payload_inputs")?; + let parent_bid = &inner.latest_execution_payload_bid; + let (withdrawals, head_block_hash) = if build_on_full { + let mut applied = state.clone(); + stf::gloas::apply_parent_execution_payload(&mut applied, parent_requests, config)?; + ( + stf::gloas::get_expected_withdrawals(&applied)?.withdrawals, + parent_bid.block_hash, + ) + } else { + ( + inner.payload_expected_withdrawals.iter().cloned().collect(), + parent_bid.parent_block_hash, + ) + }; + Ok(GloasPayloadInputs { + timestamp: stf::bellatrix::compute_timestamp_at_slot(state, state.slot(), config), + prev_randao: get_randao_mix(state, get_current_epoch(state)), + withdrawals, + head_block_hash, + parent_beacon_block_root: state.latest_block_header().hash_tree_root(), + slot_number: state.slot(), + target_gas_limit: parent_bid.gas_limit, + }) +} + +/// `get_execution_requests` (gloas `validator.md`): the execution client's +/// EIP-7685 request list into gloas's five-kind `ExecutionRequests`. +/// +/// The types must be strictly ascending and no element empty, which is +/// checked rather than trusted, since a list this node accepted would go into +/// a bid its peers then reject. +pub fn parse_gloas_execution_requests(list: &[Vec]) -> Result { + let mut requests = ExecutionRequests::default(); + let mut previous_type: Option = None; + for element in list { + let (&request_type, data) = element + .split_first() + .ok_or(Error::SpecAssert("an execution request carries its type"))?; + verify(!data.is_empty(), "an execution request is not empty")?; + verify( + previous_type.is_none_or(|previous| request_type > previous), + "execution request types are strictly ascending", + )?; + previous_type = Some(request_type); + let malformed = |_| Error::SpecAssert("an execution request list decodes as SSZ"); + match request_type { + constants::DEPOSIT_REQUEST_TYPE => { + requests.deposits = libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::WITHDRAWAL_REQUEST_TYPE => { + requests.withdrawals = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::CONSOLIDATION_REQUEST_TYPE => { + requests.consolidations = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::BUILDER_DEPOSIT_REQUEST_TYPE => { + requests.builder_deposits = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::BUILDER_EXIT_REQUEST_TYPE => { + requests.builder_exits = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + _ => return Err(Error::SpecAssert("a known execution request type")), + } + } + Ok(requests) +} + +/// Pick the attestations for a gloas block at `state`'s slot from the +/// electra-shaped pool `candidates`. +/// +/// Everything [`pack_attestations`] enforces, plus gloas `process_attestation`'s +/// payload-availability rule: `data.index` is `0` or `1`, and `0` when the +/// vote is for the block of its own slot (`is_attestation_same_slot`), which +/// `get_attestation_participation_flag_indices` asserts. A vote failing that +/// would fail the whole block, so it is left out here. +pub fn pack_gloas_attestations( + state: &BeaconState, + candidates: Vec, +) -> Vec { + let includable = |attestation: &electra::Attestation| { + let data = &attestation.data; + data.index < 2 + && is_attestation_same_slot(state, data) + .is_ok_and(|same_slot| !same_slot || data.index == 0) + }; + pack_attestations(state, candidates.into_iter().filter(includable).collect()) + .iter() + .map(gloas::Attestation::from) + .collect() +} + +/// The payload attestations for a block at `state`'s slot, aggregated from the +/// pool's `messages` (gloas `validator.md`, "Payload attestations"). +/// +/// Only the parent block's own slot is attested to: the votes must be for +/// `parent_root`, and the parent must sit exactly one slot before this block. +/// Votes sharing one `PayloadAttestationData` become one aggregate. Its bit +/// `i` is set for every position `i` of `get_ptc(state, parent_slot)` whose +/// validator has a vote on that data, and that vote's signature is aggregated +/// once per such position: a validator holding two seats sets two bits and +/// contributes twice, which is what `get_indexed_payload_attestation` followed +/// by `FastAggregateVerify` over the repeated key expects. +/// +/// An aggregate that does not verify against `state` is dropped, since one bad +/// operation fails the whole block. At most `MAX_PAYLOAD_ATTESTATIONS` are +/// returned, the ones covering the most seats first. +pub fn pack_payload_attestations( + state: &BeaconState, + parent_root: Root, + parent_slot: Slot, + messages: Vec, + config: &Config, +) -> Vec { + if parent_slot.checked_add(1) != Some(state.slot()) { + return Vec::new(); + } + let messages = messages.into_iter().filter(|message| { + message.data.beacon_block_root == parent_root && message.data.slot == parent_slot + }); + let mut packed = aggregate_payload_attestations(state, parent_slot, messages, config); + packed.sort_by_key(|(seats, _)| std::cmp::Reverse(*seats)); + packed + .into_iter() + .map(|(_, attestation)| attestation) + .take(preset::MAX_PAYLOAD_ATTESTATIONS as usize) + .collect() +} + +/// Aggregate the votes of `messages` cast at `slot` into one +/// [`PayloadAttestation`] per distinct `PayloadAttestationData`, each paired +/// with the number of committee seats it covers. +/// +/// The committee is `get_ptc(state, slot)`, so `state` must be within the +/// window that function reads (the slot's epoch, the one after it, or one +/// past `state`'s). Votes for another slot are skipped, and an aggregate that +/// does not verify against `state` is dropped. The order is the data roots', +/// not a ranking: callers that must cap the count sort by seats themselves. +pub fn aggregate_payload_attestations( + state: &BeaconState, + slot: Slot, + messages: impl IntoIterator, + config: &Config, +) -> Vec<(usize, PayloadAttestation)> { + let Ok(ptc) = get_ptc(state, slot, config) else { + return Vec::new(); + }; + + // One entry per distinct data: the first signature each validator sent. + let mut groups: BTreeMap< + Root, + ( + PayloadAttestationData, + BTreeMap, + ), + > = BTreeMap::new(); + for message in messages { + if message.data.slot != slot { + continue; + } + groups + .entry(message.data.hash_tree_root()) + .or_insert_with(|| (message.data, BTreeMap::new())) + .1 + .entry(message.validator_index) + .or_insert(message.signature); + } + + groups + .into_values() + .filter_map(|(data, votes)| { + let mut aggregation_bits = gloas::PayloadTimelinessCommitteeBits::default(); + let mut signatures = Vec::new(); + for (position, validator) in ptc.iter().enumerate() { + if let Some(signature) = votes.get(validator) { + aggregation_bits.set(position, true).ok()?; + signatures.push(*signature); + } + } + let seats = signatures.len(); + let attestation = PayloadAttestation { + aggregation_bits, + data, + signature: bls::aggregate(&signatures).ok()?, + }; + let indexed = get_indexed_payload_attestation(state, &attestation, config).ok()?; + is_valid_indexed_payload_attestation(state, &indexed).then_some((seats, attestation)) + }) + .collect() +} + +/// `get_data_column_sidecars` (gloas `builder.md`): the `NUMBER_OF_COLUMNS` +/// column sidecars of the blobs a payload carries, named by the block they +/// belong to rather than by a signed header and an inclusion proof. +/// +/// `blobs` are the raw blobs and `cell_proofs` the execution client's +/// flattened cell proofs, `CELLS_PER_EXT_BLOB` per blob in blob order +/// (`BlobsBundleV2`). The cells are recomputed from the blobs, since the +/// bundle carries only proofs. Fails when the proof count does not match, or +/// a blob is not valid. +pub fn gloas_data_column_sidecars( + beacon_block_root: Root, + slot: Slot, + blobs: &[Vec], + cell_proofs: &[ethlambda_types::beacon::primitives::KzgProof], +) -> Result> { + verify( + cell_proofs.len() == blobs.len() * preset::CELLS_PER_EXT_BLOB, + "a blob carries CELLS_PER_EXT_BLOB cell proofs", + )?; + let per_blob_cells = blobs + .iter() + .map(|blob| super::kzg::compute_cells(blob)) + .collect::>>()?; + + (0..preset::NUMBER_OF_COLUMNS) + .map(|column_index| { + let mut column = Vec::with_capacity(blobs.len()); + let mut kzg_proofs = Vec::with_capacity(blobs.len()); + for (blob, cells) in per_blob_cells.iter().enumerate() { + let cell = ethlambda_types::beacon::containers::fulu::Cell::try_from( + cells[column_index].to_bytes().to_vec(), + ) + .map_err(|_| Error::SpecAssert("len(cell) == BYTES_PER_CELL"))?; + column.push(cell); + kzg_proofs.push(cell_proofs[blob * preset::CELLS_PER_EXT_BLOB + column_index]); + } + Ok(gloas::DataColumnSidecar { + index: column_index as u64, + column: column.into(), + kzg_proofs: kzg_proofs.into(), + slot, + beacon_block_root, + }) + }) + .collect() +} + +/// What a gloas block body carries beyond what this module derives from the +/// state. +#[derive(Debug, Clone)] +pub struct GloasBlockInputs { + pub randao_reveal: BlsSignature, + pub graffiti: Bytes32, + pub attestations: Vec, + pub payload_attestations: Vec, + /// The parent envelope's requests when building on its full payload and + /// the parent is gloas; empty otherwise. + pub parent_execution_requests: ExecutionRequests, + /// The payload `engine_getPayloadV6` built, which the envelope reveals. + pub execution_payload: ExecutionPayload, + /// The blobs bundle's commitments, which the bid commits to. + pub blob_kzg_commitments: Vec, + /// The payload's own requests, which the bid commits to by root and the + /// envelope carries. + pub execution_requests: ExecutionRequests, +} + +/// A produced block and the unsigned envelope that reveals its payload. +#[derive(Debug, Clone)] +pub struct GloasProduced { + pub block: BeaconBlock, + pub envelope: ExecutionPayloadEnvelope, +} + +/// The unsigned self-built block for the slot `state` has been advanced to, with +/// its state root computed, and the unsigned envelope for its payload. +/// +/// The body votes the state's own `eth1_data`, carries no slashings, exits or +/// credential changes (this node pools none) and an empty sync aggregate. The +/// block is run through gloas's `process_block` on a copy of `state`, which +/// is also what rejects a body the network would, before anything is signed. +/// The envelope is then checked against that post-state with every condition +/// `verify_execution_payload_envelope` holds it to except the signature, so a +/// malformed build fails here and not on the network. +pub fn assemble_gloas_block( + state: &BeaconState, + inputs: GloasBlockInputs, + config: &Config, +) -> Result { + verify( + matches!(state, BeaconState::Gloas(_)), + "gloas block production runs on a gloas state", + )?; + let payload = &inputs.execution_payload; + let bid = ExecutionPayloadBid { + parent_block_hash: payload.parent_hash, + parent_block_root: state.latest_block_header().hash_tree_root(), + block_hash: payload.block_hash, + prev_randao: payload.prev_randao, + fee_recipient: payload.fee_recipient, + gas_limit: payload.gas_limit, + builder_index: constants::BUILDER_INDEX_SELF_BUILD, + slot: state.slot(), + value: 0, + execution_payment: 0, + blob_kzg_commitments: inputs.blob_kzg_commitments.into(), + execution_requests_root: inputs.execution_requests.hash_tree_root(), + }; + let body = BeaconBlockBody { + randao_reveal: inputs.randao_reveal, + eth1_data: state.eth1_data().clone(), + graffiti: inputs.graffiti, + attestations: inputs.attestations.into(), + sync_aggregate: empty_sync_aggregate(), + signed_execution_payload_bid: SignedExecutionPayloadBid { + message: bid, + signature: BlsSignature(bls::G2_POINT_AT_INFINITY), + }, + payload_attestations: inputs.payload_attestations.into(), + parent_execution_requests: inputs.parent_execution_requests, + ..BeaconBlockBody::empty() + }; + let mut block = BeaconBlock { + slot: state.slot(), + proposer_index: get_beacon_proposer_index(state)?, + // `process_slots` filled the header's state root on the way out of + // the parent's slot, so this is the parent block's root. + parent_root: state.latest_block_header().hash_tree_root(), + state_root: Root::ZERO, + body, + }; + + let mut post = state.clone(); + let BeaconState::Gloas(_) = &post else { + unreachable!("checked above") + }; + stf::gloas::process_block(&mut post, &block, config, &CommitteeCache::default())?; + block.state_root = post.hash_tree_root(); + + let envelope = ExecutionPayloadEnvelope { + payload: inputs.execution_payload, + execution_requests: inputs.execution_requests, + builder_index: constants::BUILDER_INDEX_SELF_BUILD, + beacon_block_root: block.hash_tree_root(), + parent_beacon_block_root: block.parent_root, + }; + check_envelope(&post, &envelope, config)?; + Ok(GloasProduced { block, envelope }) +} + +/// `verify_execution_payload_envelope` (gloas `fork-choice.md`) without its +/// signature check and its engine call, against `post`, the state after the +/// block the envelope belongs to. +/// +/// Mirrors [`stf::gloas::verify_execution_payload_envelope`] condition for +/// condition rather than calling it, because that function verifies the +/// signature first and the producer has none to give yet: the proposer signs +/// the envelope after this node hands it over. +fn check_envelope( + post: &BeaconState, + envelope: &ExecutionPayloadEnvelope, + config: &Config, +) -> Result<()> { + let payload = &envelope.payload; + let mut header = post.latest_block_header().clone(); + header.state_root = post.compute_state_root(); + verify( + envelope.beacon_block_root == header.hash_tree_root(), + "envelope.beacon_block_root == hash_tree_root(header)", + )?; + verify( + envelope.parent_beacon_block_root == post.latest_block_header().parent_root, + "envelope.parent_beacon_block_root == state.latest_block_header.parent_root", + )?; + let inner = gloas_state_ref(post, "check_envelope")?; + let bid = &inner.latest_execution_payload_bid; + verify( + envelope.builder_index == bid.builder_index, + "envelope.builder_index == bid.builder_index", + )?; + verify( + payload.prev_randao == bid.prev_randao, + "payload.prev_randao == bid.prev_randao", + )?; + verify( + payload.gas_limit == bid.gas_limit, + "payload.gas_limit == bid.gas_limit", + )?; + verify( + payload.block_hash == bid.block_hash, + "payload.block_hash == bid.block_hash", + )?; + verify( + envelope.execution_requests.hash_tree_root() == bid.execution_requests_root, + "hash_tree_root(envelope.execution_requests) == bid.execution_requests_root", + )?; + verify( + payload.slot_number == post.slot(), + "payload.slot_number == state.slot", + )?; + verify( + payload.parent_hash == inner.latest_block_hash, + "payload.parent_hash == state.latest_block_hash", + )?; + verify( + payload.timestamp == stf::bellatrix::compute_timestamp_at_slot(post, post.slot(), config), + "payload.timestamp == compute_time_at_slot(state, state.slot)", + )?; + verify( + payload.withdrawals.hash_tree_root() == inner.payload_expected_withdrawals.hash_tree_root(), + "hash_tree_root(payload.withdrawals) == hash_tree_root(state.payload_expected_withdrawals)", + )?; + Ok(()) +} + +/// Gloas test fixtures shared by this module's tests and by the Beacon API's +/// (`test-utils` feature): a gloas state a block can be built on, and the pieces +/// an execution client would hand over. +#[cfg(any(test, feature = "test-utils"))] +pub mod test_support { + use super::*; + use crate::beacon::ForkName; + use crate::beacon::block_production::advance_to_slot; + use crate::beacon::helpers::accessors::get_domain; + use crate::beacon::helpers::fulu::initialize_proposer_lookahead; + use crate::beacon::helpers::gloas::compute_ptc; + use crate::beacon::helpers::misc::compute_signing_root; + use crate::beacon::helpers::test_state::{sign_for, with_signing_validators_at}; + use ethlambda_types::beacon::containers::bellatrix::{ExtraData, LogsBloom}; + use ethlambda_types::beacon::primitives::Uint256; + + pub const PARENT_BLOCK_HASH: u8 = 0x11; + pub const GRANDPARENT_BLOCK_HASH: u8 = 0x10; + + /// The schedule these fixtures assume: gloas from epoch 0. + pub fn config() -> Config { + Config::mainnet().with_fork_epoch(ForkName::Gloas, 0) + } + + /// A gloas state at slot 32, the post-state of a parent block whose + /// payload was revealed, its PTC for slot 32 filled from the real registry, + /// with the lookahead and sync committee a real registry would give it. + pub fn parent_state() -> BeaconState { + let mut state = with_signing_validators_at(ForkName::Gloas, 64); + let lookahead = initialize_proposer_lookahead(&state).unwrap(); + let sync_committee = + crate::beacon::helpers::altair::get_next_sync_committee(&state).unwrap(); + let ptc = compute_ptc(&state, state.slot(), &CommitteeCache::default()).unwrap(); + let BeaconState::Gloas(inner) = &mut state else { + unreachable!("built as gloas") + }; + inner.proposer_lookahead = lookahead.try_into().unwrap(); + inner.current_sync_committee = sync_committee.clone(); + inner.next_sync_committee = sync_committee; + inner.latest_block_header.slot = inner.slot; + inner.latest_block_hash = ExecutionBlockHash::repeat_byte(GRANDPARENT_BLOCK_HASH); + inner.latest_execution_payload_bid = ExecutionPayloadBid { + parent_block_hash: ExecutionBlockHash::repeat_byte(GRANDPARENT_BLOCK_HASH), + block_hash: ExecutionBlockHash::repeat_byte(PARENT_BLOCK_HASH), + gas_limit: 30_000_000, + slot: inner.slot, + builder_index: constants::BUILDER_INDEX_SELF_BUILD, + execution_requests_root: ExecutionRequests::default().hash_tree_root(), + ..Default::default() + }; + // The window entry `get_ptc` reads for a slot of the state's own epoch. + let window_index = + (preset::SLOTS_PER_EPOCH + inner.slot % preset::SLOTS_PER_EPOCH) as usize; + inner.ptc_window[window_index] = ptc; + state + } + + /// [`parent_state`] advanced to slot 33, the state a block is built on. + pub fn state_to_build_on() -> BeaconState { + let state = parent_state(); + let slot = state.slot() + 1; + advance_to_slot(&state, slot, &config()).unwrap() + } + + /// The payload an execution client would build for `inputs`. + pub fn payload_for(inputs: &GloasPayloadInputs) -> ExecutionPayload { + ExecutionPayload { + parent_hash: inputs.head_block_hash, + fee_recipient: Default::default(), + state_root: Bytes32::repeat_byte(1), + receipts_root: Bytes32::repeat_byte(2), + logs_bloom: LogsBloom::try_from(vec![0u8; preset::BYTES_PER_LOGS_BLOOM]).unwrap(), + prev_randao: inputs.prev_randao, + block_number: 7, + gas_limit: inputs.target_gas_limit, + gas_used: 0, + timestamp: inputs.timestamp, + extra_data: ExtraData::default(), + base_fee_per_gas: Uint256::from_u128(7), + block_hash: ExecutionBlockHash::repeat_byte(0x22), + transactions: Default::default(), + withdrawals: inputs.withdrawals.clone().into(), + blob_gas_used: 0, + excess_blob_gas: 0, + block_access_list: Default::default(), + slot_number: inputs.slot_number, + } + } + + /// The proposer's real RANDAO reveal for `state`'s slot. + pub fn randao_reveal(state: &BeaconState) -> BlsSignature { + let proposer = get_beacon_proposer_index(state).unwrap(); + let epoch = get_current_epoch(state); + let domain = get_domain(state, constants::DOMAIN_RANDAO, Some(epoch)); + sign_for( + proposer as usize, + compute_signing_root(epoch.hash_tree_root(), domain), + ) + } + + /// A block and envelope for `state` carrying one blob commitment. + pub fn produce( + state: &BeaconState, + build_on_full: bool, + payload_attestations: Vec, + ) -> Result { + let requests = ExecutionRequests::default(); + let inputs = gloas_payload_inputs(state, build_on_full, &requests, &config())?; + assemble_gloas_block( + state, + GloasBlockInputs { + randao_reveal: randao_reveal(state), + graffiti: Bytes32::repeat_byte(7), + attestations: Vec::new(), + payload_attestations, + parent_execution_requests: requests, + execution_payload: payload_for(&inputs), + blob_kzg_commitments: vec![KzgCommitment([5; 48])], + execution_requests: ExecutionRequests::default(), + }, + &config(), + ) + } + + /// The state after `block` is applied to `state`, as the chain stores it. + pub fn post_state(state: &BeaconState, block: &BeaconBlock) -> BeaconState { + let mut post = state.clone(); + stf::gloas::process_block(&mut post, block, &config(), &CommitteeCache::default()).unwrap(); + post + } + + /// The proposer's signature over `block`, as a validator client gives it. + pub fn sign_block(state: &BeaconState, block: &BeaconBlock) -> BlsSignature { + let domain = get_domain(state, constants::DOMAIN_BEACON_PROPOSER, None); + sign_for( + block.proposer_index as usize, + compute_signing_root(block.hash_tree_root(), domain), + ) + } + + /// The proposer's signature over `envelope` under `DOMAIN_BEACON_BUILDER`, + /// against the block's post-state. + pub fn sign_envelope( + post: &BeaconState, + proposer: ValidatorIndex, + envelope: &ExecutionPayloadEnvelope, + ) -> BlsSignature { + let domain = get_domain(post, constants::DOMAIN_BEACON_BUILDER, None); + sign_for( + proposer as usize, + compute_signing_root(envelope.hash_tree_root(), domain), + ) + } +} + +#[cfg(test)] +mod gloas_block_production_tests { + use super::super::stf::ExecutionEngine; + use super::test_support::*; + use super::*; + use crate::beacon::helpers::accessors::get_domain; + use crate::beacon::helpers::misc::compute_signing_root; + use crate::beacon::helpers::test_state::sign_for; + + /// A vote on the parent block (slot 32) signed by `validator`. + fn ptc_message( + state: &BeaconState, + validator: ValidatorIndex, + data: PayloadAttestationData, + ) -> PayloadAttestationMessage { + let domain = get_domain( + state, + constants::DOMAIN_PTC_ATTESTER, + Some(ethlambda_types::beacon::signing::compute_epoch_at_slot( + data.slot, + )), + ); + PayloadAttestationMessage { + validator_index: validator, + data, + signature: sign_for( + validator as usize, + compute_signing_root(data.hash_tree_root(), domain), + ), + } + } + + fn parent_vote(state: &BeaconState, present: bool) -> PayloadAttestationData { + PayloadAttestationData { + beacon_block_root: state.latest_block_header().hash_tree_root(), + slot: state.slot() - 1, + payload_present: present, + blob_data_available: true, + } + } + + #[test] + fn a_self_built_block_passes_the_stf_with_a_real_signature_and_its_envelope_verifies() { + let state = state_to_build_on(); + let produced = produce(&state, true, Vec::new()).unwrap(); + let GloasProduced { block, envelope } = &produced; + + // Signed by the proposer, as a validator client would. + let signed = gloas::SignedBeaconBlock { + message: block.clone(), + signature: { + let domain = get_domain(&state, constants::DOMAIN_BEACON_PROPOSER, None); + sign_for( + block.proposer_index as usize, + compute_signing_root(block.hash_tree_root(), domain), + ) + }, + }; + let wrapped = ethlambda_types::beacon::containers::SignedBeaconBlock::Gloas(signed); + assert!(stf::verify_block_signature(&state, &wrapped)); + + // Applying it again, the way a peer importing it would, lands on the + // state root it claims. + let mut post = state.clone(); + stf::block::process_block( + &mut post, + &wrapped, + &config(), + &ExecutionEngine::valid(), + &CommitteeCache::default(), + ) + .unwrap(); + assert_eq!(block.state_root, post.hash_tree_root()); + + // The envelope, signed by the same proposer under DOMAIN_BEACON_BUILDER, + // passes the full import-side check. + let domain = get_domain(&post, constants::DOMAIN_BEACON_BUILDER, None); + let signed_envelope = gloas::SignedExecutionPayloadEnvelope { + message: envelope.clone(), + signature: sign_for( + block.proposer_index as usize, + compute_signing_root(envelope.hash_tree_root(), domain), + ), + }; + stf::gloas::verify_execution_payload_envelope( + &post, + &signed_envelope, + &config(), + &ExecutionEngine::valid(), + ) + .unwrap(); + assert_eq!(envelope.beacon_block_root, block.hash_tree_root()); + assert_eq!(envelope.parent_beacon_block_root, block.parent_root); + } + + #[test] + fn the_bid_is_a_zero_value_self_build_over_the_payload() { + let state = state_to_build_on(); + let produced = produce(&state, true, Vec::new()).unwrap(); + let signed_bid = &produced.block.body.signed_execution_payload_bid; + let bid = &signed_bid.message; + let payload = &produced.envelope.payload; + assert_eq!(bid.builder_index, constants::BUILDER_INDEX_SELF_BUILD); + assert_eq!((bid.value, bid.execution_payment), (0, 0)); + assert_eq!( + signed_bid.signature, + BlsSignature(bls::G2_POINT_AT_INFINITY) + ); + assert_eq!(bid.slot, state.slot()); + assert_eq!(bid.parent_block_hash, payload.parent_hash); + assert_eq!(bid.block_hash, payload.block_hash); + assert_eq!(bid.prev_randao, payload.prev_randao); + assert_eq!(bid.gas_limit, payload.gas_limit); + assert_eq!( + bid.parent_block_root, + state.latest_block_header().hash_tree_root() + ); + assert_eq!(bid.blob_kzg_commitments.len(), 1); + assert_eq!( + bid.execution_requests_root, + produced.envelope.execution_requests.hash_tree_root() + ); + } + + #[test] + fn building_on_full_extends_the_parent_payload_and_on_empty_skips_it() { + let state = state_to_build_on(); + let requests = ExecutionRequests::default(); + let full = gloas_payload_inputs(&state, true, &requests, &config()).unwrap(); + let empty = gloas_payload_inputs(&state, false, &requests, &config()).unwrap(); + assert_eq!( + full.head_block_hash, + ExecutionBlockHash::repeat_byte(PARENT_BLOCK_HASH) + ); + assert_eq!( + empty.head_block_hash, + ExecutionBlockHash::repeat_byte(GRANDPARENT_BLOCK_HASH) + ); + assert_eq!(full.slot_number, state.slot()); + assert_eq!(full.target_gas_limit, 30_000_000); + assert_eq!( + full.parent_beacon_block_root, + state.latest_block_header().hash_tree_root() + ); + + // Both are valid blocks, and the empty one's bid names the older hash. + let on_full = produce(&state, true, Vec::new()).unwrap(); + let on_empty = produce(&state, false, Vec::new()).unwrap(); + let parent_hash = |produced: &GloasProduced| { + produced + .block + .body + .signed_execution_payload_bid + .message + .parent_block_hash + }; + assert_eq!(parent_hash(&on_full), full.head_block_hash); + assert_eq!(parent_hash(&on_empty), empty.head_block_hash); + } + + #[test] + fn a_payload_on_the_wrong_parent_is_refused_before_signing() { + let state = state_to_build_on(); + let requests = ExecutionRequests::default(); + let inputs = gloas_payload_inputs(&state, true, &requests, &config()).unwrap(); + let mut payload = payload_for(&inputs); + payload.parent_hash = ExecutionBlockHash::repeat_byte(9); + let result = assemble_gloas_block( + &state, + GloasBlockInputs { + randao_reveal: randao_reveal(&state), + graffiti: Bytes32::ZERO, + attestations: Vec::new(), + payload_attestations: Vec::new(), + parent_execution_requests: requests, + execution_payload: payload, + blob_kzg_commitments: Vec::new(), + execution_requests: ExecutionRequests::default(), + }, + &config(), + ); + assert!(result.is_err()); + } + + #[test] + fn a_withdrawal_set_the_state_does_not_expect_is_refused() { + let state = state_to_build_on(); + let requests = ExecutionRequests::default(); + let inputs = gloas_payload_inputs(&state, true, &requests, &config()).unwrap(); + let mut payload = payload_for(&inputs); + payload.withdrawals = vec![Withdrawal { + index: 99, + validator_index: 1, + address: Default::default(), + amount: 1, + }] + .into(); + let produced = assemble_gloas_block( + &state, + GloasBlockInputs { + randao_reveal: randao_reveal(&state), + graffiti: Bytes32::ZERO, + attestations: Vec::new(), + payload_attestations: Vec::new(), + parent_execution_requests: requests, + execution_payload: payload, + blob_kzg_commitments: Vec::new(), + execution_requests: ExecutionRequests::default(), + }, + &config(), + ); + assert!(produced.is_err()); + } + + #[test] + fn payload_attestations_set_every_seat_and_aggregate_every_signature() { + let state = state_to_build_on(); + let ptc = get_ptc(&state, state.slot() - 1, &config()).unwrap(); + let first = ptc[0]; + let second = *ptc.iter().find(|validator| **validator != first).unwrap(); + let data = parent_vote(&state, true); + let messages = vec![ + ptc_message(&state, first, data), + ptc_message(&state, second, data), + // Another view of the payload: its own aggregate. + ptc_message(&state, first, parent_vote(&state, false)), + ]; + let parent_root = state.latest_block_header().hash_tree_root(); + let packed = + pack_payload_attestations(&state, parent_root, state.slot() - 1, messages, &config()); + + assert_eq!(packed.len(), 2); + let aggregate = &packed[0]; + assert_eq!(aggregate.data, data); + let seats = |validator: ValidatorIndex| ptc.iter().filter(|v| **v == validator).count(); + let expected_bits: Vec = ptc + .iter() + .enumerate() + .filter(|(_, validator)| **validator == first || **validator == second) + .map(|(position, _)| position) + .collect(); + let set: Vec = (0..preset::PTC_SIZE) + .filter(|&position| aggregate.aggregation_bits.get(position).unwrap_or(false)) + .collect(); + assert_eq!(set, expected_bits); + assert_eq!(set.len(), seats(first) + seats(second)); + + // It rides in a block that passes the full STF. + let produced = produce(&state, true, packed).unwrap(); + assert_eq!(produced.block.body.payload_attestations.len(), 2); + } + + #[test] + fn payload_attestations_for_another_parent_or_a_distant_slot_are_left_out() { + let state = state_to_build_on(); + let ptc = get_ptc(&state, state.slot() - 1, &config()).unwrap(); + let data = parent_vote(&state, true); + let message = ptc_message(&state, ptc[0], data); + let parent_root = state.latest_block_header().hash_tree_root(); + + // Not the parent block. + assert!( + pack_payload_attestations( + &state, + Root::repeat_byte(9), + state.slot() - 1, + vec![message.clone()], + &config() + ) + .is_empty() + ); + // The parent is two slots back: nothing to attest to. + assert!( + pack_payload_attestations( + &state, + parent_root, + state.slot() - 2, + vec![message], + &config() + ) + .is_empty() + ); + // A forged signature does not verify, so the aggregate is dropped. + let mut forged = ptc_message(&state, ptc[0], data); + forged.signature = ptc_message(&state, ptc[0], parent_vote(&state, false)).signature; + assert!( + pack_payload_attestations( + &state, + parent_root, + state.slot() - 1, + vec![forged], + &config() + ) + .is_empty() + ); + } + + /// A single-committee vote at slot 32 signed by committee 0's first member. + fn vote(state: &BeaconState, beacon_block_root: Root, index: u64) -> electra::Attestation { + use crate::beacon::helpers::accessors::{get_beacon_committee, get_block_root}; + let epoch = get_current_epoch(state); + let data = ethlambda_types::beacon::containers::shared::AttestationData { + slot: state.slot() - 1, + index, + beacon_block_root, + source: state.current_justified_checkpoint(), + target: ethlambda_types::beacon::containers::shared::Checkpoint { + epoch, + root: get_block_root(state, epoch).unwrap(), + }, + }; + let members = get_beacon_committee(state, data.slot, 0).unwrap(); + let domain = get_domain(state, constants::DOMAIN_BEACON_ATTESTER, Some(epoch)); + let mut aggregation_bits = electra::AggregationBits::with_length(members.len()).unwrap(); + aggregation_bits.set(0, true).unwrap(); + let mut committee_bits = electra::CommitteeBits::default(); + committee_bits.set(0, true).unwrap(); + electra::Attestation { + aggregation_bits, + data, + signature: sign_for( + members[0] as usize, + compute_signing_root(data.hash_tree_root(), domain), + ), + committee_bits, + } + } + + #[test] + fn attestations_honor_the_gloas_payload_index_rules() { + let state = state_to_build_on(); + let parent_root = state.latest_block_header().hash_tree_root(); + // A vote for an older block may name either payload status. + let elsewhere = Root::repeat_byte(1); + assert_eq!( + pack_gloas_attestations(&state, vec![vote(&state, elsewhere, 1)]).len(), + 1 + ); + // A vote for the block of its own slot must say 0, and none says 2. + assert!(pack_gloas_attestations(&state, vec![vote(&state, parent_root, 1)]).is_empty()); + assert_eq!( + pack_gloas_attestations(&state, vec![vote(&state, parent_root, 0)]).len(), + 1 + ); + assert!(pack_gloas_attestations(&state, vec![vote(&state, elsewhere, 2)]).is_empty()); + + // The packed vote rides in a block that passes the full STF. + let packed = pack_gloas_attestations(&state, vec![vote(&state, parent_root, 0)]); + let requests = ExecutionRequests::default(); + let inputs = gloas_payload_inputs(&state, true, &requests, &config()).unwrap(); + assemble_gloas_block( + &state, + GloasBlockInputs { + randao_reveal: randao_reveal(&state), + graffiti: Bytes32::ZERO, + attestations: packed, + payload_attestations: Vec::new(), + parent_execution_requests: requests, + execution_payload: payload_for(&inputs), + blob_kzg_commitments: Vec::new(), + execution_requests: ExecutionRequests::default(), + }, + &config(), + ) + .unwrap(); + } + + #[test] + fn column_sidecars_carry_every_cell_and_verify_against_the_commitments() { + // A blob of small field elements, valid and cheap to commit to. + let mut blob = vec![0u8; preset::BYTES_PER_BLOB]; + for (i, element) in blob.chunks_mut(32).enumerate() { + element[31] = (i % 200) as u8; + } + let commitment = crate::beacon::kzg::blob_to_kzg_commitment(&blob).unwrap(); + let (_, proofs) = crate::beacon::kzg::compute_cells_and_kzg_proofs(&blob).unwrap(); + let root = Root::repeat_byte(3); + let sidecars = gloas_data_column_sidecars(root, 33, &[blob], &proofs[..]).unwrap(); + + assert_eq!(sidecars.len(), preset::NUMBER_OF_COLUMNS); + for (index, sidecar) in sidecars.iter().enumerate() { + assert_eq!((sidecar.index, sidecar.slot), (index as u64, 33)); + assert_eq!(sidecar.beacon_block_root, root); + assert!( + crate::beacon::fork_choice::gloas_verify_data_column_sidecar( + sidecar, + &[commitment] + ) + ); + } + for index in [0, 77, preset::NUMBER_OF_COLUMNS - 1] { + assert!( + crate::beacon::fork_choice::gloas_verify_data_column_sidecar_kzg_proofs( + &sidecars[index], + &[commitment] + ) + .unwrap() + ); + } + // A proof count that does not fit the blobs is refused. + assert!( + gloas_data_column_sidecars(root, 33, &[vec![0; preset::BYTES_PER_BLOB]], &[]).is_err() + ); + } + + #[test] + fn a_gloas_request_list_round_trips_and_a_malformed_one_is_refused() { + let requests = ExecutionRequests { + builder_exits: vec![gloas::BuilderExitRequest::default()].into(), + ..Default::default() + }; + let list = stf::gloas::get_execution_requests_list(&requests); + assert_eq!(parse_gloas_execution_requests(&list).unwrap(), requests); + assert_eq!( + parse_gloas_execution_requests(&[]).unwrap(), + ExecutionRequests::default() + ); + let exit = vec![constants::BUILDER_EXIT_REQUEST_TYPE, 1]; + let deposit = vec![constants::DEPOSIT_REQUEST_TYPE, 1]; + assert!(parse_gloas_execution_requests(&[exit, deposit]).is_err()); + assert!( + parse_gloas_execution_requests(&[vec![constants::BUILDER_DEPOSIT_REQUEST_TYPE]]) + .is_err() + ); + assert!(parse_gloas_execution_requests(&[vec![0x7f, 0]]).is_err()); + } +} diff --git a/crates/blockchain/state_transition/src/beacon/gossip/mod.rs b/crates/blockchain/state_transition/src/beacon/gossip/mod.rs index 945255d5..187f612b 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/mod.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/mod.rs @@ -497,7 +497,7 @@ pub(crate) fn is_gloas_slot(config: &Config, slot: Slot) -> bool { /// /// The specification lets a vote on an unseen envelope be queued and asks the /// node to request it by root; neither is done here, so it is an `IGNORE`. -pub(crate) fn verify_attestation_payload_status( +pub fn verify_attestation_payload_status( store: &Store, data: &AttestationData, ) -> Result<(), Outcome> { diff --git a/crates/blockchain/state_transition/src/beacon/helpers/gloas.rs b/crates/blockchain/state_transition/src/beacon/helpers/gloas.rs index a8f97f9d..23f7fd84 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/gloas.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/gloas.rs @@ -715,6 +715,32 @@ pub fn get_ptc( }) } +/// The slot of `epoch` in which each of `validators` sits in the payload +/// timeliness committee, from `get_ptc_assignment` (gloas `validator.md`). +/// +/// The specification answers one validator at a time, each call walking the +/// epoch's slots; this walks them once for the whole set. A validator with +/// several seats gets its first slot, as `get_ptc_assignment` returns, and one +/// in no committee of the epoch is absent. `epoch` must be one [`get_ptc`] +/// can answer from `state`. +pub fn get_ptc_assignments( + state: &BeaconState, + epoch: Epoch, + validators: &std::collections::HashSet, + config: &Config, +) -> Result> { + let mut assignments = std::collections::HashMap::new(); + let first_slot = compute_start_slot_at_epoch(epoch); + for slot in first_slot..first_slot + preset::SLOTS_PER_EPOCH { + for validator_index in get_ptc(state, slot, config)? { + if validators.contains(&validator_index) { + assignments.entry(validator_index).or_insert(slot); + } + } + } + Ok(assignments) +} + /// The same payload attestation with its attesters named rather than /// bit-encoded, the payload-attestation counterpart of /// [`super::electra::get_indexed_attestation`]. diff --git a/crates/blockchain/state_transition/src/beacon/mod.rs b/crates/blockchain/state_transition/src/beacon/mod.rs index fc9e4145..1a4471fa 100644 --- a/crates/blockchain/state_transition/src/beacon/mod.rs +++ b/crates/blockchain/state_transition/src/beacon/mod.rs @@ -75,10 +75,12 @@ pub mod bls; pub mod das; pub mod fork_choice; pub mod genesis; +pub mod gloas_block_production; pub mod gossip; pub mod hash; pub mod helpers; pub mod kzg; +pub mod payload_attestation_pool; pub mod precheck; pub mod stf; pub mod upgrade; diff --git a/crates/blockchain/state_transition/src/beacon/payload_attestation_pool.rs b/crates/blockchain/state_transition/src/beacon/payload_attestation_pool.rs new file mode 100644 index 00000000..f7cbfc05 --- /dev/null +++ b/crates/blockchain/state_transition/src/beacon/payload_attestation_pool.rs @@ -0,0 +1,170 @@ +//! Payload timeliness committee votes, held until a proposer packs them. +//! +//! Gloas `validator.md` ("Payload attestations"): the proposer of slot `N` +//! listens to `payload_attestation_message`, keeps the votes on its parent +//! block (slot `N - 1`), and aggregates every vote sharing one +//! `PayloadAttestationData` into a single `PayloadAttestation` for its body. +//! +//! Only messages that already passed validation go in (gossip's +//! `payload_attestation_message` verdict, or the Beacon API's pool endpoint +//! after the same checks), since that is what makes combining their +//! signatures safe: one bad share fails the whole aggregate, and with it the +//! block that carries it. +//! +//! The method signatures below are the shared contract between what fills the +//! pool and block production, which only calls [`messages_for`]. +//! +//! [`messages_for`]: PayloadAttestationPool::messages_for + +use std::{ + collections::BTreeMap, + sync::{Arc, Mutex}, +}; + +use ethlambda_types::beacon::{ + containers::gloas::{PayloadAttestationData, PayloadAttestationMessage}, + primitives::{Root, Slot}, +}; + +/// The pool, shared between whatever fills it (the Beacon API's pool +/// endpoint, the `payload_attestation_message` gossip verdict) and what reads +/// it (block production, `GET /eth/v1/beacon/pool/payload_attestations`). +/// There must be exactly one per node. +pub type SharedPayloadAttestationPool = Arc>; + +/// Votes sharing one `PayloadAttestationData`, by voting validator. The data +/// has no `Hash`/`Ord`, and a slot holds only a handful of distinct values +/// (the committee splits at most a few ways), so a linear scan over a `Vec` +/// is both simple and fast. +type VotesByData = Vec<( + PayloadAttestationData, + BTreeMap, +)>; + +#[derive(Debug, Default)] +pub struct PayloadAttestationPool { + /// slot -> data -> validator index -> message. Keyed by slot first so + /// pruning and the per-slot queries never visit unrelated slots. + messages: BTreeMap, +} + +impl PayloadAttestationPool { + /// Record a validated message. Returns `false` (and keeps the first) when + /// the pool already holds a message from the same validator for the same + /// `data`. Drops every message for a slot more than one before the + /// message's own. + pub fn insert(&mut self, message: PayloadAttestationMessage) -> bool { + self.prune_before(message.data.slot.saturating_sub(1)); + let by_data = self.messages.entry(message.data.slot).or_default(); + let votes = match by_data.iter().position(|(data, _)| *data == message.data) { + Some(position) => &mut by_data[position].1, + None => { + by_data.push((message.data, BTreeMap::new())); + &mut by_data.last_mut().expect("just pushed").1 + } + }; + if votes.contains_key(&message.validator_index) { + return false; + } + votes.insert(message.validator_index, message); + true + } + + /// Every held message voting on `beacon_block_root` at `slot`, in any + /// order, one per (validator, data). + pub fn messages_for( + &self, + slot: Slot, + beacon_block_root: Root, + ) -> Vec { + self.messages + .get(&slot) + .into_iter() + .flatten() + .filter(|(data, _)| data.beacon_block_root == beacon_block_root) + .flat_map(|(_, votes)| votes.values().cloned()) + .collect() + } + + /// Every held message, or only those for `slot` when given. + pub fn all(&self, slot: Option) -> Vec { + self.messages + .iter() + .filter(|(held, _)| slot.is_none_or(|slot| **held == slot)) + .flat_map(|(_, by_data)| by_data.iter()) + .flat_map(|(_, votes)| votes.values().cloned()) + .collect() + } + + /// Drop every message for a slot before `slot`. + pub fn prune_before(&mut self, slot: Slot) { + self.messages = self.messages.split_off(&slot); + } +} + +#[cfg(test)] +mod tests { + use super::*; + use ethlambda_types::beacon::primitives::BlsSignature; + + fn message( + validator_index: u64, + slot: Slot, + root: u8, + present: bool, + ) -> PayloadAttestationMessage { + PayloadAttestationMessage { + validator_index, + data: PayloadAttestationData { + beacon_block_root: Root::from([root; 32]), + slot, + payload_present: present, + blob_data_available: true, + }, + signature: BlsSignature::default(), + } + } + + #[test] + fn insert_dedups_per_validator_and_data() { + let mut pool = PayloadAttestationPool::default(); + assert!(pool.insert(message(1, 10, 1, true))); + assert!(!pool.insert(message(1, 10, 1, true))); + // Same validator, different data: an equivocation, kept as its own entry. + assert!(pool.insert(message(1, 10, 1, false))); + assert!(pool.insert(message(2, 10, 1, true))); + assert_eq!(pool.all(Some(10)).len(), 3); + } + + #[test] + fn messages_for_filters_by_slot_and_root() { + let mut pool = PayloadAttestationPool::default(); + pool.insert(message(1, 10, 1, true)); + pool.insert(message(2, 10, 2, true)); + pool.insert(message(3, 11, 1, true)); + let held = pool.messages_for(10, Root::from([1; 32])); + assert_eq!(held.len(), 1); + assert_eq!(held[0].validator_index, 1); + assert!(pool.messages_for(12, Root::from([1; 32])).is_empty()); + } + + #[test] + fn insert_prunes_to_current_and_previous_slot() { + let mut pool = PayloadAttestationPool::default(); + pool.insert(message(1, 8, 1, true)); + pool.insert(message(1, 9, 1, true)); + pool.insert(message(1, 10, 1, true)); + let slots: Vec = pool.all(None).iter().map(|m| m.data.slot).collect(); + assert_eq!(slots, vec![9, 10]); + } + + #[test] + fn prune_before_drops_older_slots() { + let mut pool = PayloadAttestationPool::default(); + pool.insert(message(1, 5, 1, true)); + pool.insert(message(1, 6, 1, true)); + pool.prune_before(6); + assert_eq!(pool.all(None).len(), 1); + assert!(pool.all(Some(5)).is_empty()); + } +} diff --git a/crates/common/types/src/beacon/containers/gloas.rs b/crates/common/types/src/beacon/containers/gloas.rs index 8c6731e0..51d236b7 100644 --- a/crates/common/types/src/beacon/containers/gloas.rs +++ b/crates/common/types/src/beacon/containers/gloas.rs @@ -259,7 +259,17 @@ pub struct BuilderExitRequest { /// What the payload timeliness committee attests to: whether the slot's /// payload was revealed on time and whether its blob data is available. #[derive( - Debug, Clone, Copy, Default, PartialEq, Eq, serde::Serialize, SszEncode, SszDecode, HashTreeRoot, + Debug, + Clone, + Copy, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, )] pub struct PayloadAttestationData { pub beacon_block_root: Root, @@ -281,7 +291,17 @@ pub struct PayloadAttestation { /// One payload timeliness committee member's unaggregated vote, gossiped /// before an aggregator folds it into a [`PayloadAttestation`]. -#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, SszEncode, SszDecode, HashTreeRoot)] +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] pub struct PayloadAttestationMessage { #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] pub validator_index: ValidatorIndex, @@ -464,17 +484,74 @@ pub struct NewPayloadRequest { /// Electra's shape (EIP-7549's wide `committee_bits`/`aggregation_bits`), /// with `aggregation_bits` now the unbounded [`AggregationBits`] rather than /// electra's bounded `SszBitlist`. -#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, SszEncode, SszDecode, HashTreeRoot)] +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] #[ssz(progressive_container)] pub struct Attestation { - #[serde(serialize_with = "crate::beacon::serde_helpers::ssz_hex::serialize")] + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] pub aggregation_bits: AggregationBits, pub data: AttestationData, pub signature: BlsSignature, - #[serde(serialize_with = "crate::beacon::serde_helpers::ssz_hex::serialize")] + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] pub committee_bits: CommitteeBits, } +impl From<&super::electra::Attestation> for Attestation { + /// The same vote in gloas's container: every field carries over unchanged, + /// only `aggregation_bits` moves from electra's bounded bitlist to the + /// progressive one. A pool that holds electra-shaped aggregates (the + /// shape gossip and the Beacon API agree on until the fork) serves gloas + /// readers through this. + fn from(attestation: &super::electra::Attestation) -> Self { + let source = &attestation.aggregation_bits; + let mut aggregation_bits = AggregationBits::with_length(source.len()); + for index in (0..source.len()).filter(|&index| source.get(index) == Some(true)) { + aggregation_bits + .set(index, true) + .expect("the index is below the length the bitlist was built with"); + } + Self { + aggregation_bits, + data: attestation.data, + signature: attestation.signature, + committee_bits: attestation.committee_bits.clone(), + } + } +} + +impl TryFrom<&Attestation> for super::electra::Attestation { + type Error = libssz_types::TypeError; + + /// The inverse of the conversion above, for a gloas aggregate entering a + /// pool that stores electra's shape. Fails only when the bits exceed + /// electra's `MAX_VALIDATORS_PER_SLOT` bound, which no aggregate a real + /// committee assignment produces can reach. + fn try_from(attestation: &Attestation) -> Result { + let source = &attestation.aggregation_bits; + let mut aggregation_bits = super::electra::AggregationBits::with_length(source.len())?; + for index in (0..source.len()).filter(|&index| source.get(index) == Some(true)) { + aggregation_bits + .set(index, true) + .expect("the index is below the length the bitlist was built with"); + } + Ok(Self { + aggregation_bits, + data: attestation.data, + signature: attestation.signature, + committee_bits: attestation.committee_bits.clone(), + }) + } +} + /// An attestation with its attesters named rather than bit-encoded, the same /// role electra's plays, with `attesting_indices` now the unbounded /// [`AttestingIndices`]. @@ -499,7 +576,17 @@ pub struct AttesterSlashing { /// An aggregate together with proof that its aggregator was selected to /// produce it. Unchanged in shape from electra: `aggregate` simply carries /// gloas's own, wider [`Attestation`] now. -#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, SszEncode, SszDecode, HashTreeRoot)] +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] pub struct AggregateAndProof { #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] pub aggregator_index: ValidatorIndex, @@ -507,7 +594,17 @@ pub struct AggregateAndProof { pub selection_proof: BlsSignature, } -#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, SszEncode, SszDecode, HashTreeRoot)] +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] pub struct SignedAggregateAndProof { pub message: AggregateAndProof, pub signature: BlsSignature, diff --git a/crates/net/api/src/lib.rs b/crates/net/api/src/lib.rs index 705e98f8..7c3c3305 100644 --- a/crates/net/api/src/lib.rs +++ b/crates/net/api/src/lib.rs @@ -354,6 +354,25 @@ pub trait RpcToP2P: Send + Sync { /// delivers a node its own messages, so without the second half this node /// would not follow its own proposal. Checked by the caller as above. fn publish_beacon_block(&self, block: SignedBeaconBlock) -> Result<(), ActorError>; + /// Gossip a gloas execution payload envelope a validator client signed on + /// `execution_payload`, and its blobs' data column sidecars on their + /// subnets, then hand both to the chain actor: as for a block, gossip never + /// delivers a node its own messages. Checked by the caller as above. + /// + /// Boxed for the reason [`P2PToBlockChain::new_execution_payload_envelope`] + /// is: the envelope carries a whole execution payload. + fn publish_execution_payload_envelope( + &self, + envelope: Box, + sidecars: Vec, + ) -> Result<(), ActorError>; + /// Gossip a payload timeliness committee member's vote on + /// `payload_attestation_message` and hand it to the chain actor. Checked by + /// the caller as above. + fn publish_payload_attestation_message( + &self, + message: PayloadAttestationMessage, + ) -> Result<(), ActorError>; } // --- Init messages --- diff --git a/crates/net/engine/src/building.rs b/crates/net/engine/src/building.rs index e7493d9e..b407c932 100644 --- a/crates/net/engine/src/building.rs +++ b/crates/net/engine/src/building.rs @@ -11,7 +11,7 @@ //! The response is decoded straight into `ethlambda-types`' own containers, so //! the block producer never sees a wire shape. -use ethlambda_types::beacon::containers::{bellatrix, capella, deneb}; +use ethlambda_types::beacon::containers::{bellatrix, capella, deneb, gloas}; use ethlambda_types::beacon::primitives::{ Bytes32, ExecutionAddress, H160, H256, KzgCommitment, KzgProof, Root, U256, Uint256, }; @@ -60,6 +60,36 @@ impl Serialize for PayloadAttributesV3 { } } +/// `PayloadAttributesV4` (Amsterdam): `PayloadAttributesV3` plus the slot the +/// payload is built for and the gas limit the execution client should steer +/// toward. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PayloadAttributesV4 { + /// V3's five fields, serialized flat alongside the two below. + pub v3: PayloadAttributesV3, + /// The payload's `slotNumber` (EIP-7843): the beacon slot, not derived + /// from `timestamp` by the execution client. + pub slot_number: u64, + /// The `gasLimit` the payload should move toward, which the execution + /// client steps to within the protocol's per-block bound. + pub target_gas_limit: u64, +} + +impl Serialize for PayloadAttributesV4 { + fn serialize(&self, serializer: S) -> Result { + let mut value = serde_json::to_value(&self.v3).map_err(serde::ser::Error::custom)?; + let object = value + .as_object_mut() + .expect("PayloadAttributesV3 serializes as an object"); + object.insert("slotNumber".into(), quantity(self.slot_number).into()); + object.insert( + "targetGasLimit".into(), + quantity(self.target_gas_limit).into(), + ); + value.serialize(serializer) + } +} + /// The execution client's name for one build process, `DATA` of eight bytes. #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] pub struct PayloadId(pub [u8; 8]); @@ -90,6 +120,18 @@ pub struct BuiltPayload { pub execution_requests: Vec>, } +/// `engine_getPayloadV6`'s answer, decoded: [`BuiltPayload`] with Amsterdam's +/// `ExecutionPayloadV4`, which gloas's envelope carries. +#[derive(Debug, Clone)] +pub struct BuiltGloasPayload { + pub execution_payload: gloas::ExecutionPayload, + /// What the payload pays its fee recipient, in wei. + pub block_value: Uint256, + pub blobs_bundle: BlobsBundle, + /// The EIP-7685 request list, each entry its type byte then its data. + pub execution_requests: Vec>, +} + /// `BlobsBundleV2`: the payload's blobs, their commitments, and every blob's /// `CELLS_PER_EXT_BLOB` cell proofs, flattened blob by blob. #[derive(Debug, Clone, Default)] @@ -109,6 +151,17 @@ pub(crate) struct GetPayloadV5Response { execution_requests: Vec, } +/// `engine_getPayloadV6`'s answer: V5's, with `ExecutionPayloadV4`. +#[derive(Deserialize)] +#[serde(rename_all = "camelCase")] +pub(crate) struct GetPayloadV6Response { + execution_payload: ExecutionPayloadJson, + block_value: String, + blobs_bundle: BlobsBundleJson, + #[serde(default)] + execution_requests: Vec, +} + #[derive(Deserialize)] #[serde(rename_all = "camelCase")] struct ExecutionPayloadJson { @@ -129,6 +182,12 @@ struct ExecutionPayloadJson { withdrawals: Vec, blob_gas_used: String, excess_blob_gas: String, + /// `ExecutionPayloadV4` only (Amsterdam). + #[serde(default)] + block_access_list: Option, + /// `ExecutionPayloadV4` only (Amsterdam). + #[serde(default)] + slot_number: Option, } #[derive(Deserialize)] @@ -160,18 +219,7 @@ impl TryFrom for BuiltPayload { .map_err(|_| decode("a transaction exceeds MAX_BYTES_PER_TRANSACTION")) }) .collect::, _>>()?; - let withdrawals = payload - .withdrawals - .iter() - .map(|withdrawal| { - Ok(capella::Withdrawal { - index: parse_quantity(&withdrawal.index)?, - validator_index: parse_quantity(&withdrawal.validator_index)?, - address: H160(parse_fixed(&withdrawal.address)?), - amount: parse_quantity(&withdrawal.amount)?, - }) - }) - .collect::, EngineError>>()?; + let withdrawals = decode_withdrawals(&payload.withdrawals)?; let execution_payload = deneb::ExecutionPayload { parent_hash: H256(parse_fixed(&payload.parent_hash)?), @@ -199,38 +247,110 @@ impl TryFrom for BuiltPayload { excess_blob_gas: parse_quantity(&payload.excess_blob_gas)?, }; - let bundle = response.blobs_bundle; - let blobs_bundle = BlobsBundle { - commitments: bundle - .commitments - .iter() - .map(|text| parse_fixed(text).map(KzgCommitment)) - .collect::>()?, - proofs: bundle - .proofs - .iter() - .map(|text| parse_fixed(text).map(KzgProof)) - .collect::>()?, - blobs: bundle - .blobs - .iter() - .map(|text| parse_data(text)) - .collect::>()?, - }; + let blobs_bundle = decode_blobs_bundle(response.blobs_bundle)?; Ok(BuiltPayload { execution_payload, block_value: parse_uint256(&response.block_value)?, blobs_bundle, - execution_requests: response - .execution_requests - .iter() - .map(|text| parse_data(text)) - .collect::>()?, + execution_requests: decode_requests(&response.execution_requests)?, }) } } +impl TryFrom for BuiltGloasPayload { + type Error = EngineError; + + fn try_from(response: GetPayloadV6Response) -> Result { + let payload = response.execution_payload; + let transactions = payload + .transactions + .iter() + .map(|transaction| parse_data(transaction).map(gloas::Transaction::from)) + .collect::, _>>()?; + let withdrawals = decode_withdrawals(&payload.withdrawals)?; + let block_access_list = payload + .block_access_list + .as_deref() + .ok_or_else(|| decode("ExecutionPayloadV4 carries no blockAccessList"))?; + let slot_number = payload + .slot_number + .as_deref() + .ok_or_else(|| decode("ExecutionPayloadV4 carries no slotNumber"))?; + + let execution_payload = gloas::ExecutionPayload { + parent_hash: H256(parse_fixed(&payload.parent_hash)?), + fee_recipient: H160(parse_fixed(&payload.fee_recipient)?), + state_root: H256(parse_fixed(&payload.state_root)?), + receipts_root: H256(parse_fixed(&payload.receipts_root)?), + logs_bloom: bellatrix::LogsBloom::try_from(parse_data(&payload.logs_bloom)?) + .map_err(|_| decode("logsBloom is not BYTES_PER_LOGS_BLOOM long"))?, + prev_randao: H256(parse_fixed(&payload.prev_randao)?), + block_number: parse_quantity(&payload.block_number)?, + gas_limit: parse_quantity(&payload.gas_limit)?, + gas_used: parse_quantity(&payload.gas_used)?, + timestamp: parse_quantity(&payload.timestamp)?, + extra_data: bellatrix::ExtraData::try_from(parse_data(&payload.extra_data)?) + .map_err(|_| decode("extraData exceeds MAX_EXTRA_DATA_BYTES"))?, + base_fee_per_gas: parse_uint256(&payload.base_fee_per_gas)?, + block_hash: H256(parse_fixed(&payload.block_hash)?), + transactions: transactions.into(), + withdrawals: withdrawals.into(), + blob_gas_used: parse_quantity(&payload.blob_gas_used)?, + excess_blob_gas: parse_quantity(&payload.excess_blob_gas)?, + block_access_list: parse_data(block_access_list)?.into(), + slot_number: parse_quantity(slot_number)?, + }; + + Ok(BuiltGloasPayload { + execution_payload, + block_value: parse_uint256(&response.block_value)?, + blobs_bundle: decode_blobs_bundle(response.blobs_bundle)?, + execution_requests: decode_requests(&response.execution_requests)?, + }) + } +} + +fn decode_withdrawals( + withdrawals: &[WithdrawalJson], +) -> Result, EngineError> { + withdrawals + .iter() + .map(|withdrawal| { + Ok(capella::Withdrawal { + index: parse_quantity(&withdrawal.index)?, + validator_index: parse_quantity(&withdrawal.validator_index)?, + address: H160(parse_fixed(&withdrawal.address)?), + amount: parse_quantity(&withdrawal.amount)?, + }) + }) + .collect() +} + +fn decode_blobs_bundle(bundle: BlobsBundleJson) -> Result { + Ok(BlobsBundle { + commitments: bundle + .commitments + .iter() + .map(|text| parse_fixed(text).map(KzgCommitment)) + .collect::>()?, + proofs: bundle + .proofs + .iter() + .map(|text| parse_fixed(text).map(KzgProof)) + .collect::>()?, + blobs: bundle + .blobs + .iter() + .map(|text| parse_data(text)) + .collect::>()?, + }) +} + +fn decode_requests(requests: &[String]) -> Result>, EngineError> { + requests.iter().map(|text| parse_data(text)).collect() +} + fn decode(message: &str) -> EngineError { EngineError::Decode(message.to_string()) } @@ -375,4 +495,74 @@ mod tests { assert_eq!(uint256(&parse_uint256(&max).unwrap()), max); assert!(parse_uint256(&format!("0x{}", "f".repeat(65))).is_err()); } + + fn v6_json() -> serde_json::Value { + let mut json = response_json(); + json["executionPayload"]["blockAccessList"] = "0xc180".into(); + json["executionPayload"]["slotNumber"] = "0x1000".into(); + json["blobsBundle"] = serde_json::json!({ + "commitments": [format!("0x{}", "aa".repeat(48))], + "proofs": [format!("0x{}", "bb".repeat(48)), format!("0x{}", "cc".repeat(48))], + "blobs": [format!("0x{}", "01".repeat(8))], + }); + json["executionRequests"] = serde_json::json!(["0x0011", "0x02ff"]); + json + } + + #[test] + fn a_get_payload_v6_answer_decodes_into_the_gloas_containers() { + let response: GetPayloadV6Response = serde_json::from_value(v6_json()).unwrap(); + let built = BuiltGloasPayload::try_from(response).unwrap(); + let payload = &built.execution_payload; + assert_eq!(payload.slot_number, 0x1000); + assert_eq!(payload.block_access_list.as_ref(), &[0xc1, 0x80]); + assert_eq!(payload.block_hash, H256([0x55; 32])); + assert_eq!(payload.transactions.len(), 1); + assert_eq!(payload.withdrawals[0].amount, 3); + assert_eq!(uint256(&built.block_value), "0x1bc16d674ec80000"); + assert_eq!(built.blobs_bundle.commitments.len(), 1); + assert_eq!(built.blobs_bundle.proofs.len(), 2); + assert_eq!(built.blobs_bundle.blobs, vec![vec![1u8; 8]]); + assert_eq!( + built.execution_requests, + vec![vec![0x00, 0x11], vec![0x02, 0xff]] + ); + } + + #[test] + fn a_v6_answer_without_a_slot_number_is_a_decode_error() { + let mut json = v6_json(); + json["executionPayload"] + .as_object_mut() + .unwrap() + .remove("slotNumber"); + let response: GetPayloadV6Response = serde_json::from_value(json).unwrap(); + assert!(matches!( + BuiltGloasPayload::try_from(response), + Err(EngineError::Decode(_)) + )); + } + + #[test] + fn payload_attributes_v4_append_the_slot_and_target_gas_limit() { + let attributes = PayloadAttributesV4 { + v3: PayloadAttributesV3 { + timestamp: 12, + prev_randao: H256([1; 32]), + suggested_fee_recipient: H160([2; 20]), + withdrawals: vec![], + parent_beacon_block_root: Root::repeat_byte(4), + }, + slot_number: 4096, + target_gas_limit: 60_000_000, + }; + let json = serde_json::to_value(&attributes).unwrap(); + assert_eq!(json["timestamp"], "0xc"); + assert_eq!(json["slotNumber"], "0x1000"); + assert_eq!(json["targetGasLimit"], "0x3938700"); + assert_eq!( + json["parentBeaconBlockRoot"], + format!("0x{}", "04".repeat(32)) + ); + } } diff --git a/crates/net/engine/src/client.rs b/crates/net/engine/src/client.rs index 25a4db2e..fb16ba6d 100644 --- a/crates/net/engine/src/client.rs +++ b/crates/net/engine/src/client.rs @@ -311,6 +311,36 @@ impl EngineClient { response.try_into() } + /// `engine_forkchoiceUpdatedV4` with `payloadAttributes`: Amsterdam's fork + /// choice notification plus a request to build a payload for the slot the + /// attributes name, with the consensus client's custody set. + /// + /// Returns the head's status and the build process's id; the id is `None` + /// when the execution client declined to build. + pub async fn forkchoice_updated_v4_with_attributes( + &self, + state: &ForkchoiceStateV1, + attributes: &crate::building::PayloadAttributesV4, + custody_columns: Option, + ) -> Result<(PayloadStatusV1, Option), EngineError> { + let params = json!([state, attributes, custody_columns]); + let response: ForkchoiceUpdatedResponse = + self.call("engine_forkchoiceUpdatedV4", params).await?; + Ok((response.payload_status, response.payload_id)) + } + + /// `engine_getPayloadV6`: [`Self::get_payload`] for Amsterdam, answering + /// with `ExecutionPayloadV4` (`blockAccessList` and `slotNumber` included). + pub async fn get_payload_v6( + &self, + payload_id: crate::building::PayloadId, + ) -> Result { + let response: crate::building::GetPayloadV6Response = self + .call("engine_getPayloadV6", json!([payload_id])) + .await?; + response.try_into() + } + /// `engine_exchangeCapabilities`. Returns what the execution client says it /// supports. pub async fn exchange_capabilities(&self, ours: &[&str]) -> Result, EngineError> { @@ -355,7 +385,11 @@ impl EngineClient { // network that does not schedule gloas never needs it, and an // execution client that predates Amsterdam is right not to offer it. if gloas_scheduled { - for required in ["engine_newPayloadV5", "engine_forkchoiceUpdatedV4"] { + for required in [ + "engine_newPayloadV5", + "engine_forkchoiceUpdatedV4", + "engine_getPayloadV6", + ] { if !theirs.iter().any(|method| method == required) { warn!( method = required, diff --git a/crates/net/engine/src/lib.rs b/crates/net/engine/src/lib.rs index f3cf1d73..742cd0bb 100644 --- a/crates/net/engine/src/lib.rs +++ b/crates/net/engine/src/lib.rs @@ -44,5 +44,6 @@ pub const ETHLAMBDA_ENGINE_CAPABILITIES: &[&str] = &[ "engine_forkchoiceUpdatedV3", "engine_forkchoiceUpdatedV4", "engine_getPayloadV5", + "engine_getPayloadV6", "engine_getClientVersionV1", ]; diff --git a/crates/net/p2p/src/beacon/verdict.rs b/crates/net/p2p/src/beacon/verdict.rs index ebebdef9..5d820eed 100644 --- a/crates/net/p2p/src/beacon/verdict.rs +++ b/crates/net/p2p/src/beacon/verdict.rs @@ -18,7 +18,7 @@ use ethlambda_network_api::{AggregateArrival, BlockArrival, BlockSource}; use ethlambda_state_transition::beacon::gossip::{self, IgnoreReason, Outcome}; use ethlambda_state_transition::beacon::helpers::accessors::CommitteeCacheExt as _; use ethlambda_storage::{CacheKey, Store}; -use ethlambda_types::beacon::containers::electra::SingleAttestation; +use ethlambda_types::beacon::containers::electra::{self, SingleAttestation}; use ethlambda_types::beacon::containers::gloas::{ PayloadAttestationMessage, SignedExecutionPayloadEnvelope, }; @@ -199,6 +199,17 @@ impl Validated { { pool_aggregator_attestation(server, attestation); } + if let Self::PayloadAttestation(message) = &self + && outcome == Outcome::Accept + { + // Block production packs these, and a node with no chain actor + // attached still serves `GET .../pool/payload_attestations`. + server + .payload_attestation_pool + .lock() + .expect("payload attestation pool lock") + .insert(message.clone()); + } let Some(blockchain) = &server.blockchain else { return; }; @@ -268,18 +279,26 @@ impl Validated { /// block fails the whole block. Pooling on arrival also means a slot's /// aggregates, published two thirds of the way through it, are in the pool /// when the next slot's block is asked for at its start, rather than -/// waiting for the chain actor's next tick. Only electra's shape is pooled, -/// since the pool holds electra attestations and electra is the earliest fork -/// this node produces blocks for. +/// waiting for the chain actor's next tick. The pool holds electra's shape, so +/// a gloas aggregate is converted to it first. fn pool_gossip_aggregate(server: &P2PServer, aggregate: &SignedAggregateAndProof) { - let SignedAggregateAndProof::Electra(signed) = aggregate else { - return; + let pooled = match aggregate { + SignedAggregateAndProof::Electra(signed) => signed.message.aggregate.clone(), + // Gloas's aggregate converts to the pool's electra shape; the bits + // only fail to fit electra's bound for a size no committee reaches. + SignedAggregateAndProof::Gloas(signed) => { + let Ok(converted) = electra::Attestation::try_from(&signed.message.aggregate) else { + return; + }; + converted + } + SignedAggregateAndProof::Phase0(_) => return, }; server .attestation_pool .lock() .expect("attestation pool lock poisoned") - .insert_aggregate(signed.message.aggregate.clone()); + .insert_aggregate(pooled); } /// Pool an accepted subnet attestation for a validator client's aggregator. @@ -507,7 +526,9 @@ mod tests { use ethlambda_state_transition::beacon::gossip::{QueueReason, RejectReason}; use ethlambda_types::attestation::{SignedAggregatedAttestation, SignedAttestation}; use ethlambda_types::beacon::config::Config; - use ethlambda_types::beacon::containers::{AttestationData, Checkpoint, electra, phase0}; + use ethlambda_types::beacon::containers::{ + AttestationData, Checkpoint, electra, gloas, phase0, + }; use spawned_concurrency::error::ActorError; use super::*; @@ -826,6 +847,34 @@ mod tests { ); } + /// A gloas aggregate is pooled as the electra-shaped aggregate it converts + /// to, since the pool holds electra's shape. + #[tokio::test] + async fn an_accepted_gloas_aggregate_is_pooled_in_the_pools_shape() { + let server = unconnected_beacon_server(Config::mainnet(), 0).await; + let SignedAggregateAndProof::Electra(electra_signed) = electra_aggregate(5, 1) else { + unreachable!("built as electra") + }; + let expected = electra_signed.message.aggregate; + let gloas_signed = gloas::SignedAggregateAndProof { + message: gloas::AggregateAndProof { + aggregator_index: 1, + aggregate: gloas::Attestation::from(&expected), + selection_proof: Default::default(), + }, + signature: Default::default(), + }; + Validated::Aggregate { + aggregate: Box::new(SignedAggregateAndProof::Gloas(gloas_signed)), + attesting_indices: Vec::new(), + } + .forward(&server, Instant::now(), Outcome::Accept); + assert_eq!( + server.attestation_pool.lock().unwrap().block_candidates(), + vec![expected] + ); + } + /// Only `Accept` means the signatures were verified; anything else must /// stay out of the pool, since one unverified attestation fails the /// whole block it is packed into. @@ -1030,6 +1079,36 @@ mod tests { assert_eq!(received.try_recv(), Ok(Sent::PayloadAttestation)); } + /// An accepted payload attestation is pooled for block production, with + /// or without a chain actor attached; any other outcome stays out, since + /// one unverified share fails the aggregate it is packed into. + #[tokio::test] + async fn an_accepted_payload_attestation_is_pooled_and_others_are_not() { + let server = unconnected_beacon_server(Config::mainnet(), 0).await; + let forward = |outcome| { + Validated::PayloadAttestation(payload_attestation(5, 1)).forward( + &server, + Instant::now(), + outcome, + ) + }; + forward(Outcome::Ignore(IgnoreReason::Overloaded)); + forward(Outcome::Reject(RejectReason::NotInPtc)); + assert!( + server + .payload_attestation_pool + .lock() + .unwrap() + .all(None) + .is_empty() + ); + forward(Outcome::Accept); + assert_eq!( + server.payload_attestation_pool.lock().unwrap().all(None), + vec![payload_attestation(5, 1)] + ); + } + /// The pool an aggregate or a subnet attestation draws its stateful-check /// permit from is not the pool a block or a column draws from: exhausting /// one must leave the other untouched, or a burst on this topic could diff --git a/crates/net/p2p/src/gossipsub/handler.rs b/crates/net/p2p/src/gossipsub/handler.rs index 4a18771e..1b664a47 100644 --- a/crates/net/p2p/src/gossipsub/handler.rs +++ b/crates/net/p2p/src/gossipsub/handler.rs @@ -788,6 +788,147 @@ pub async fn publish_beacon_block(server: &mut P2PServer, block: SignedBeaconBlo } } +/// Gossip a gloas execution payload envelope a validator client signed, +/// handed over by the Beacon API, on `execution_payload`, its data column +/// sidecars on their subnets, and pass all of them to the chain actor. +/// +/// Like [`publish_beacon_block`], the only way this node takes its own +/// envelope in, since gossipsub never delivers a node its own messages. The +/// envelope goes to the actor as it is. It is safe to hand over before the +/// block it reveals has been imported, which is the usual case (the validator +/// client publishes the envelope as soon as the block is on its way): the +/// actor holds an envelope for a block it has not imported (`awaiting_block`), +/// or for one it has stored but not imported (`awaiting_import`), and applies +/// it when the block's post-state exists. +/// +/// The sidecars take the route every sidecar gossip did not accept takes, the +/// chain checks (`column_checks::check_and_forward`), which parks one whose +/// block is not imported yet and hands it back once it is. Their subnet +/// topics are published to whether or not this node custodies the subnet: +/// gossipsub sends to a topic it is not subscribed to through its fanout +/// peers. +pub async fn publish_execution_payload_envelope( + server: &mut P2PServer, + envelope: ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope, + sidecars: Vec, +) { + let slot = envelope.message.payload.slot_number; + let block_root = envelope.message.beacon_block_root; + let Some(beacon) = server.wire.beacon() else { + error!( + slot, + "An execution payload envelope reached a lean node; dropping it" + ); + return; + }; + let Some(digest) = beacon.publish_digest(slot) else { + warn!( + slot, + "No held fork digest covers this envelope's slot; not publishing" + ); + return; + }; + let topic = IdentTopic::new(beacon_topics::topic_name( + digest, + beacon_topics::EXECUTION_PAYLOAD, + )); + server + .swarm_handle + .publish(topic, compress_message(&envelope.to_ssz())); + info!( + slot, + builder_index = envelope.message.builder_index, + block_root = %ShortRoot(&block_root.0), + "Published execution payload envelope to gossipsub" + ); + + let mut published_columns = 0usize; + for sidecar in &sidecars { + let DataColumnSidecar::Gloas(gloas_sidecar) = sidecar else { + warn!( + slot, + "Skipping a non-gloas sidecar published with a gloas envelope" + ); + continue; + }; + let subnet_id = gloas_sidecar.index + % ethlambda_types::beacon::constants::DATA_COLUMN_SIDECAR_SUBNET_COUNT; + let topic = IdentTopic::new(beacon_topics::data_column_topic_name(digest, subnet_id)); + server + .swarm_handle + .publish(topic, compress_message(&gloas_sidecar.to_ssz())); + published_columns += 1; + } + if published_columns > 0 { + info!( + slot, + block_root = %ShortRoot(&block_root.0), + columns = published_columns, + "Published data column sidecars to gossipsub" + ); + } + + if let Some(blockchain) = server.blockchain.clone() { + let _ = blockchain + .new_execution_payload_envelope(Box::new(envelope), BlockArrival::now()) + .inspect_err(|err| error!(%err, "Failed to hand the published envelope to the chain")); + } + crate::beacon::column_checks::check_and_forward(server, sidecars); +} + +/// Gossip a payload attestation message a validator client signed, handed +/// over by the Beacon API, on `payload_attestation_message`, and pass it to +/// the chain actor. +/// +/// The API validated it with the checks gossip would apply, so it goes out +/// as is. Gossipsub never delivers a node its own message, so the chain actor +/// is handed it here to count the vote in its fork choice. The seen cache is +/// marked as well: a peer that echoes the vote back would otherwise pass +/// triage and be validated and forwarded a second time. +pub async fn publish_payload_attestation_message( + server: &mut P2PServer, + message: ethlambda_types::beacon::containers::gloas::PayloadAttestationMessage, +) { + let slot = message.data.slot; + let validator = message.validator_index; + let Some(beacon) = server.wire.beacon() else { + error!( + slot, + "A payload attestation reached a lean node; dropping it" + ); + return; + }; + let Some(digest) = beacon.publish_digest(slot) else { + warn!( + slot, + "No held fork digest covers this payload attestation's slot; not publishing" + ); + return; + }; + let topic = IdentTopic::new(beacon_topics::topic_name( + digest, + beacon_topics::PAYLOAD_ATTESTATION_MESSAGE, + )); + server + .swarm_handle + .publish(topic, compress_message(&message.to_ssz())); + server.seen_payload_attestations.record(slot, validator); + info!( + slot, + validator, + block_root = %ShortRoot(&message.data.beacon_block_root.0), + payload_present = message.data.payload_present, + "Published payload attestation to gossipsub" + ); + if let Some(ref blockchain) = server.blockchain { + let _ = blockchain + .new_payload_attestation_message(message, BlockArrival::now()) + .inspect_err( + |err| error!(%err, "Failed to hand the published payload attestation to the chain"), + ); + } +} + /// The beacon wall-clock slot, from the wire's genesis and slot duration. fn beacon_wall_slot(wire: &BeaconWire) -> u64 { let genesis_ms = wire.genesis_time.saturating_mul(1000); diff --git a/crates/net/p2p/src/gossipsub/mod.rs b/crates/net/p2p/src/gossipsub/mod.rs index 608584a2..1701c5f3 100644 --- a/crates/net/p2p/src/gossipsub/mod.rs +++ b/crates/net/p2p/src/gossipsub/mod.rs @@ -7,5 +7,6 @@ pub use handler::{ handle_gossip_message, join_aggregator_subnets, leave_expired_aggregator_subnets, prune_attestation_pool, publish_aggregated_attestation, publish_attestation, publish_beacon_aggregate, publish_beacon_attestation, publish_beacon_block, publish_block, + publish_execution_payload_envelope, publish_payload_attestation_message, }; pub use messages::{aggregation_topic, attestation_subnet_topic, block_topic, topic_kind}; diff --git a/crates/net/p2p/src/lib.rs b/crates/net/p2p/src/lib.rs index cb164b72..ff52d419 100644 --- a/crates/net/p2p/src/lib.rs +++ b/crates/net/p2p/src/lib.rs @@ -45,16 +45,19 @@ use ethlambda_network_api::{ }, rpc_to_p2p::{ PublishBeaconAggregate, PublishBeaconAttestation, PublishBeaconBlock, + PublishExecutionPayloadEnvelope, PublishPayloadAttestationMessage, SubscribeAttestationSubnets, }, }; use ethlambda_state_transition::beacon::aggregate::MAX_AGGREGATES_PER_SLOT; -use ethlambda_state_transition::beacon::attestation_pool::SharedAttestationPool; use ethlambda_state_transition::beacon::gossip::{ SeenBlockColumns, SeenBlocks, SeenColumns, aggregate::SeenAggregates, attestation::SeenAttestations, envelope::SeenEnvelopes, payload_attestation::SeenPayloadAttestations, }; +use ethlambda_state_transition::beacon::{ + attestation_pool::SharedAttestationPool, payload_attestation_pool::SharedPayloadAttestationPool, +}; use ethlambda_storage::{Chain, Store}; use ethlambda_types::beacon::preset::{MAX_VALIDATORS_PER_COMMITTEE, SLOTS_PER_EPOCH}; use ethlambda_types::primitives::H256; @@ -1089,6 +1092,7 @@ impl P2P { node_names: HashMap, discovery: Option, attestation_pool: SharedAttestationPool, + payload_attestation_pool: SharedPayloadAttestationPool, ) -> Result { let discovery = match discovery { Some(config) => Some(spawn_discovery(config).await?), @@ -1151,6 +1155,7 @@ impl P2P { ATTESTATION_VALIDATION_PERMITS, )), attestation_pool, + payload_attestation_pool, aggregator_subnets: HashMap::new(), }; let discovery_enabled = server.discovery.is_some(); @@ -1286,6 +1291,11 @@ pub struct P2PServer { /// touches it. pub(crate) attestation_pool: SharedAttestationPool, + /// Accepted `payload_attestation_message` votes, shared with the Beacon + /// API that serves and fills the same pool. Filled by `verdict::forward`; + /// lean never touches it. + pub(crate) payload_attestation_pool: SharedPayloadAttestationPool, + /// The attestation subnets joined for a validator client's aggregators, /// each with the last slot it is needed for. Short-lived by design: never /// advertised in `attnets`, and left once the slot has passed. The @@ -1590,6 +1600,18 @@ impl Handler for P2PServer { } } +impl Handler for P2PServer { + async fn handle(&mut self, msg: PublishExecutionPayloadEnvelope, _ctx: &Context) { + gossipsub::publish_execution_payload_envelope(self, *msg.envelope, msg.sidecars).await; + } +} + +impl Handler for P2PServer { + async fn handle(&mut self, msg: PublishPayloadAttestationMessage, _ctx: &Context) { + gossipsub::publish_payload_attestation_message(self, msg.message).await; + } +} + impl Handler for P2PServer { async fn handle(&mut self, msg: SubscribeAttestationSubnets, _ctx: &Context) { gossipsub::join_aggregator_subnets(self, msg.subnets); @@ -2760,6 +2782,7 @@ pub(crate) mod test_support { crate::ATTESTATION_VALIDATION_PERMITS, )), attestation_pool: Default::default(), + payload_attestation_pool: Default::default(), aggregator_subnets: HashMap::new(), } } diff --git a/crates/net/p2p/src/req_resp/handlers.rs b/crates/net/p2p/src/req_resp/handlers.rs index 6f24bcfc..daad0659 100644 --- a/crates/net/p2p/src/req_resp/handlers.rs +++ b/crates/net/p2p/src/req_resp/handlers.rs @@ -2789,6 +2789,7 @@ pub(crate) mod tests { crate::ATTESTATION_VALIDATION_PERMITS, )), attestation_pool: Default::default(), + payload_attestation_pool: Default::default(), aggregator_subnets: HashMap::new(), } } diff --git a/crates/net/rpc/src/beacon/gloas_proposal.rs b/crates/net/rpc/src/beacon/gloas_proposal.rs new file mode 100644 index 00000000..fb9218a1 --- /dev/null +++ b/crates/net/rpc/src/beacon/gloas_proposal.rs @@ -0,0 +1,1453 @@ +//! Gloas block production and payload envelope publication. +//! +//! `POST /eth/v4/validator/blocks/{slot}` (`produceBlockV4`) builds a +//! self-built block with this node's own execution client and hands the +//! validator client the block, and, with `include_payload`, the unsigned +//! envelope that reveals its payload plus the blobs and cell proofs. The three +//! pieces come back separately because a gloas proposer signs the block and +//! the envelope on their own: the block goes out first through +//! `POST /eth/v2/beacon/blocks` (see `proposal.rs`), the envelope after it +//! through `POST /eth/v1/beacon/execution_payload_envelopes`, which also +//! publishes the data columns of the blobs (a builder's duty in gloas, which a +//! self-building proposer therefore takes on). +//! +//! `GET /eth/v1/validator/execution_payload_envelopes/{slot}/{beacon_block_root}` +//! serves the envelope back for a validator client that asked for the block +//! with `include_payload=false`. +//! +//! This node never takes a builder's bid: the `BuilderConfig` request body is +//! decoded, as the specification requires of a body that cannot be, and +//! otherwise ignored. +//! +//! What `produceBlockV4` builds is kept in a small cache, keyed by slot and +//! block root and holding the current and the previous slot only, so the +//! envelope and the blobs can be served and published without the caller +//! sending them back (`Eth-Blob-Data-Included: false`). + +use std::{ + collections::BTreeMap, + sync::{Arc, Mutex}, + time::{Duration, Instant}, +}; + +use axum::{ + Extension, Router, + body::Bytes, + extract::{DefaultBodyLimit, Path, Query, State, rejection::QueryRejection}, + http::{HeaderMap, HeaderValue, StatusCode, header}, + response::{IntoResponse, Response}, + routing::{get, post}, +}; +use ethlambda_blockchain::checkpoint_hash; +use ethlambda_engine::{ + CustodyColumns, EngineClient, ForkchoiceStateV1, + building::{BuiltGloasPayload, PayloadAttributesV3, PayloadAttributesV4}, +}; +use ethlambda_network_api::RpcToP2PRef; +use ethlambda_state_transition::beacon::{ + attestation_pool::SharedAttestationPool, + block_production::advance_to_slot, + bls, + fork_choice::{ + get_head_node, gloas_verify_data_column_sidecar, + gloas_verify_data_column_sidecar_kzg_proofs, should_build_on_full, + }, + gloas_block_production::{ + GloasBlockInputs, GloasPayloadInputs, GloasProduced, assemble_gloas_block, + gloas_data_column_sidecars, gloas_payload_inputs, pack_gloas_attestations, + pack_payload_attestations, parse_gloas_execution_requests, + }, + helpers::{ + accessors::{get_beacon_proposer_index, get_domain}, + misc::compute_signing_root, + }, + payload_attestation_pool::SharedPayloadAttestationPool, + stf::gloas::verify_execution_payload_envelope_signature, +}; +use ethlambda_storage::Store; +use ethlambda_types::{ + beacon::{ + config::Config, + constants, + containers::{ + self, BeaconState, DataColumnSidecar, + deneb::Blob, + gloas::{ + BeaconBlock, ExecutionPayloadEnvelope, ExecutionRequests, + SignedExecutionPayloadEnvelope, + }, + }, + fork::ForkName, + fork_choice::{ForkChoiceNode, PayloadStatus}, + primitives::{BlsSignature, Bytes32, ExecutionAddress, HashTreeRoot as _, KzgProof, Slot}, + signing::compute_epoch_at_slot, + }, + primitives::H256, +}; +use libssz::{SszDecode as _, SszEncode as _}; +use libssz_derive::{SszDecode, SszEncode}; +use serde::Deserialize; +use tracing::{debug, info, warn}; + +use crate::beacon::{ + ApiError, + proposal::{Blobs, CellKzgProofs, decimal, require_gloas_slot}, + validator::{FeeRecipients, head}, +}; +use crate::shared::content::{Encoding, ssz_response, with_consensus_version}; +// The node's custody set as the Beacon API's handles carry it, renamed so it +// cannot be confused with the engine's bitfield of the same name. +use crate::CustodyColumns as NodeCustodyColumns; + +/// How long `publishExecutionPayloadEnvelope` waits for a block this node has +/// not imported yet. +/// +/// A validator client publishes the envelope right after `publishBlockV2` +/// returns, and that returns once the block is handed to the network actor, not +/// once the chain actor has imported it, so the envelope routinely arrives +/// first. +const BLOCK_WAIT: Duration = Duration::from_secs(4); +const BLOCK_POLL_INTERVAL: Duration = Duration::from_millis(50); + +/// What `produceBlockV4` built for one block, kept for the envelope endpoints. +#[derive(Debug, Clone)] +struct CachedPayload { + envelope: ExecutionPayloadEnvelope, + blobs: Vec>, + cell_proofs: Vec, +} + +/// The current and previous slots' productions, by `(slot, block root)`. +#[derive(Debug, Clone, Default)] +pub(crate) struct PayloadCache(Arc>>); + +impl PayloadCache { + fn insert(&self, slot: Slot, block_root: H256, payload: CachedPayload) { + let mut entries = self.0.lock().expect("payload cache lock poisoned"); + entries.insert((slot, block_root), payload); + let keep_from = slot.saturating_sub(1); + entries.retain(|(held, _), _| *held >= keep_from); + } + + fn get(&self, slot: Slot, block_root: H256) -> Option { + self.0 + .lock() + .expect("payload cache lock poisoned") + .get(&(slot, block_root)) + .cloned() + } +} + +/// Gloas's `BlockContents`, the Beacon API's envelope for a self-built block +/// with its payload: not a consensus container, so it lives with the API. +#[derive(Debug, Clone, PartialEq, SszEncode, SszDecode, serde::Serialize)] +pub(crate) struct GloasBlockContents { + pub(crate) block: BeaconBlock, + pub(crate) execution_payload_envelope: ExecutionPayloadEnvelope, + #[serde(serialize_with = "ethlambda_types::beacon::serde_helpers::seq::serialize")] + pub(crate) kzg_proofs: CellKzgProofs, + #[serde(serialize_with = "ethlambda_types::beacon::serde_helpers::ssz_hex_seq::serialize")] + pub(crate) blobs: Blobs, +} + +/// Gloas's `SignedExecutionPayloadEnvelopeContents`, what the stateless form of +/// `publishExecutionPayloadEnvelope` receives. +#[derive(Debug, Clone, PartialEq, SszEncode, SszDecode)] +pub(crate) struct GloasSignedEnvelopeContents { + pub(crate) signed_execution_payload_envelope: SignedExecutionPayloadEnvelope, + pub(crate) kzg_proofs: CellKzgProofs, + pub(crate) blobs: Blobs, +} + +pub(crate) fn routes() -> Router { + Router::new() + .route("/eth/v4/validator/blocks/{slot}", post(post_produce_block)) + .route( + "/eth/v1/validator/execution_payload_envelopes/{slot}/{beacon_block_root}", + get(get_envelope), + ) + .route( + "/eth/v1/beacon/execution_payload_envelopes", + post(post_envelope).layer(DefaultBodyLimit::max(super::MAX_PUBLISH_BODY_BYTES)), + ) + .layer(Extension(PayloadCache::default())) +} + +fn is_ssz(headers: &HeaderMap) -> bool { + headers + .get(header::CONTENT_TYPE) + .and_then(|value| value.to_str().ok()) + .is_some_and(|value| value.starts_with(crate::SSZ_CONTENT_TYPE)) +} + +fn consensus_version(headers: &HeaderMap) -> Option { + headers + .get("eth-consensus-version") + .and_then(|value| value.to_str().ok()) + .and_then(ForkName::parse) +} + +/// `BuilderConfig` as it arrives as JSON. Only decoded: this node takes no +/// builder bids, so nothing past the shape is read. +#[derive(Debug, Deserialize)] +struct BuilderConfigJson { + #[serde(with = "ethlambda_types::beacon::serde_helpers::quoted_or_bare")] + #[allow(dead_code)] + min_bid: u64, + #[serde(with = "ethlambda_types::beacon::serde_helpers::quoted_or_bare")] + #[allow(dead_code)] + builder_boost_factor: u64, + builders: Vec, +} + +/// Decodes the request body as a `BuilderConfig`, returning how many builder +/// entries it names. A body that does not decode is invalid per the +/// specification. +/// +/// The SSZ form is `min_bid`, `builder_boost_factor` and the offset of the +/// `builders` list, so a decodable body is at least that long and the offset +/// points just past those three fields; the entries themselves are not parsed. +fn decode_builder_config(headers: &HeaderMap, body: &[u8]) -> Result { + if is_ssz(headers) { + let offset = body + .get(16..20) + .map(|bytes| u32::from_le_bytes(bytes.try_into().expect("four bytes"))); + return match offset { + Some(20) => Ok(usize::from(body.len() > 20)), + _ => Err(ApiError::BadRequest("the body is not a BuilderConfig")), + }; + } + serde_json::from_slice::(body) + .map(|config| config.builders.len()) + .map_err(|_| ApiError::BadRequest("the body is not a BuilderConfig")) +} + +#[derive(Debug, Deserialize)] +struct ProduceQuery { + randao_reveal: BlsSignature, + #[serde(default)] + graffiti: Option, + include_payload: bool, + /// Accepted for compatibility. Not honored: the block's state root comes + /// from running it through the state transition, which verifies the reveal. + #[serde(default)] + #[allow(dead_code)] + skip_randao_verification: Option, +} + +/// `POST /eth/v4/validator/blocks/{slot}`. +#[allow(clippy::too_many_arguments)] +async fn post_produce_block( + Path(slot): Path, + query: Result, QueryRejection>, + State(store): State, + Extension(engine): Extension>, + Extension(pool): Extension, + Extension(ptc_pool): Extension, + Extension(fee_recipients): Extension, + Extension(custody): Extension, + Extension(cache): Extension, + headers: HeaderMap, + body: Bytes, +) -> Response { + let Ok(slot) = slot.parse::() else { + return ApiError::BadRequest("invalid slot").into_response(); + }; + let Ok(Query(query)) = query else { + return ApiError::BadRequest( + "randao_reveal and include_payload are required, graffiti is a 32-byte hex string", + ) + .into_response(); + }; + if consensus_version(&headers).is_some_and(|fork| fork != ForkName::Gloas) { + return ApiError::BadRequest("Eth-Consensus-Version must be gloas").into_response(); + } + // Ahead of the engine check: a slot this node cannot build is a 400 on any + // node, not a 503 on one with no execution client. + if let Err(err) = require_gloas_slot( + &store.config(), + slot, + "this endpoint serves gloas slots only", + ) { + return err.into_response(); + } + let builders = match decode_builder_config(&headers, &body) { + Ok(builders) => builders, + Err(err) => return err.into_response(), + }; + if builders > 0 { + debug!(%slot, builders, "Ignoring the builders of the block production request"); + } + let Some(engine) = engine else { + return ApiError::ServiceUnavailable( + "no execution client configured to build a payload with", + ) + .into_response(); + }; + + let graffiti = query.graffiti.unwrap_or(Bytes32::ZERO); + let produced = produce( + &store, + &engine, + &pool, + &ptc_pool, + &fee_recipients, + &custody, + slot, + query.randao_reveal, + graffiti, + ) + .await; + let produced = match produced { + Ok(produced) => produced, + Err(err) => return err.into_response(), + }; + let Produced { + built, + block, + envelope, + payload_value, + } = produced; + + let block_root = block.hash_tree_root(); + cache.insert( + slot, + block_root, + CachedPayload { + envelope: envelope.clone(), + blobs: built.blobs_bundle.blobs.clone(), + cell_proofs: built.blobs_bundle.proofs.clone(), + }, + ); + + let accept = headers.get(header::ACCEPT).and_then(|v| v.to_str().ok()); + let encoding = Encoding::from_accept(accept); + let included = query.include_payload; + let mut response = if included { + let (Ok(kzg_proofs), Ok(blobs)) = ( + CellKzgProofs::try_from(built.blobs_bundle.proofs), + blobs_list(built.blobs_bundle.blobs), + ) else { + return ApiError::Internal("the blobs bundle exceeds the block's bounds") + .into_response(); + }; + let contents = GloasBlockContents { + block, + execution_payload_envelope: envelope, + kzg_proofs, + blobs, + }; + match encoding { + Encoding::Ssz => ssz_response(contents.to_ssz()), + Encoding::Json => block_json(&payload_value, true, &contents), + } + } else { + match encoding { + Encoding::Ssz => ssz_response(block.to_ssz()), + Encoding::Json => block_json(&payload_value, false, &block), + } + }; + let response_headers = response.headers_mut(); + response_headers.insert( + "eth-execution-payload-included", + HeaderValue::from_static(if included { "true" } else { "false" }), + ); + if let Ok(value) = HeaderValue::from_str(&payload_value) { + response_headers.insert("eth-execution-payload-value", value); + } + // Not computed: nothing here reads it, and the builder comparison it + // exists for does not happen on this node. + response_headers.insert("eth-consensus-block-value", HeaderValue::from_static("0")); + with_consensus_version(response, ForkName::Gloas) +} + +fn block_json(payload_value: &str, included: bool, data: &T) -> Response { + crate::json_response(serde_json::json!({ + "version": ForkName::Gloas.as_str(), + "consensus_block_value": "0", + "execution_payload_value": payload_value, + "execution_payload_included": included, + "data": data, + })) +} + +fn blobs_list(blobs: Vec>) -> Result { + let blobs = blobs + .into_iter() + .map(|blob| Blob::try_from(blob).map_err(|_| ())) + .collect::, ()>>()?; + Blobs::try_from(blobs).map_err(|_| ()) +} + +/// Everything `produceBlockV4` built. +struct Produced { + built: BuiltGloasPayload, + block: BeaconBlock, + envelope: ExecutionPayloadEnvelope, + payload_value: String, +} + +/// What the build needs from the chain, read and advanced off the runtime. +struct Prepared { + state: BeaconState, + inputs: GloasPayloadInputs, + proposer: u64, + head_root: H256, + head_slot: Slot, + /// The parent envelope's requests when building on its full payload and + /// the parent is gloas; empty otherwise. + parent_requests: ExecutionRequests, +} + +#[allow(clippy::too_many_arguments)] +async fn produce( + store: &Store, + engine: &EngineClient, + pool: &SharedAttestationPool, + ptc_pool: &SharedPayloadAttestationPool, + fee_recipients: &FeeRecipients, + custody: &NodeCustodyColumns, + slot: Slot, + randao_reveal: BlsSignature, + graffiti: Bytes32, +) -> Result { + let prepare_store = store.clone(); + let prepared = + tokio::task::spawn_blocking(move || prepare(&prepare_store, slot, randao_reveal)) + .await + .map_err(|_| ApiError::Internal("preparing the block failed"))??; + + let built = build_payload(store, engine, fee_recipients, custody, &prepared).await?; + let bundle = &built.blobs_bundle; + if bundle.commitments.len() != bundle.blobs.len() + || bundle.proofs.len() + != bundle.blobs.len() * ethlambda_types::beacon::preset::CELLS_PER_EXT_BLOB + { + warn!(%slot, "The execution client's blobs bundle is inconsistent"); + return Err(ApiError::ServiceUnavailable( + "the execution client returned an inconsistent blobs bundle", + )); + } + let execution_requests = parse_gloas_execution_requests(&built.execution_requests) + .map_err(|_| ApiError::Internal("the execution client's request list is malformed"))?; + let payload_value = decimal(&built.block_value); + + let candidates = pool + .lock() + .expect("attestation pool lock poisoned") + .block_candidates(); + let messages = ptc_pool + .lock() + .expect("payload attestation pool lock poisoned") + .messages_for(slot.saturating_sub(1), prepared.head_root); + let config = store.config(); + let assembled = { + let built = built.clone(); + tokio::task::spawn_blocking(move || { + assemble( + &config, + prepared, + built, + execution_requests, + candidates, + messages, + randao_reveal, + graffiti, + ) + }) + .await + .map_err(|_| ApiError::Internal("assembling the block failed"))?? + }; + let GloasProduced { block, envelope } = assembled; + info!( + %slot, + proposer = block.proposer_index, + attestations = block.body.attestations.len(), + payload_attestations = block.body.payload_attestations.len(), + blobs = built.blobs_bundle.blobs.len(), + "Produced gloas block" + ); + Ok(Produced { + built, + block, + envelope, + payload_value, + }) +} + +/// The chain-side half of production: pick the parent payload branch, advance +/// the head state to `slot` and derive the payload inputs from it. +fn prepare(store: &Store, slot: Slot, randao_reveal: BlsSignature) -> Result { + let config = store.config(); + let (head_root, head_state) = head(store)?; + let head_slot = head_state.slot(); + if slot <= head_slot { + return Err(ApiError::BadRequest("slot is not after the head block")); + } + + // The payload branch of the head: the one fork choice recorded, or, when + // it has not been recorded for this head yet, a fresh walk. + let payload_status = match store.head_payload_status() { + Some(status) => status, + None => { + get_head_node(store, &config) + .map_err(|_| ApiError::Internal("fork choice could not name the head"))? + .payload_status + } + }; + let node = ForkChoiceNode { + root: head_root, + payload_status, + }; + let build_on_full = should_build_on_full(store, node, slot) + .map_err(|_| ApiError::Internal("could not decide which parent payload to build on"))?; + debug_assert_ne!(payload_status, PayloadStatus::Pending); + + let head_is_gloas = matches!( + store.get_signed_block(&head_root), + Ok(Some(containers::SignedBeaconBlock::Gloas(_))) + ); + let parent_requests = if build_on_full && head_is_gloas { + store + .get_execution_payload_envelope(&head_root) + .map_err(|_| ApiError::Internal("store read failed"))? + .ok_or(ApiError::ServiceUnavailable( + "the parent's payload envelope is not held to build on", + ))? + .message + .execution_requests + } else { + ExecutionRequests::default() + }; + + let state = advance_to_slot(&head_state, slot, &config) + .map_err(|_| ApiError::Internal("advancing the head state failed"))?; + let proposer = get_beacon_proposer_index(&state) + .map_err(|_| ApiError::Internal("no proposer for the slot"))?; + verify_randao_reveal(&state, proposer, randao_reveal)?; + let inputs = gloas_payload_inputs(&state, build_on_full, &parent_requests, &config) + .map_err(|_| ApiError::Internal("computing the payload attributes failed"))?; + Ok(Prepared { + state, + inputs, + proposer, + head_root, + head_slot, + parent_requests, + }) +} + +/// A reveal that does not verify would fail the block's own state transition +/// later with an opaque error; here it is the caller's mistake and says so. +fn verify_randao_reveal( + state: &BeaconState, + proposer: u64, + randao_reveal: BlsSignature, +) -> Result<(), ApiError> { + let epoch = compute_epoch_at_slot(state.slot()); + let domain = get_domain(state, constants::DOMAIN_RANDAO, Some(epoch)); + let signing_root = compute_signing_root(epoch.hash_tree_root(), domain); + let pubkey = state + .validator(proposer) + .map_err(|_| ApiError::Internal("the proposer is not in the registry"))? + .pubkey; + if bls::verify(&pubkey, signing_root, &randao_reveal) { + Ok(()) + } else { + Err(ApiError::BadRequest("invalid randao_reveal")) + } +} + +/// Ask the execution client to build on the chosen parent payload for the +/// prepared slot, then collect what it built. +/// +/// Collected straight away rather than after waiting, as the fulu path does: the +/// execution client starts with a valid payload and improves it, so an early +/// `getPayload` always answers, and the validator client asks at the start of +/// the slot, when the block is due. +async fn build_payload( + store: &Store, + engine: &EngineClient, + fee_recipients: &FeeRecipients, + custody: &NodeCustodyColumns, + prepared: &Prepared, +) -> Result { + let inputs = &prepared.inputs; + let fee_recipient = fee_recipients + .lock() + .expect("fee recipient lock poisoned") + .get(&prepared.proposer) + .copied() + .unwrap_or_else(|| { + warn!( + proposer = prepared.proposer, + "No fee recipient prepared for the proposer; using the zero address" + ); + ExecutionAddress::ZERO + }); + let forkchoice = ForkchoiceStateV1 { + head_block_hash: inputs.head_block_hash, + safe_block_hash: checkpoint_hash(store, store.beacon_justified_checkpoint().root), + finalized_block_hash: checkpoint_hash(store, store.beacon_finalized_checkpoint().root), + }; + let attributes = PayloadAttributesV4 { + v3: PayloadAttributesV3 { + timestamp: inputs.timestamp, + prev_randao: inputs.prev_randao, + suggested_fee_recipient: fee_recipient, + withdrawals: inputs.withdrawals.clone(), + parent_beacon_block_root: inputs.parent_beacon_block_root, + }, + slot_number: inputs.slot_number, + target_gas_limit: inputs.target_gas_limit, + }; + let custody_columns = CustodyColumns::from_indices(custody.0.iter().copied()); + let (status, payload_id) = engine + .forkchoice_updated_v4_with_attributes(&forkchoice, &attributes, custody_columns) + .await + .map_err(|err| { + warn!(%err, "forkchoiceUpdatedV4 with payload attributes failed"); + ApiError::ServiceUnavailable("the execution client did not start building") + })?; + let Some(payload_id) = payload_id else { + warn!(status = ?status.status, "The execution client declined to build a payload"); + return Err(ApiError::ServiceUnavailable( + "the execution client declined to build a payload", + )); + }; + engine.get_payload_v6(payload_id).await.map_err(|err| { + warn!(%err, "getPayloadV6 failed"); + ApiError::ServiceUnavailable("the execution client did not return a payload") + }) +} + +/// The pure half: the pooled operations and the built payload into a block and +/// its envelope. +#[allow(clippy::too_many_arguments)] +fn assemble( + config: &Config, + prepared: Prepared, + built: BuiltGloasPayload, + execution_requests: ExecutionRequests, + candidates: Vec, + messages: Vec, + randao_reveal: BlsSignature, + graffiti: Bytes32, +) -> Result { + let Prepared { + state, + head_root, + head_slot, + parent_requests, + .. + } = prepared; + let attestations = pack_gloas_attestations(&state, candidates); + let payload_attestations = + pack_payload_attestations(&state, head_root, head_slot, messages, config); + let commitments = built.blobs_bundle.commitments; + let inputs = |attestations, payload_attestations| GloasBlockInputs { + randao_reveal, + graffiti, + attestations, + payload_attestations, + parent_execution_requests: parent_requests.clone(), + execution_payload: built.execution_payload.clone(), + blob_kzg_commitments: commitments.clone(), + execution_requests: execution_requests.clone(), + }; + let operations = attestations.len() + payload_attestations.len(); + match assemble_gloas_block(&state, inputs(attestations, payload_attestations), config) { + Ok(produced) => Ok(produced), + // The packers check every operation's signature against this state, so + // this should not happen; but a block without them still earns the + // proposal, and one that fails to build earns nothing. + Err(err) if operations > 0 => { + warn!(slot = state.slot(), %err, "Block with operations failed to build; retrying without"); + assemble_gloas_block(&state, inputs(Vec::new(), Vec::new()), config) + .map_err(|_| ApiError::Internal("the block failed to build")) + } + Err(_) => Err(ApiError::Internal("the block failed to build")), + } +} + +/// `GET /eth/v1/validator/execution_payload_envelopes/{slot}/{beacon_block_root}`. +async fn get_envelope( + Path((slot, block_root)): Path<(String, String)>, + Extension(cache): Extension, + headers: HeaderMap, +) -> Response { + let Ok(slot) = slot.parse::() else { + return ApiError::BadRequest("invalid slot").into_response(); + }; + let Some(block_root) = parse_root(&block_root) else { + return ApiError::BadRequest("invalid beacon_block_root").into_response(); + }; + let Some(cached) = cache.get(slot, block_root) else { + return ApiError::NotFound("execution payload envelope not available for the slot") + .into_response(); + }; + let accept = headers.get(header::ACCEPT).and_then(|v| v.to_str().ok()); + let response = match Encoding::from_accept(accept) { + Encoding::Ssz => ssz_response(cached.envelope.to_ssz()), + Encoding::Json => crate::json_response(serde_json::json!({ + "version": ForkName::Gloas.as_str(), + "data": cached.envelope, + })), + }; + with_consensus_version(response, ForkName::Gloas) +} + +fn parse_root(text: &str) -> Option { + let bytes: [u8; 32] = hex::decode(text.strip_prefix("0x")?) + .ok()? + .try_into() + .ok()?; + Some(H256(bytes)) +} + +/// The block and its post-state for `root`, waiting up to `timeout` for the +/// chain actor to import them. +async fn wait_for_block( + store: &Store, + root: H256, + timeout: Duration, +) -> Option<(containers::SignedBeaconBlock, Arc)> { + let deadline = Instant::now() + timeout; + loop { + if let (Ok(Some(block)), Ok(Some(state))) = + (store.get_signed_block(&root), store.get_state(&root)) + { + return Some((block, state)); + } + if Instant::now() >= deadline { + return None; + } + tokio::time::sleep(BLOCK_POLL_INTERVAL).await; + } +} + +/// `POST /eth/v1/beacon/execution_payload_envelopes`, SSZ only. +/// +/// The envelope must fulfill the bid of a gloas block this node holds (waiting +/// briefly for the block, see [`BLOCK_WAIT`]), carry the proposer's signature +/// under `DOMAIN_BEACON_BUILDER`, and come with the blobs the bid commits to, +/// from the body (`Eth-Blob-Data-Included: true`) or from what +/// `produceBlockV4` cached (`false`). The data columns are built from the +/// blobs, checked against the bid's commitments, and gossiped with the +/// envelope. +async fn post_envelope( + State(store): State, + Extension(p2p): Extension, + Extension(cache): Extension, + headers: HeaderMap, + body: Bytes, +) -> Response { + post_envelope_waiting(&store, &p2p, &cache, &headers, &body, BLOCK_WAIT).await +} + +async fn post_envelope_waiting( + store: &Store, + p2p: &RpcToP2PRef, + cache: &PayloadCache, + headers: &HeaderMap, + body: &[u8], + block_wait: Duration, +) -> Response { + if consensus_version(headers) != Some(ForkName::Gloas) { + return ApiError::BadRequest("Eth-Consensus-Version must be gloas").into_response(); + } + if !is_ssz(headers) { + return ( + StatusCode::UNSUPPORTED_MEDIA_TYPE, + "envelopes are accepted as application/octet-stream only", + ) + .into_response(); + } + let blob_data_included = match headers + .get("eth-blob-data-included") + .and_then(|value| value.to_str().ok()) + { + Some("true") => true, + Some("false") => false, + _ => { + return ApiError::BadRequest("Eth-Blob-Data-Included must be true or false") + .into_response(); + } + }; + let (signed, supplied) = if blob_data_included { + match GloasSignedEnvelopeContents::from_ssz_bytes(body) { + Ok(contents) => ( + contents.signed_execution_payload_envelope, + Some(( + contents + .blobs + .iter() + .map(|blob| blob.to_vec()) + .collect::>(), + contents.kzg_proofs.to_vec(), + )), + ), + Err(_) => { + return ApiError::BadRequest( + "the body is not a gloas SignedExecutionPayloadEnvelopeContents", + ) + .into_response(); + } + } + } else { + match SignedExecutionPayloadEnvelope::from_ssz_bytes(body) { + Ok(signed) => (signed, None), + Err(_) => { + return ApiError::BadRequest( + "the body is not a gloas SignedExecutionPayloadEnvelope", + ) + .into_response(); + } + } + }; + + let block_root = signed.message.beacon_block_root; + let Some((block, post_state)) = wait_for_block(store, block_root, block_wait).await else { + return ApiError::BadRequest("unknown block: the envelope's block is not held") + .into_response(); + }; + let containers::SignedBeaconBlock::Gloas(block) = block else { + return ApiError::BadRequest("the envelope's block is not a gloas block").into_response(); + }; + let block_slot = block.message.slot; + let bid = block.message.body.signed_execution_payload_bid.message; + if signed.message.builder_index != bid.builder_index + || signed.message.payload.block_hash != bid.block_hash + { + return ApiError::BadRequest("the envelope does not fulfill the block's bid") + .into_response(); + } + + let (blobs, cell_proofs) = match supplied { + Some(supplied) => supplied, + None => match cache.get(block_slot, block_root) { + Some(cached) => (cached.blobs, cached.cell_proofs), + None if bid.blob_kzg_commitments.is_empty() => (Vec::new(), Vec::new()), + None => { + return ApiError::BadRequest( + "no cached blobs for the block: send them with Eth-Blob-Data-Included: true", + ) + .into_response(); + } + }, + }; + if blobs.len() != bid.blob_kzg_commitments.len() { + return ApiError::BadRequest("the blobs do not match the bid's commitments") + .into_response(); + } + + // Signature and the columns' proofs are CPU-bound; off the runtime. + let commitments: Vec<_> = bid.blob_kzg_commitments.iter().copied().collect(); + let envelope_for_check = signed.clone(); + let checked = tokio::task::spawn_blocking(move || { + if !verify_execution_payload_envelope_signature(&post_state, &envelope_for_check) + .map_err(|_| ApiError::BadRequest("the envelope's signer is unknown"))? + { + return Err(ApiError::BadRequest("invalid envelope signature")); + } + let sidecars = if blobs.is_empty() { + Vec::new() + } else { + let sidecars = gloas_data_column_sidecars(block_root, block_slot, &blobs, &cell_proofs) + .map_err(|_| { + ApiError::BadRequest("the blobs and proofs do not form valid columns") + })?; + let verified = sidecars.iter().all(|sidecar| { + gloas_verify_data_column_sidecar(sidecar, &commitments) + && gloas_verify_data_column_sidecar_kzg_proofs(sidecar, &commitments) + .unwrap_or(false) + }); + if !verified { + return Err(ApiError::BadRequest( + "the data columns do not verify against the bid's commitments", + )); + } + sidecars + }; + Ok(sidecars) + }) + .await; + let sidecars = match checked { + Ok(Ok(sidecars)) => sidecars, + Ok(Err(err)) => return err.into_response(), + Err(_) => return ApiError::Internal("verifying the envelope failed").into_response(), + }; + + let sidecars: Vec = + sidecars.into_iter().map(DataColumnSidecar::Gloas).collect(); + if p2p + .publish_execution_payload_envelope(Box::new(signed), sidecars) + .is_err() + { + return ApiError::Internal("the network actor is not running").into_response(); + } + StatusCode::OK.into_response() +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::test_utils::{RecordingNetwork, beacon_store_with_config}; + use axum::{body::Body, http::Request}; + use ethlambda_state_transition::beacon::gloas_block_production::test_support::{ + config, post_state, produce as produce_block, sign_block, sign_envelope, state_to_build_on, + }; + use ethlambda_state_transition::beacon::helpers::test_state::with_signing_validators_at; + use http_body_util::BodyExt as _; + use tower::ServiceExt as _; + + async fn respond(app: Router, request: Request) -> (StatusCode, serde_json::Value) { + let response = app.oneshot(request).await.unwrap(); + let status = response.status(); + let body = response.into_body().collect().await.unwrap().to_bytes(); + (status, serde_json::from_slice(&body).unwrap_or_default()) + } + + /// A store holding a fulu head, under a schedule that puts every slot at + /// gloas: the head is behind the fork, as on a node running across it. + fn gloas_scheduled_store() -> (Store, Slot) { + let state = with_signing_validators_at(ForkName::Fulu, 64); + let head_slot = state.slot(); + let (store, _root) = beacon_store_with_config(state, config()); + (store, head_slot) + } + + fn produce_app(store: Store, engine: Option) -> Router { + routes() + .with_state(store) + .layer(Extension(engine)) + .layer(Extension(SharedAttestationPool::default())) + .layer(Extension(SharedPayloadAttestationPool::default())) + .layer(Extension(FeeRecipients::default())) + .layer(Extension(NodeCustodyColumns::default())) + } + + fn produce_request(slot: Slot, query: &str, body: &str) -> Request { + Request::post(format!( + "/eth/v4/validator/blocks/{slot}?randao_reveal=0x{}{query}", + "00".repeat(96) + )) + .header("eth-consensus-version", "gloas") + .header(header::CONTENT_TYPE, "application/json") + .body(Body::from(body.to_string())) + .unwrap() + } + + #[tokio::test] + async fn an_envelope_over_the_default_body_limit_is_not_a_413() { + let (store, _) = gloas_scheduled_store(); + let network: RpcToP2PRef = std::sync::Arc::new(RecordingNetwork::default()); + let app = routes().with_state(store).layer(Extension(network)); + let request = Request::post("/eth/v1/beacon/execution_payload_envelopes") + .header("eth-consensus-version", "gloas") + .header("eth-blob-data-included", "true") + .header(header::CONTENT_TYPE, crate::SSZ_CONTENT_TYPE) + .body(Body::from(vec![0u8; 3 * 1024 * 1024])) + .unwrap(); + let (status, _) = respond(app, request).await; + assert_ne!(status, StatusCode::PAYLOAD_TOO_LARGE); + } + + const EMPTY_CONFIG: &str = r#"{"min_bid":"0","builder_boost_factor":"0","builders":[]}"#; + + #[tokio::test] + async fn producing_a_pre_gloas_slot_is_a_400_even_with_no_execution_client() { + let state = with_signing_validators_at(ForkName::Fulu, 64); + let head_slot = state.slot(); + // The default schedule never reaches gloas. + let (store, _root) = beacon_store_with_config(state, Config::mainnet()); + let app = produce_app(store, None); + let (status, json) = respond( + app, + produce_request(head_slot + 1, "&include_payload=true", EMPTY_CONFIG), + ) + .await; + assert_eq!(status, StatusCode::BAD_REQUEST); + assert_eq!(json["message"], "this endpoint serves gloas slots only"); + } + + #[tokio::test] + async fn producing_without_an_execution_client_is_a_503() { + let (store, head_slot) = gloas_scheduled_store(); + let app = produce_app(store, None); + let (status, _) = respond( + app, + produce_request(head_slot + 1, "&include_payload=true", EMPTY_CONFIG), + ) + .await; + assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE); + } + + #[tokio::test] + async fn a_missing_include_payload_or_an_undecodable_body_is_a_400() { + let (store, head_slot) = gloas_scheduled_store(); + let (status, _) = respond( + produce_app(store.clone(), None), + produce_request(head_slot + 1, "", EMPTY_CONFIG), + ) + .await; + assert_eq!(status, StatusCode::BAD_REQUEST); + + let (status, json) = respond( + produce_app(store.clone(), None), + produce_request(head_slot + 1, "&include_payload=true", "not a config"), + ) + .await; + assert_eq!(status, StatusCode::BAD_REQUEST); + assert_eq!(json["message"], "the body is not a BuilderConfig"); + + // A wrong fork header is refused too. + let request = Request::post(format!( + "/eth/v4/validator/blocks/{}?randao_reveal=0x{}&include_payload=true", + head_slot + 1, + "00".repeat(96) + )) + .header("eth-consensus-version", "fulu") + .body(Body::from(EMPTY_CONFIG)) + .unwrap(); + let (status, _) = respond(produce_app(store, None), request).await; + assert_eq!(status, StatusCode::BAD_REQUEST); + } + + #[test] + fn a_builder_config_decodes_as_json_or_ssz() { + let mut headers = HeaderMap::new(); + assert_eq!( + decode_builder_config(&headers, EMPTY_CONFIG.as_bytes()).unwrap(), + 0 + ); + assert!(decode_builder_config(&headers, b"{}").is_err()); + + headers.insert( + header::CONTENT_TYPE, + HeaderValue::from_static(crate::SSZ_CONTENT_TYPE), + ); + // min_bid, boost factor, then the offset of an empty builders list. + let mut ssz = vec![0u8; 16]; + ssz.extend_from_slice(&20u32.to_le_bytes()); + assert_eq!(decode_builder_config(&headers, &ssz).unwrap(), 0); + assert!(decode_builder_config(&headers, &ssz[..19]).is_err()); + } + + #[test] + fn the_cache_keeps_the_current_and_previous_slot_only() { + let cache = PayloadCache::default(); + let state = state_to_build_on(); + let produced = produce_block(&state, true, Vec::new()).unwrap(); + let entry = CachedPayload { + envelope: produced.envelope, + blobs: Vec::new(), + cell_proofs: Vec::new(), + }; + for slot in [10, 11, 12] { + cache.insert(slot, H256::repeat_byte(slot as u8), entry.clone()); + } + assert!(cache.get(10, H256::repeat_byte(10)).is_none()); + assert!(cache.get(11, H256::repeat_byte(11)).is_some()); + assert!(cache.get(12, H256::repeat_byte(12)).is_some()); + // The root is part of the key: a re-orged block gets no envelope. + assert!(cache.get(12, H256::repeat_byte(99)).is_none()); + } + + /// A store holding the built block and its post-state (the way the chain + /// actor stores an import), with the gloas schedule. + fn store_with_block( + state: &BeaconState, + produced: &GloasProduced, + signature: BlsSignature, + ) -> (Store, H256, BeaconState) { + let (mut store, _anchor) = beacon_store_with_config(state.clone(), config()); + let root = produced.block.hash_tree_root(); + let post = post_state(state, &produced.block); + store + .insert_signed_block( + root, + containers::SignedBeaconBlock::Gloas(containers::gloas::SignedBeaconBlock { + message: produced.block.clone(), + signature, + }), + ) + .unwrap(); + store.insert_state(root, post.clone()).unwrap(); + (store, root, post) + } + + fn envelope_request(included: Option<&str>, body: Vec) -> (HeaderMap, Vec) { + let mut headers = HeaderMap::new(); + headers.insert("eth-consensus-version", HeaderValue::from_static("gloas")); + headers.insert( + header::CONTENT_TYPE, + HeaderValue::from_static(crate::SSZ_CONTENT_TYPE), + ); + if let Some(included) = included { + headers.insert( + "eth-blob-data-included", + HeaderValue::from_str(included).unwrap(), + ); + } + (headers, body) + } + + /// A produced block with no blobs, so the envelope needs no columns. + fn produced_without_blobs(state: &BeaconState) -> GloasProduced { + use ethlambda_state_transition::beacon::gloas_block_production::{ + GloasBlockInputs, assemble_gloas_block, gloas_payload_inputs, + test_support::{payload_for, randao_reveal}, + }; + let requests = ExecutionRequests::default(); + let inputs = gloas_payload_inputs(state, true, &requests, &config()).unwrap(); + assemble_gloas_block( + state, + GloasBlockInputs { + randao_reveal: randao_reveal(state), + graffiti: Bytes32::ZERO, + attestations: Vec::new(), + payload_attestations: Vec::new(), + parent_execution_requests: requests, + execution_payload: payload_for(&inputs), + blob_kzg_commitments: Vec::new(), + execution_requests: ExecutionRequests::default(), + }, + &config(), + ) + .unwrap() + } + + fn signed_envelope( + post: &BeaconState, + produced: &GloasProduced, + ) -> SignedExecutionPayloadEnvelope { + SignedExecutionPayloadEnvelope { + message: produced.envelope.clone(), + signature: sign_envelope(post, produced.block.proposer_index, &produced.envelope), + } + } + + #[tokio::test] + async fn a_signed_envelope_for_a_held_block_is_gossiped() { + let state = state_to_build_on(); + let produced = produced_without_blobs(&state); + let signature = sign_block(&state, &produced.block); + let (store, _root, post) = store_with_block(&state, &produced, signature); + let signed = signed_envelope(&post, &produced); + + let network = Arc::new(RecordingNetwork::default()); + let p2p: RpcToP2PRef = network.clone(); + let (headers, body) = envelope_request(Some("false"), signed.to_ssz()); + let response = post_envelope_waiting( + &store, + &p2p, + &PayloadCache::default(), + &headers, + &body, + Duration::from_millis(100), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let published = network.envelopes.lock().unwrap(); + assert_eq!(published.len(), 1); + assert_eq!(published[0].0, signed); + assert!(published[0].1.is_empty()); + } + + #[tokio::test] + async fn an_envelope_for_a_block_that_arrives_late_waits_for_it() { + let state = state_to_build_on(); + let produced = produced_without_blobs(&state); + let signature = sign_block(&state, &produced.block); + let (store, root, post) = store_with_block(&state, &produced, signature); + let signed = signed_envelope(&post, &produced); + // The block is imported only after the request is already waiting: + // an empty store to start with, filled in by a second task. + let (empty_store, _anchor) = beacon_store_with_config(state.clone(), config()); + let mut late_store = empty_store.clone(); + let block = store.get_signed_block(&root).unwrap().unwrap(); + let importer = tokio::spawn(async move { + tokio::time::sleep(Duration::from_millis(200)).await; + late_store.insert_signed_block(root, block).unwrap(); + late_store.insert_state(root, post).unwrap(); + }); + + let network = Arc::new(RecordingNetwork::default()); + let p2p: RpcToP2PRef = network.clone(); + let (headers, body) = envelope_request(Some("false"), signed.to_ssz()); + let response = post_envelope_waiting( + &empty_store, + &p2p, + &PayloadCache::default(), + &headers, + &body, + Duration::from_secs(5), + ) + .await; + importer.await.unwrap(); + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(network.envelopes.lock().unwrap().len(), 1); + } + + #[tokio::test] + async fn an_envelope_for_an_unknown_block_is_a_400_after_the_wait() { + let state = state_to_build_on(); + let produced = produced_without_blobs(&state); + let post = post_state(&state, &produced.block); + let signed = signed_envelope(&post, &produced); + let (store, _anchor) = beacon_store_with_config(state, config()); + + let network = Arc::new(RecordingNetwork::default()); + let p2p: RpcToP2PRef = network.clone(); + let (headers, body) = envelope_request(Some("false"), signed.to_ssz()); + let started = Instant::now(); + let response = post_envelope_waiting( + &store, + &p2p, + &PayloadCache::default(), + &headers, + &body, + Duration::from_millis(150), + ) + .await; + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + assert!(started.elapsed() >= Duration::from_millis(150)); + assert!(network.envelopes.lock().unwrap().is_empty()); + } + + #[tokio::test] + async fn a_bad_signature_a_wrong_bid_and_a_wrong_media_type_are_refused() { + let state = state_to_build_on(); + let produced = produced_without_blobs(&state); + let signature = sign_block(&state, &produced.block); + let (store, _root, post) = store_with_block(&state, &produced, signature); + let network = Arc::new(RecordingNetwork::default()); + let p2p: RpcToP2PRef = network.clone(); + let cache = PayloadCache::default(); + let wait = Duration::from_millis(100); + + // Signed by someone else. + let mut forged = signed_envelope(&post, &produced); + forged.signature = + sign_envelope(&post, produced.block.proposer_index + 1, &produced.envelope); + let (headers, body) = envelope_request(Some("false"), forged.to_ssz()); + let response = post_envelope_waiting(&store, &p2p, &cache, &headers, &body, wait).await; + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + // A payload other than the one the bid commits to. + let mut other = signed_envelope(&post, &produced); + other.message.payload.block_hash = H256::repeat_byte(0x77); + let (headers, body) = envelope_request(Some("false"), other.to_ssz()); + let response = post_envelope_waiting(&store, &p2p, &cache, &headers, &body, wait).await; + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + // JSON is not accepted. + let signed = signed_envelope(&post, &produced); + let (mut headers, body) = envelope_request(Some("false"), signed.to_ssz()); + headers.insert( + header::CONTENT_TYPE, + HeaderValue::from_static("application/json"), + ); + let response = post_envelope_waiting(&store, &p2p, &cache, &headers, &body, wait).await; + assert_eq!(response.status(), StatusCode::UNSUPPORTED_MEDIA_TYPE); + + // The blob-data header is required. + let (headers, body) = envelope_request(None, signed.to_ssz()); + let response = post_envelope_waiting(&store, &p2p, &cache, &headers, &body, wait).await; + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + assert!(network.envelopes.lock().unwrap().is_empty()); + } + + #[tokio::test] + async fn the_cached_envelope_is_served_for_its_slot_and_root_only() { + let state = state_to_build_on(); + let produced = produced_without_blobs(&state); + let root = produced.block.hash_tree_root(); + let slot = produced.block.slot; + let (store, _anchor) = beacon_store_with_config(state, config()); + let app = routes().with_state(store); + // The route layer's own cache cannot be reached from here, so the + // handler is exercised directly through a router with a seeded one. + let cache = PayloadCache::default(); + cache.insert( + slot, + root, + CachedPayload { + envelope: produced.envelope.clone(), + blobs: Vec::new(), + cell_proofs: Vec::new(), + }, + ); + let seeded = Router::new() + .route( + "/eth/v1/validator/execution_payload_envelopes/{slot}/{beacon_block_root}", + get(get_envelope), + ) + .layer(Extension(cache)); + let uri = format!( + "/eth/v1/validator/execution_payload_envelopes/{slot}/0x{}", + hex::encode(root.0) + ); + let (status, json) = respond( + seeded.clone(), + Request::get(&uri).body(Body::empty()).unwrap(), + ) + .await; + assert_eq!(status, StatusCode::OK); + assert_eq!(json["version"], "gloas"); + assert_eq!( + json["data"]["builder_index"], + constants::BUILDER_INDEX_SELF_BUILD.to_string() + ); + + let ssz_request = Request::get(&uri) + .header(header::ACCEPT, "application/octet-stream") + .body(Body::empty()) + .unwrap(); + let response = seeded.clone().oneshot(ssz_request).await.unwrap(); + let bytes = response.into_body().collect().await.unwrap().to_bytes(); + assert_eq!(bytes.as_ref(), produced.envelope.to_ssz().as_slice()); + + let other = format!( + "/eth/v1/validator/execution_payload_envelopes/{slot}/0x{}", + "11".repeat(32) + ); + let (status, _) = respond(seeded, Request::get(other).body(Body::empty()).unwrap()).await; + assert_eq!(status, StatusCode::NOT_FOUND); + // The unseeded production router answers 404 too. + let (status, _) = respond(app, Request::get(&uri).body(Body::empty()).unwrap()).await; + assert_eq!(status, StatusCode::NOT_FOUND); + } + + /// A block carrying one real blob, its signed envelope, and the body the + /// stateless form of the publish endpoint takes. + struct WithBlob { + state: BeaconState, + produced: GloasProduced, + blob: Vec, + proofs: Vec, + } + + fn with_blob() -> WithBlob { + use ethlambda_state_transition::beacon::{ + gloas_block_production::test_support::{payload_for, randao_reveal}, + kzg::{blob_to_kzg_commitment, compute_cells_and_kzg_proofs}, + }; + let state = state_to_build_on(); + let mut blob = vec![0u8; ethlambda_types::beacon::preset::BYTES_PER_BLOB]; + for (i, element) in blob.chunks_mut(32).enumerate() { + element[31] = (i % 100) as u8; + } + let commitment = blob_to_kzg_commitment(&blob).unwrap(); + let (_, proofs) = compute_cells_and_kzg_proofs(&blob).unwrap(); + let requests = ExecutionRequests::default(); + let inputs = gloas_payload_inputs(&state, true, &requests, &config()).unwrap(); + let produced = assemble_gloas_block( + &state, + GloasBlockInputs { + randao_reveal: randao_reveal(&state), + graffiti: Bytes32::ZERO, + attestations: Vec::new(), + payload_attestations: Vec::new(), + parent_execution_requests: requests, + execution_payload: payload_for(&inputs), + blob_kzg_commitments: vec![commitment], + execution_requests: ExecutionRequests::default(), + }, + &config(), + ) + .unwrap(); + WithBlob { + state, + produced, + blob, + proofs: proofs.to_vec(), + } + } + + #[tokio::test] + async fn an_envelope_with_blobs_gossips_every_column_and_a_tampered_blob_is_refused() { + let WithBlob { + state, + produced, + blob, + proofs, + } = with_blob(); + let signature = sign_block(&state, &produced.block); + let (store, root, post) = store_with_block(&state, &produced, signature); + let signed = signed_envelope(&post, &produced); + let wait = Duration::from_millis(100); + let contents = |blob: &[u8]| GloasSignedEnvelopeContents { + signed_execution_payload_envelope: signed.clone(), + kzg_proofs: CellKzgProofs::try_from(proofs.clone()).unwrap(), + blobs: blobs_list(vec![blob.to_vec()]).unwrap(), + }; + + // Stateless: the blobs travel with the envelope. + let network = Arc::new(RecordingNetwork::default()); + let p2p: RpcToP2PRef = network.clone(); + let (headers, body) = envelope_request(Some("true"), contents(&blob).to_ssz()); + let response = post_envelope_waiting( + &store, + &p2p, + &PayloadCache::default(), + &headers, + &body, + wait, + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + { + let published = network.envelopes.lock().unwrap(); + assert_eq!(published.len(), 1); + assert_eq!( + published[0].1.len(), + ethlambda_types::beacon::preset::NUMBER_OF_COLUMNS + ); + } + + // A different blob under the same proofs does not verify. + let mut tampered = blob.clone(); + tampered[63] ^= 1; + let rejected = Arc::new(RecordingNetwork::default()); + let p2p: RpcToP2PRef = rejected.clone(); + let (headers, body) = envelope_request(Some("true"), contents(&tampered).to_ssz()); + let response = post_envelope_waiting( + &store, + &p2p, + &PayloadCache::default(), + &headers, + &body, + wait, + ) + .await; + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + // Stateful: nothing cached means a 400, and the cached blobs are used when present. + let (headers, body) = envelope_request(Some("false"), signed.to_ssz()); + let response = post_envelope_waiting( + &store, + &p2p, + &PayloadCache::default(), + &headers, + &body, + wait, + ) + .await; + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + let cache = PayloadCache::default(); + cache.insert( + produced.block.slot, + root, + CachedPayload { + envelope: produced.envelope.clone(), + blobs: vec![blob], + cell_proofs: proofs, + }, + ); + let response = post_envelope_waiting(&store, &p2p, &cache, &headers, &body, wait).await; + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(rejected.envelopes.lock().unwrap().len(), 1); + } +} diff --git a/crates/net/rpc/src/beacon/mod.rs b/crates/net/rpc/src/beacon/mod.rs index c047aa29..3b94fbb1 100644 --- a/crates/net/rpc/src/beacon/mod.rs +++ b/crates/net/rpc/src/beacon/mod.rs @@ -11,7 +11,6 @@ use axum::{ response::{IntoResponse, Response}, }; use ethlambda_storage::Store; -use ethlambda_types::beacon::{config::Config, fork::ForkName, primitives::Epoch}; use serde::Serialize; use crate::shared::block_id::IdError; @@ -20,15 +19,29 @@ pub(crate) mod blocks; pub(crate) mod config; pub(crate) mod envelopes; pub(crate) mod genesis; +pub(crate) mod gloas_proposal; pub(crate) mod headers; pub(crate) mod node; pub(crate) mod pool; pub(crate) mod proposal; +pub(crate) mod ptc; pub(crate) mod states; pub(crate) mod validator; #[cfg(test)] mod validator_client_tests; +/// Request-body cap for the routes that accept blocks, envelopes and blobs. +/// +/// Axum's default is 2 MiB, which a fulu `SignedBlockContents` outgrows at +/// about 14 blobs (each blob is 131072 bytes, plus 128 cell proofs of 48 bytes, +/// 6144 bytes, per blob). The preset's `MAX_BLOB_COMMITMENTS_PER_BLOCK` (4096, +/// roughly 550 MB) is no practical bound, so this is sized from what the blob +/// schedule allows in practice instead: a generous 128 blobs and their cell +/// proofs come to about 17.5 MiB, a maximal execution payload (the gloas +/// envelope carries it whole) takes up to roughly 16 MiB more, and the rest is +/// margin for the block and its operations. Applied per route, never router-wide. +pub(crate) const MAX_PUBLISH_BODY_BYTES: usize = 64 * 1024 * 1024; + /// The wrapper every fork-versioned Beacon API payload travels in. /// /// The fork travels here rather than inside `data` because the containers are @@ -61,36 +74,6 @@ pub(crate) enum ApiError { ServiceUnavailable(&'static str), } -/// Refuses `epoch` when the node's schedule places it at gloas: validator -/// duties are not supported from that fork. -/// -/// The node follows gloas but only as a follower. It has no builder bid -/// handling, no payload-timeliness committee duty and no gloas proposer -/// flow, and its attestation pool holds electra-shaped votes. An answer for a -/// gloas epoch would carry fulu's shapes under gloas's fork (a vote a -/// validator client would sign as valid, an aggregate of the wrong shape), so -/// it is refused by name, as the submit endpoints refuse the gloas header. -/// Deliberately not [`ethlambda_types::beacon::fork::ForkName::is_followed`]: -/// following a fork and serving its validator duties are different questions. -pub(crate) fn refuse_validator_duties_from_gloas( - config: &Config, - epoch: Epoch, - message: &'static str, -) -> Result<(), ApiError> { - match config.fork_at_epoch(epoch) { - ForkName::Phase0 - | ForkName::Altair - | ForkName::Bellatrix - | ForkName::Capella - | ForkName::Deneb - | ForkName::Electra - | ForkName::Fulu => Ok(()), - // `Config::fork_at_epoch` never returns Lean; refused like gloas - // rather than answered. - ForkName::Gloas | ForkName::Lean => Err(ApiError::BadRequest(message)), - } -} - impl From for ApiError { fn from(err: IdError) -> Self { match err { @@ -133,6 +116,8 @@ pub(crate) fn routes(version: &'static str, peer_id: String) -> Router { .merge(validator::routes()) .merge(pool::routes()) .merge(proposal::routes()) + .merge(gloas_proposal::routes()) + .merge(ptc::routes()) } #[cfg(test)] @@ -152,18 +137,6 @@ mod tests { assert_eq!(json["message"], "block not found"); } - #[test] - fn validator_duties_are_refused_from_the_gloas_epoch_even_though_gloas_is_followed() { - let mut config = Config::mainnet(); - config.gloas_fork_epoch = 10; - assert!(ForkName::Gloas.is_followed()); - let refuse = |epoch| refuse_validator_duties_from_gloas(&config, epoch, "refused"); - assert!(refuse(0).is_ok()); - assert!(refuse(9).is_ok()); - assert!(matches!(refuse(10), Err(ApiError::BadRequest("refused")))); - assert!(matches!(refuse(11), Err(ApiError::BadRequest("refused")))); - } - #[test] fn a_malformed_id_is_a_400_and_an_absent_one_a_404() { assert_eq!( diff --git a/crates/net/rpc/src/beacon/pool.rs b/crates/net/rpc/src/beacon/pool.rs index 9ce9ecb0..88ad1bab 100644 --- a/crates/net/rpc/src/beacon/pool.rs +++ b/crates/net/rpc/src/beacon/pool.rs @@ -4,7 +4,8 @@ //! `POST /eth/v2/validator/aggregate_and_proofs`, how it publishes the result. //! //! Each attestation is checked against the electra `beacon_attestation_{subnet_id}` -//! gossip conditions (p2p-interface) that can be evaluated here, then +//! gossip conditions (p2p-interface) that can be evaluated here (gloas's +//! payload-status rule on `data.index` too, at a gloas slot), then //! published on its subnet. Validating before publishing is not optional: a //! peer that relays invalid attestations has its gossipsub score cut, and //! enough of that disconnects it. @@ -29,7 +30,7 @@ use ethlambda_state_transition::beacon::{ attestation_pool::SharedAttestationPool, bls, gossip::attestation::compute_subnet_for_attestation, - gossip::{Outcome, aggregate}, + gossip::{IgnoreReason, Outcome, RejectReason, aggregate, verify_attestation_payload_status}, helpers::accessors::{CommitteeCacheExt as _, get_domain}, }; use ethlambda_storage::Store; @@ -39,6 +40,7 @@ use ethlambda_types::{ containers::{ BeaconState, SignedAggregateAndProof, electra::{self, SingleAttestation}, + gloas, }, fork::ForkName, primitives::{CommitteeIndex, Epoch, Root, Slot}, @@ -49,7 +51,7 @@ use ethlambda_types::{ use serde::{Deserialize, Serialize}; use tracing::{debug, warn}; -use crate::beacon::{ApiError, refuse_validator_duties_from_gloas, validator::head}; +use crate::beacon::{ApiError, validator::head}; pub(crate) fn routes() -> Router { Router::new() @@ -90,9 +92,10 @@ async fn post_pool_attestations( headers: HeaderMap, body: Bytes, ) -> Response { - if let Err(err) = require_electra_or_fulu(&headers) { - return err.into_response(); - } + let header_fork = match submission_fork(&headers) { + Ok(fork) => fork, + Err(err) => return err.into_response(), + }; let Ok(attestations) = serde_json::from_slice::>(&body) else { return ApiError::BadRequest("invalid request body").into_response(); }; @@ -109,7 +112,11 @@ async fn post_pool_attestations( for (index, attestation) in attestations.into_iter().enumerate() { let slot = attestation.data.slot; let validator = attestation.attester_index; - let checked = validate(&store, &state, &attestation, now_ms); + let checked = if header_matches_slot(&store, header_fork, slot) { + validate(&store, &state, &attestation, now_ms) + } else { + Err("Eth-Consensus-Version does not match the attestation slot's fork") + }; // Pooled as well as published: gossip never delivers a node its own // messages, so without this an aggregator served by this node would // be missing its own validator client's votes. @@ -172,8 +179,14 @@ fn validate( return Err("attestation is older than the previous epoch"); } - // [REJECT] data.index == 0, and the target epoch is the slot's. - if data.index != 0 { + // [REJECT] data.index == 0 (electra), or at a gloas slot 0 or 1, the + // payload-present flag; the target epoch is the slot's. + let at_gloas = config.fork_at_epoch(attestation_epoch) == ForkName::Gloas; + if at_gloas { + if data.index > 1 { + return Err("data.index must be 0 or 1 from gloas on"); + } + } else if data.index != 0 { return Err("data.index must be zero from electra on"); } if data.target.epoch != attestation_epoch { @@ -195,6 +208,11 @@ fn validate( { return Err("target root is not the voted block's checkpoint"); } + // [New in Gloas:EIP7732] The payload flag agrees with what this node knows + // of the voted block's payload. + if at_gloas { + verify_attestation_payload_status(store, data).map_err(payload_status_failure)?; + } // [REJECT] The committee index is in range, and the attester is in it. let epoch_committees = store.committee_cache().committees(state, data.target.epoch); @@ -235,22 +253,35 @@ fn validate( }) } -/// The endpoints here take electra's containers, which exist from that fork -/// on and which fulu keeps; `Eth-Consensus-Version` is required to say so. -/// Gloas changed the attestation (a progressive list, and `data.index` as the -/// payload-availability signal) and the pool rules that go with it, neither of -/// which this node models, so it is refused by name rather than read as -/// electra. -fn require_electra_or_fulu(headers: &HeaderMap) -> Result<(), ApiError> { +/// Why a gloas vote failed [`verify_attestation_payload_status`]. The gossip +/// rule's `IGNORE`s are failures here too: a vote this node would not +/// propagate must not be published on a validator client's say-so. +fn payload_status_failure(outcome: Outcome) -> &'static str { + match outcome { + Outcome::Reject(RejectReason::SameSlotPayloadFlag) => { + "a same-slot attestation cannot vote for the payload" + } + Outcome::Ignore(IgnoreReason::PayloadEnvelopeUnseen) => { + "the voted block's payload has not been seen and verified" + } + Outcome::Ignore(IgnoreReason::PayloadOptimistic) => { + "the voted block's payload is optimistic" + } + _ => "the voted block's payload status could not be checked", + } +} + +/// The fork a submit endpoint's `Eth-Consensus-Version` names. The endpoints +/// take electra's containers, which fulu keeps, and gloas's, which are +/// electra's own for a `SingleAttestation` and a new shape for an aggregate; +/// anything earlier is refused. +fn submission_fork(headers: &HeaderMap) -> Result { let fork = headers .get("eth-consensus-version") .and_then(|value| value.to_str().ok()) .and_then(ForkName::parse); match fork { - Some(ForkName::Electra | ForkName::Fulu) => Ok(()), - Some(ForkName::Gloas) => Err(ApiError::BadRequest( - "Eth-Consensus-Version gloas is not supported", - )), + Some(fork @ (ForkName::Electra | ForkName::Fulu | ForkName::Gloas)) => Ok(fork), // `ForkName::parse` never returns Lean: it is absent from `ALL`. Some( ForkName::Phase0 @@ -261,11 +292,20 @@ fn require_electra_or_fulu(headers: &HeaderMap) -> Result<(), ApiError> { | ForkName::Lean, ) | None => Err(ApiError::BadRequest( - "Eth-Consensus-Version must name electra or fulu", + "Eth-Consensus-Version must name electra, fulu or gloas", )), } } +/// Whether `header` is a version the fork at `slot` accepts: gloas's slots need +/// the gloas header, and electra's and fulu's, whose containers are the same, +/// accept either of theirs. +fn header_matches_slot(store: &Store, header: ForkName, slot: Slot) -> bool { + let slot_is_gloas = + store.config().fork_at_epoch(compute_epoch_at_slot(slot)) == ForkName::Gloas; + slot_is_gloas == (header == ForkName::Gloas) +} + /// `200` when nothing failed, else the Beacon API's `IndexedErrorMessage` /// naming each failed item by position; the rest were still published. fn batch_response(failures: Vec, message: &'static str) -> Response { @@ -295,11 +335,27 @@ async fn post_aggregate_and_proofs( headers: HeaderMap, body: Bytes, ) -> Response { - if let Err(err) = require_electra_or_fulu(&headers) { - return err.into_response(); - } - let Ok(aggregates) = serde_json::from_slice::>(&body) - else { + let header_fork = match submission_fork(&headers) { + Ok(fork) => fork, + Err(err) => return err.into_response(), + }; + // Gloas's aggregate is its own container, so the header picks the decoder. + let aggregates = if header_fork == ForkName::Gloas { + serde_json::from_slice::>(&body).map(|signed| { + signed + .into_iter() + .map(SignedAggregateAndProof::Gloas) + .collect() + }) + } else { + serde_json::from_slice::>(&body).map(|signed| { + signed + .into_iter() + .map(SignedAggregateAndProof::Electra) + .collect() + }) + }; + let Ok(aggregates): Result, _> = aggregates else { return ApiError::BadRequest("invalid request body").into_response(); }; @@ -310,27 +366,38 @@ async fn post_aggregate_and_proofs( let capacity = std::num::NonZeroUsize::MIN; let mut failures = Vec::new(); for (index, aggregate) in aggregates.into_iter().enumerate() { - let inner = aggregate.message.aggregate.clone(); - let aggregate = SignedAggregateAndProof::Electra(aggregate); let slot = aggregate.slot(); let aggregator = aggregate.aggregator_index(); let seen = aggregate::SeenAggregates::new(capacity, capacity); - let checked = aggregate::cheap_checks(&seen, &store, &aggregate, now_ms) - .and_then(|()| aggregate::stateful_checks(&store, &aggregate).map(|_| ())); - let published = checked - .map_err(|outcome: Outcome| { - warn!(%slot, aggregator, ?outcome, "Refused a submitted aggregate"); - "aggregate failed validation" - }) - .and_then(|()| { - // Recorded for block production, which packs the aggregates - // this node has validated. + let checked = if header_matches_slot(&store, header_fork, slot) { + aggregate::cheap_checks(&seen, &store, &aggregate, now_ms) + .and_then(|()| aggregate::stateful_checks(&store, &aggregate).map(|_| ())) + .map_err(|outcome: Outcome| { + warn!(%slot, aggregator, ?outcome, "Refused a submitted aggregate"); + "aggregate failed validation" + }) + } else { + Err("Eth-Consensus-Version does not match the aggregate slot's fork") + }; + let published = checked.and_then(|()| { + // Recorded for block production, which packs the aggregates this + // node has validated. The pool holds electra's shape, which a + // gloas aggregate converts to. + let pooled = match &aggregate { + SignedAggregateAndProof::Electra(signed) => Some(signed.message.aggregate.clone()), + SignedAggregateAndProof::Gloas(signed) => { + electra::Attestation::try_from(&signed.message.aggregate).ok() + } + SignedAggregateAndProof::Phase0(_) => None, + }; + if let Some(pooled) = pooled { pool.lock() .expect("attestation pool lock poisoned") - .insert_aggregate(inner); - p2p.publish_beacon_aggregate(aggregate) - .map_err(|_| "the network actor is not running") - }); + .insert_aggregate(pooled); + } + p2p.publish_beacon_aggregate(aggregate) + .map_err(|_| "the network actor is not running") + }); match published { Ok(()) => debug!(%slot, aggregator, "Accepted aggregate for gossip"), Err(message) => failures.push(Failure { index, message }), @@ -361,16 +428,6 @@ async fn get_aggregate_attestation( let fork = store .config() .fork_at_epoch(compute_epoch_at_slot(query.slot)); - // The pool holds electra-shaped votes, and the response is labelled with - // the slot's fork, so a gloas slot would be served as gloas with a body of - // the wrong shape. - if let Err(err) = refuse_validator_duties_from_gloas( - &store.config(), - compute_epoch_at_slot(query.slot), - "gloas aggregates are not supported", - ) { - return err.into_response(); - } let aggregate = pool .lock() .expect("attestation pool lock poisoned") @@ -382,9 +439,16 @@ async fn get_aggregate_attestation( let Some(aggregate) = aggregate else { return ApiError::NotFound("no matching attestations to aggregate").into_response(); }; + // The pool holds electra-shaped votes; a gloas slot is served the same + // vote in gloas's container, whose JSON is the same shape. + let data = if fork == ForkName::Gloas { + serde_json::json!(gloas::Attestation::from(&aggregate)) + } else { + serde_json::json!(aggregate) + }; let response = crate::json_response(serde_json::json!({ "version": fork.as_str(), - "data": aggregate, + "data": data, })); crate::shared::content::with_consensus_version(response, fork) } @@ -408,7 +472,7 @@ mod tests { use std::sync::Arc; use super::*; - use crate::test_utils::{RecordingNetwork, beacon_store_at}; + use crate::test_utils::RecordingNetwork; use axum::{body::Body, http::Request}; use ethlambda_state_transition::beacon::helpers::{ accessors::get_beacon_committee, @@ -425,27 +489,47 @@ mod tests { head_root: Root, network: Arc, pool: SharedAttestationPool, + /// The `Eth-Consensus-Version` the fixture's fork submits under. + version: &'static str, } /// A fulu head state, stored in the current wall-clock epoch so the /// submitted attestations are neither future nor stale. fn fixture() -> Fixture { - let mut state = with_signing_validators_at(ForkName::Fulu, 64); - let BeaconState::Fulu(fulu) = &mut state else { - unreachable!("built as fulu") + fixture_at(ForkName::Fulu, 0) + } + + /// A head of `fork`'s state (fulu, or gloas under a schedule with gloas + /// from epoch 0) `slots_back` slots before the first slot of the wall + /// clock's epoch. At zero the head is its own epoch's checkpoint block and + /// the target root of a vote at its slot; further back, a vote in the + /// wall-clock epoch's first slot is for an earlier block. + fn fixture_at(fork: ForkName, slots_back: u64) -> Fixture { + let mut state = with_signing_validators_at(fork, 64); + let config = match fork { + ForkName::Gloas => Config::mainnet().with_fork_epoch(ForkName::Gloas, 0), + _ => Config::mainnet(), }; - // At the first slot of the wall clock's epoch, so the head is its own - // epoch's checkpoint block and the attestation's target root. - let (probe, _) = beacon_store_at(BeaconState::Fulu(fulu.clone())); + let (probe, _) = crate::test_utils::beacon_store_with_config(state.clone(), config.clone()); let wall_epoch = compute_epoch_at_slot(crate::beacon::node::wall_slot(&probe)); - fulu.slot = compute_start_slot_at_epoch(wall_epoch); - let (store, head_root) = beacon_store_at(state.clone()); + let head_slot = compute_start_slot_at_epoch(wall_epoch) - slots_back; + match &mut state { + BeaconState::Fulu(fulu) => fulu.slot = head_slot, + BeaconState::Gloas(gloas) => gloas.slot = head_slot, + _ => unreachable!("built as fulu or gloas"), + } + let (store, head_root) = crate::test_utils::beacon_store_with_config(state.clone(), config); Fixture { store, state, head_root, network: Arc::new(RecordingNetwork::default()), pool: SharedAttestationPool::default(), + version: if fork == ForkName::Gloas { + "gloas" + } else { + "fulu" + }, } } @@ -453,15 +537,35 @@ mod tests { /// voting for the head at the head's own slot. fn attestation(fixture: &Fixture, committee_index: u64, position: usize) -> SingleAttestation { let slot = fixture.state.slot(); + attestation_for( + fixture, + slot, + (fixture.head_root, fixture.head_root), + 0, + committee_index, + position, + ) + } + + /// A correctly signed vote at `slot` for the block and target roots in + /// `(block, target)`, carrying `index` as its `data.index`. + fn attestation_for( + fixture: &Fixture, + slot: u64, + (block, target): (Root, Root), + index: u64, + committee_index: u64, + position: usize, + ) -> SingleAttestation { let epoch = compute_epoch_at_slot(slot); let data = AttestationData { slot, - index: 0, - beacon_block_root: fixture.head_root, + index, + beacon_block_root: block, source: fixture.state.current_justified_checkpoint(), target: Checkpoint { epoch, - root: fixture.head_root, + root: target, }, }; let committee = get_beacon_committee(&fixture.state, slot, committee_index).unwrap(); @@ -479,6 +583,14 @@ mod tests { async fn submit( fixture: &Fixture, attestations: &[SingleAttestation], + ) -> (StatusCode, serde_json::Value) { + submit_as(fixture, attestations, fixture.version).await + } + + async fn submit_as( + fixture: &Fixture, + attestations: &[SingleAttestation], + version: &str, ) -> (StatusCode, serde_json::Value) { let network: RpcToP2PRef = fixture.network.clone(); let app = routes() @@ -487,7 +599,7 @@ mod tests { .layer(Extension(fixture.pool.clone())); let request = Request::post("/eth/v2/beacon/pool/attestations") .header("content-type", "application/json") - .header("eth-consensus-version", "fulu") + .header("eth-consensus-version", version) .body(Body::from(serde_json::to_vec(attestations).unwrap())) .unwrap(); let response = app.oneshot(request).await.unwrap(); @@ -771,40 +883,211 @@ mod tests { } #[tokio::test] - async fn an_aggregate_query_at_a_gloas_slot_is_refused_by_name() { - let mut fixture = fixture(); - let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, 0); - let (store, _) = crate::test_utils::beacon_store_with_config(fixture.state.clone(), config); - fixture.store = store; - let (status, json) = get_aggregate(&fixture, Root::ZERO, fixture.state.slot(), 0).await; + async fn the_gloas_header_needs_a_gloas_slot_and_the_older_ones_a_pre_gloas_slot() { + let gloas = fixture_at(ForkName::Gloas, 0); + let vote = attestation(&gloas, 0, 0); + let (status, json) = submit_as(&gloas, std::slice::from_ref(&vote), "fulu").await; + assert_eq!(status, StatusCode::BAD_REQUEST); + assert_eq!( + json["failures"][0]["message"], + "Eth-Consensus-Version does not match the attestation slot's fork" + ); + assert!(gloas.network.published.lock().unwrap().is_empty()); + + let fulu = fixture(); + let vote = attestation(&fulu, 0, 0); + let (status, _) = submit_as(&fulu, std::slice::from_ref(&vote), "gloas").await; assert_eq!(status, StatusCode::BAD_REQUEST); - assert_eq!(json["message"], "gloas aggregates are not supported"); + assert!(fulu.network.published.lock().unwrap().is_empty()); } #[tokio::test] - async fn a_gloas_fork_header_is_refused_by_name() { - for path in [ - "/eth/v2/beacon/pool/attestations", - "/eth/v2/validator/aggregate_and_proofs", - ] { - let fixture = fixture(); - let network: RpcToP2PRef = fixture.network.clone(); - let app = routes() - .with_state(fixture.store.clone()) - .layer(Extension(network)) - .layer(Extension(fixture.pool.clone())); - let request = Request::post(path) - .header("eth-consensus-version", "gloas") - .body(Body::from("[]")) - .unwrap(); - let response = app.oneshot(request).await.unwrap(); - assert_eq!(response.status(), StatusCode::BAD_REQUEST, "{path}"); - let body = response.into_body().collect().await.unwrap().to_bytes(); - let json: serde_json::Value = serde_json::from_slice(&body).unwrap(); - assert_eq!( - json["message"], "Eth-Consensus-Version gloas is not supported", - "{path}" - ); - } + async fn a_gloas_vote_in_the_voted_blocks_own_slot_is_published_with_index_zero() { + let fixture = fixture_at(ForkName::Gloas, 0); + let vote = attestation(&fixture, 0, 0); + let (status, json) = submit(&fixture, std::slice::from_ref(&vote)).await; + assert_eq!(status, StatusCode::OK, "{json}"); + assert_eq!(fixture.network.published.lock().unwrap().len(), 1); + } + + #[tokio::test] + async fn a_same_slot_gloas_vote_for_the_payload_is_refused() { + let fixture = fixture_at(ForkName::Gloas, 0); + let slot = fixture.state.slot(); + let roots = (fixture.head_root, fixture.head_root); + let vote = attestation_for(&fixture, slot, roots, 1, 0, 0); + let (status, json) = submit(&fixture, &[vote]).await; + assert_eq!(status, StatusCode::BAD_REQUEST); + assert_eq!( + json["failures"][0]["message"], + "a same-slot attestation cannot vote for the payload" + ); + assert!(fixture.network.published.lock().unwrap().is_empty()); + } + + #[tokio::test] + async fn a_gloas_index_past_one_is_refused() { + let fixture = fixture_at(ForkName::Gloas, 3); + let slot = fixture.state.slot() + 3; + let roots = (fixture.head_root, fixture.head_root); + let vote = attestation_for(&fixture, slot, roots, 2, 0, 0); + let (status, json) = submit(&fixture, &[vote]).await; + assert_eq!(status, StatusCode::BAD_REQUEST); + assert_eq!( + json["failures"][0]["message"], + "data.index must be 0 or 1 from gloas on" + ); + } + + /// The voted block is the last pre-gloas one, whose payload this node + /// treats as FULL, so a vote for it with `index` 1 is valid. + #[tokio::test] + async fn a_gloas_vote_for_an_earlier_pre_gloas_block_may_name_its_payload() { + let fixture = fixture_at(ForkName::Gloas, 3); + let slot = fixture.state.slot() + 3; + let roots = (fixture.head_root, fixture.head_root); + let vote = attestation_for(&fixture, slot, roots, 1, 0, 0); + let (status, json) = submit(&fixture, &[vote]).await; + assert_eq!(status, StatusCode::OK, "{json}"); + assert_eq!(fixture.network.published.lock().unwrap().len(), 1); + } + + #[tokio::test] + async fn a_gloas_vote_for_a_payload_never_seen_is_refused() { + let mut fixture = fixture_at(ForkName::Gloas, 3); + let head_slot = fixture.state.slot(); + let block = crate::test_utils::gloas_beacon_block( + head_slot + 1, + fixture.head_root, + Root::repeat_byte(1), + Root::repeat_byte(2), + ); + let block_root = block.message_hash_tree_root(); + fixture + .store + .insert_signed_block(block_root, block) + .expect("insert gloas block"); + let slot = head_slot + 3; + let roots = (block_root, block_root); + let empty = attestation_for(&fixture, slot, roots, 0, 0, 0); + let full = attestation_for(&fixture, slot, roots, 1, 0, 1); + let (status, json) = submit(&fixture, &[empty, full]).await; + assert_eq!(status, StatusCode::BAD_REQUEST, "{json}"); + let failures = json["failures"].as_array().unwrap(); + assert_eq!(failures.len(), 1); + assert_eq!(failures[0]["index"], 1); + assert_eq!( + failures[0]["message"], + "the voted block's payload has not been seen and verified" + ); + fixture.network = Arc::new(RecordingNetwork::default()); + } + + /// `aggregate` as a gloas message, signed like [`signed_aggregate`]. + fn signed_gloas_aggregate( + fixture: &Fixture, + aggregator: u64, + aggregate: &electra::Attestation, + ) -> gloas::SignedAggregateAndProof { + use ethlambda_types::beacon::constants::{ + DOMAIN_AGGREGATE_AND_PROOF, DOMAIN_SELECTION_PROOF, + }; + let slot = aggregate.data.slot; + let epoch = compute_epoch_at_slot(slot); + let selection_domain = get_domain(&fixture.state, DOMAIN_SELECTION_PROOF, Some(epoch)); + let selection_proof = sign_for( + aggregator as usize, + compute_signing_root(slot.hash_tree_root(), selection_domain), + ); + let message = gloas::AggregateAndProof { + aggregator_index: aggregator, + aggregate: gloas::Attestation::from(aggregate), + selection_proof, + }; + let domain = get_domain(&fixture.state, DOMAIN_AGGREGATE_AND_PROOF, Some(epoch)); + let signature = sign_for( + aggregator as usize, + compute_signing_root(message.hash_tree_root(), domain), + ); + gloas::SignedAggregateAndProof { message, signature } + } + + /// A gloas aggregator's slot: votes in, the aggregate out as gloas's + /// container, and the signed gloas aggregate published and pooled. + #[tokio::test] + async fn a_gloas_aggregator_can_fetch_and_publish_its_aggregate() { + let fixture = fixture_at(ForkName::Gloas, 0); + let slot = fixture.state.slot(); + let committee = get_beacon_committee(&fixture.state, slot, 0).unwrap(); + let votes: Vec = (0..committee.len()) + .map(|position| attestation(&fixture, 0, position)) + .collect(); + let (status, json) = submit(&fixture, &votes).await; + assert_eq!(status, StatusCode::OK, "{json}"); + + let data = votes[0].data; + let (status, json) = get_aggregate(&fixture, data.hash_tree_root(), slot, 0).await; + assert_eq!(status, StatusCode::OK); + assert_eq!(json["version"], "gloas"); + let served: gloas::Attestation = serde_json::from_value(json["data"].clone()).unwrap(); + let pooled = fixture + .pool + .lock() + .unwrap() + .aggregate(data.hash_tree_root(), slot, 0) + .unwrap(); + assert_eq!(served, gloas::Attestation::from(&pooled)); + + let signed = signed_gloas_aggregate(&fixture, committee[0], &pooled); + let network: RpcToP2PRef = fixture.network.clone(); + let app = routes() + .with_state(fixture.store.clone()) + .layer(Extension(network)) + .layer(Extension(fixture.pool.clone())); + let request = Request::post("/eth/v2/validator/aggregate_and_proofs") + .header("content-type", "application/json") + .header("eth-consensus-version", "gloas") + .body(Body::from( + serde_json::to_vec(std::slice::from_ref(&signed)).unwrap(), + )) + .unwrap(); + let response = app.oneshot(request).await.unwrap(); + let status = response.status(); + let body = response.into_body().collect().await.unwrap().to_bytes(); + assert_eq!(status, StatusCode::OK, "{body:?}"); + let published = fixture.network.aggregates.lock().unwrap(); + assert_eq!(published.len(), 1); + assert_eq!(published[0], SignedAggregateAndProof::Gloas(signed)); + } + + #[tokio::test] + async fn a_gloas_aggregate_with_a_forged_signature_is_refused() { + let fixture = fixture_at(ForkName::Gloas, 0); + let slot = fixture.state.slot(); + let committee = get_beacon_committee(&fixture.state, slot, 0).unwrap(); + let votes: Vec = (0..committee.len()) + .map(|position| attestation(&fixture, 0, position)) + .collect(); + submit(&fixture, &votes).await; + let pooled = fixture + .pool + .lock() + .unwrap() + .aggregate(votes[0].data.hash_tree_root(), slot, 0) + .unwrap(); + let mut forged = signed_gloas_aggregate(&fixture, committee[0], &pooled); + forged.signature = signed_gloas_aggregate(&fixture, committee[1], &pooled).signature; + let network: RpcToP2PRef = fixture.network.clone(); + let app = routes() + .with_state(fixture.store.clone()) + .layer(Extension(network)) + .layer(Extension(fixture.pool.clone())); + let request = Request::post("/eth/v2/validator/aggregate_and_proofs") + .header("eth-consensus-version", "gloas") + .body(Body::from(serde_json::to_vec(&[forged]).unwrap())) + .unwrap(); + let response = app.oneshot(request).await.unwrap(); + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + assert!(fixture.network.aggregates.lock().unwrap().is_empty()); } } diff --git a/crates/net/rpc/src/beacon/proposal.rs b/crates/net/rpc/src/beacon/proposal.rs index 6de77cf4..52be05c6 100644 --- a/crates/net/rpc/src/beacon/proposal.rs +++ b/crates/net/rpc/src/beacon/proposal.rs @@ -15,7 +15,7 @@ use axum::{ Extension, Router, body::Bytes, - extract::{Path, Query, State}, + extract::{DefaultBodyLimit, Path, Query, State}, http::{HeaderMap, HeaderValue, StatusCode, header}, response::{IntoResponse, Response}, routing::{get, post}, @@ -89,7 +89,10 @@ pub(crate) struct FuluSignedBlockContents { pub(crate) fn routes() -> Router { Router::new() .route("/eth/v3/validator/blocks/{slot}", get(get_block)) - .route("/eth/v2/beacon/blocks", post(post_block)) + .route( + "/eth/v2/beacon/blocks", + post(post_block).layer(DefaultBodyLimit::max(super::MAX_PUBLISH_BODY_BYTES)), + ) } /// `POST /eth/v2/beacon/blocks`, SSZ-encoded `SignedBlockContents`. @@ -111,8 +114,8 @@ async fn post_block( .get("eth-consensus-version") .and_then(|value| value.to_str().ok()) .and_then(ForkName::parse); - if fork != Some(ForkName::Fulu) { - return ApiError::BadRequest("Eth-Consensus-Version must be fulu").into_response(); + if !matches!(fork, Some(ForkName::Fulu | ForkName::Gloas)) { + return ApiError::BadRequest("Eth-Consensus-Version must be fulu or gloas").into_response(); } let is_ssz = headers .get(header::CONTENT_TYPE) @@ -125,6 +128,9 @@ async fn post_block( ) .into_response(); } + if fork == Some(ForkName::Gloas) { + return post_gloas_block(&store, &p2p, &body).await; + } let Ok(contents) = FuluSignedBlockContents::from_ssz_bytes(&body) else { return ApiError::BadRequest("the body is not fulu SignedBlockContents").into_response(); }; @@ -170,6 +176,88 @@ async fn post_block( StatusCode::OK.into_response() } +/// The gloas half of `publishBlockV2`: a bare SSZ `SignedBeaconBlock`, since a +/// gloas block carries no payload and no blobs of its own (the envelope and +/// the data columns follow through `publishExecutionPayloadEnvelope`). +/// +/// Checked as the fulu one is, before it goes anywhere: the parent is held, +/// the slot is after the parent's and scheduled at gloas, the proposer is the +/// slot's, and the proposer's signature verifies against the parent's state +/// advanced to the slot. The advance and the signature are CPU-bound and run +/// off the runtime. +async fn post_gloas_block(store: &Store, p2p: &RpcToP2PRef, body: &[u8]) -> Response { + let Ok(signed) = containers::gloas::SignedBeaconBlock::from_ssz_bytes(body) else { + return ApiError::BadRequest("the body is not a gloas SignedBeaconBlock").into_response(); + }; + let block = containers::SignedBeaconBlock::Gloas(signed); + let config = store.config(); + if let Err(err) = require_gloas_slot( + &config, + block.slot(), + "blocks are accepted for gloas slots only", + ) { + return err.into_response(); + } + let parent_state = match store.get_state(&block.parent_root()) { + Ok(Some(state)) => state, + Ok(None) => return ApiError::BadRequest("the block's parent is not held").into_response(), + Err(_) => return ApiError::Internal("store read failed").into_response(), + }; + if block.slot() <= parent_state.slot() { + return ApiError::BadRequest("the block is not after its parent").into_response(); + } + let verdict = { + let block = block.clone(); + tokio::task::spawn_blocking(move || { + let state = advance_to_slot(&parent_state, block.slot(), &config) + .map_err(|_| ApiError::Internal("advancing the parent state failed"))?; + let expected = + ethlambda_state_transition::beacon::helpers::accessors::get_beacon_proposer_index( + &state, + ) + .map_err(|_| ApiError::Internal("no proposer for the slot"))?; + if block.proposer_index() != expected { + return Err(ApiError::BadRequest( + "the block's proposer is not the slot's", + )); + } + if !verify_block_signature(&state, &block) { + return Err(ApiError::BadRequest("invalid block signature")); + } + Ok(()) + }) + .await + }; + match verdict { + Ok(Ok(())) => {} + Ok(Err(err)) => return err.into_response(), + Err(_) => return ApiError::Internal("verifying the block failed").into_response(), + } + if p2p.publish_beacon_block(block).is_err() { + return ApiError::Internal("the network actor is not running").into_response(); + } + StatusCode::OK.into_response() +} + +/// Refuses a slot the schedule does not place at gloas. +pub(crate) fn require_gloas_slot( + config: &Config, + slot: Slot, + message: &'static str, +) -> Result<(), ApiError> { + match config.fork_at_epoch(compute_epoch_at_slot(slot)) { + ForkName::Gloas => Ok(()), + ForkName::Phase0 + | ForkName::Altair + | ForkName::Bellatrix + | ForkName::Capella + | ForkName::Deneb + | ForkName::Electra + | ForkName::Fulu + | ForkName::Lean => Err(ApiError::BadRequest(message)), + } +} + #[derive(Debug, Deserialize)] struct ProduceQuery { randao_reveal: BlsSignature, @@ -409,7 +497,7 @@ async fn build_payload( } /// A wei amount as the decimal string the Beacon API's value fields carry. -fn decimal(value: ðlambda_types::beacon::primitives::Uint256) -> String { +pub(crate) fn decimal(value: ðlambda_types::beacon::primitives::Uint256) -> String { let hex = uint256(value); u128::from_str_radix(hex.trim_start_matches("0x"), 16) .map(|value| value.to_string()) @@ -490,4 +578,70 @@ mod tests { assert_eq!(status, StatusCode::BAD_REQUEST); assert_eq!(json["message"], "blocks are accepted for fulu slots only"); } + + #[tokio::test] + async fn a_block_over_the_default_body_limit_is_not_a_413() { + let (store, _) = gloas_scheduled_store(); + let network: RpcToP2PRef = Arc::new(RecordingNetwork::default()); + let app = routes().with_state(store).layer(Extension(network)); + let request = Request::post("/eth/v2/beacon/blocks") + .header("eth-consensus-version", "fulu") + .header(header::CONTENT_TYPE, crate::SSZ_CONTENT_TYPE) + .body(Body::from(vec![0u8; 3 * 1024 * 1024])) + .unwrap(); + let (status, _) = respond(app, request).await; + assert_ne!(status, StatusCode::PAYLOAD_TOO_LARGE); + } + + #[tokio::test] + async fn a_signed_gloas_block_is_gossiped_and_a_forged_one_is_refused() { + use ethlambda_state_transition::beacon::gloas_block_production::test_support::{ + config, parent_state, produce, sign_block, state_to_build_on, + }; + let parent = parent_state(); + let state = state_to_build_on(); + let produced = produce(&state, true, Vec::new()).unwrap(); + let (mut store, _anchor) = beacon_store_with_config(parent.clone(), config()); + store + .insert_state(produced.block.parent_root, parent) + .unwrap(); + + let signed = containers::gloas::SignedBeaconBlock { + signature: sign_block(&state, &produced.block), + message: produced.block.clone(), + }; + let post = |store: Store, + network: Arc, + signed: &containers::gloas::SignedBeaconBlock| { + let network: RpcToP2PRef = network; + let app = routes().with_state(store).layer(Extension(network)); + let request = Request::post("/eth/v2/beacon/blocks") + .header("eth-consensus-version", "gloas") + .header(header::CONTENT_TYPE, crate::SSZ_CONTENT_TYPE) + .body(Body::from(signed.to_ssz())) + .unwrap(); + respond(app, request) + }; + + let network = Arc::new(RecordingNetwork::default()); + let (status, _) = post(store.clone(), network.clone(), &signed).await; + assert_eq!(status, StatusCode::OK); + assert_eq!(network.blocks.lock().unwrap().len(), 1); + + // A signature from another validator, and a block whose parent is not held. + let rejected = Arc::new(RecordingNetwork::default()); + let mut forged = signed.clone(); + forged.signature = sign_block(&state, &{ + let mut other = produced.block.clone(); + other.proposer_index += 1; + other + }); + let (status, _) = post(store.clone(), rejected.clone(), &forged).await; + assert_eq!(status, StatusCode::BAD_REQUEST); + let mut orphan = signed; + orphan.message.parent_root = H256::repeat_byte(5); + let (status, _) = post(store, rejected.clone(), &orphan).await; + assert_eq!(status, StatusCode::BAD_REQUEST); + assert!(rejected.blocks.lock().unwrap().is_empty()); + } } diff --git a/crates/net/rpc/src/beacon/ptc.rs b/crates/net/rpc/src/beacon/ptc.rs new file mode 100644 index 00000000..e827d3e5 --- /dev/null +++ b/crates/net/rpc/src/beacon/ptc.rs @@ -0,0 +1,1166 @@ +//! The payload timeliness committee's validator-facing endpoints. +//! +//! - `POST /eth/v1/validator/duties/ptc/{epoch}`: which slot of an epoch each +//! requested validator sits in the committee (`get_ptc_assignment`). +//! - `GET /eth/v1/validator/payload_attestation_data`: what a committee member +//! signs for a slot. +//! - `POST /eth/v1/beacon/pool/payload_attestations`: a member's signed vote, +//! validated as gossip would, pooled for block production and gossiped. +//! - `GET /eth/v1/beacon/pool/payload_attestations`: what the pool holds, +//! aggregated per slot and data. + +use std::collections::{BTreeMap, HashSet}; +use std::sync::Arc; + +use axum::{ + Extension, Router, + body::Bytes, + extract::{Path, Query, State}, + http::{HeaderMap, StatusCode, header}, + response::{IntoResponse, Response}, + routing::{get, post}, +}; +use ethlambda_blockchain::SyncStatusController; +use ethlambda_blockchain::metrics::SyncStatus; +use ethlambda_network_api::RpcToP2PRef; +use ethlambda_state_transition::beacon::{ + fork_choice::{get_current_slot, get_payload_due_ms}, + gloas_block_production::aggregate_payload_attestations, + gossip::{ + Outcome, + payload_attestation::{SeenPayloadAttestations, cheap_checks, stateful_checks}, + }, + helpers::{accessors::get_block_root_at_slot, gloas::get_ptc_assignments}, + payload_attestation_pool::SharedPayloadAttestationPool, + preset, + stf::process_slots, +}; +use ethlambda_storage::Store; +use ethlambda_types::{ + beacon::{ + containers::{ + BeaconState, SignedBeaconBlock, + gloas::{PayloadAttestation, PayloadAttestationData, PayloadAttestationMessage}, + }, + fork::ForkName, + primitives::{BlsPubkey, Epoch, Slot, ValidatorIndex}, + signing::{compute_epoch_at_slot, compute_start_slot_at_epoch}, + }, + primitives::H256, +}; +use libssz::SszEncode; +use serde::{Deserialize, Serialize}; +use tracing::{debug, warn}; + +use crate::{ + CustodyColumns, + beacon::{ApiError, validator::head}, + shared::content::{Encoding, ssz_response, with_consensus_version}, +}; + +pub(crate) fn routes() -> Router { + Router::new() + .route( + "/eth/v1/validator/duties/ptc/{epoch}", + post(post_ptc_duties), + ) + .route( + "/eth/v1/validator/payload_attestation_data", + get(get_payload_attestation_data), + ) + .route( + "/eth/v1/beacon/pool/payload_attestations", + get(get_pool_payload_attestations).post(post_pool_payload_attestations), + ) +} + +// --------------------------------------------------------------------------- +// Duties +// --------------------------------------------------------------------------- + +#[derive(Debug, Serialize)] +struct PtcDuty { + pubkey: BlsPubkey, + #[serde(with = "ethlambda_types::beacon::serde_helpers::quoted_or_bare")] + validator_index: ValidatorIndex, + #[serde(with = "ethlambda_types::beacon::serde_helpers::quoted_or_bare")] + slot: Slot, +} + +/// `POST /eth/v1/validator/duties/ptc/{epoch}`. +/// +/// The body is a JSON array of quoted validator indices. Each answers with the +/// first slot of `epoch` whose committee holds it: a validator with several +/// seats still gets one duty per epoch, and one in no committee gets none, as +/// does an unknown index. +/// +/// `epoch` may be at most one past the current one. A gloas epoch is answered +/// from the head state's `ptc_window`, which `get_ptc` can read for the +/// state's epoch, the one before and the next one. Anything else (the first +/// gloas epoch while the head is still fulu, or an epoch the head has not +/// caught up to) is answered from a copy of the head state advanced to the +/// epoch's first slot, crossing the gloas upgrade where it falls. That runs on +/// a blocking thread: on a large registry the epoch transitions are seconds +/// of work, not something to hold a runtime worker for. +/// +/// An epoch before gloas has no committee, so the answer is empty rather than +/// an error: a validator client asks every epoch and reads the empty list as +/// "no duty". +/// +/// `dependent_root` is the block root at `compute_start_slot_at_epoch(epoch - 1) - 1` +/// (the genesis block's at the start), the same definition the attester duties +/// use: the committee window is fixed by the state at the end of the epoch +/// before the previous one. +async fn post_ptc_duties( + Path(epoch): Path, + State(store): State, + body: Bytes, +) -> Response { + let Ok(epoch) = epoch.parse::() else { + return ApiError::BadRequest("invalid epoch").into_response(); + }; + let wanted = match parse_validator_indices(&body) { + Ok(wanted) => wanted, + Err(err) => return err.into_response(), + }; + let computed = tokio::task::spawn_blocking(move || ptc_duties(&store, epoch, &wanted)).await; + match computed { + Ok(Ok(body)) => crate::json_response(body), + Ok(Err(err)) => err.into_response(), + Err(_) => ApiError::Internal("computing the duties failed").into_response(), + } +} + +fn parse_validator_indices(body: &[u8]) -> Result, ApiError> { + let indices = serde_json::from_slice::>(body) + .map_err(|_| ApiError::BadRequest("invalid request body"))?; + if indices.is_empty() { + return Err(ApiError::BadRequest("no validator indices")); + } + indices + .iter() + .map(|index| index.parse::()) + .collect::>() + .map_err(|_| ApiError::BadRequest("invalid validator index")) +} + +fn ptc_duties( + store: &Store, + epoch: Epoch, + wanted: &HashSet, +) -> Result { + let config = store.config(); + let (head_root, head_state) = head(store)?; + let state_epoch = compute_epoch_at_slot(head_state.slot()); + // The wall clock can be ahead of the head (a node that missed slots) and, + // in a follower's store, behind it; either way an epoch more than one past + // the later of the two is not one a validator is asked about yet. + let clock_epoch = compute_epoch_at_slot(get_current_slot(store, &config)); + if epoch > state_epoch.max(clock_epoch) + 1 { + return Err(ApiError::BadRequest( + "epoch is more than one past the current", + )); + } + + let dependent_slot = compute_start_slot_at_epoch(epoch.saturating_sub(1)).saturating_sub(1); + let dependent_root = if dependent_slot >= head_state.slot() { + head_root + } else { + get_block_root_at_slot(&head_state, dependent_slot) + .map_err(|_| ApiError::Internal("dependent slot is outside the state's root window"))? + }; + let respond = |duties: Vec| { + serde_json::json!({ + "dependent_root": dependent_root, + "execution_optimistic": store.is_beacon_optimistic(head_root), + "data": duties, + }) + }; + + if epoch < config.gloas_fork_epoch { + return Ok(respond(Vec::new())); + } + if epoch + 1 < state_epoch { + return Err(ApiError::BadRequest( + "epoch is more than one before the head state's", + )); + } + + let readable_from_head = matches!(*head_state, BeaconState::Gloas(_)) + && epoch <= state_epoch + preset::MIN_SEED_LOOKAHEAD; + let state: Arc = if readable_from_head { + head_state + } else { + let start_slot = compute_start_slot_at_epoch(epoch); + let mut advanced = (*head_state).clone(); + process_slots(&mut advanced, start_slot, &config) + .map_err(|_| ApiError::Internal("advancing the head state failed"))?; + Arc::new(advanced) + }; + + let assignments = get_ptc_assignments(&state, epoch, wanted, &config) + .map_err(|_| ApiError::Internal("the payload timeliness committee is unavailable"))?; + let mut duties: Vec = assignments + .into_iter() + .filter_map(|(validator_index, slot)| { + let pubkey = state.validator(validator_index).ok()?.pubkey; + Some(PtcDuty { + pubkey, + validator_index, + slot, + }) + }) + .collect(); + duties.sort_by_key(|duty| (duty.slot, duty.validator_index)); + Ok(respond(duties)) +} + +// --------------------------------------------------------------------------- +// Payload attestation data +// --------------------------------------------------------------------------- + +#[derive(Debug, Deserialize)] +struct PayloadAttestationDataQuery { + slot: Slot, +} + +/// `GET /eth/v1/validator/payload_attestation_data?slot`. +/// +/// For the canonical block of `slot`, meaning the block of that very slot on +/// the head's chain (the head itself when it is one). A slot with no such block +/// answers 204, which tells the validator client to cast no vote: a member +/// votes on the block of its slot, and a slot that went empty has none. +/// +/// - `payload_present`: an envelope for the block reached this node before +/// `get_payload_due_ms()` into the slot, the specification's rule. A late +/// envelope, or one this node never saw, is `false`. +/// - `blob_data_available`: the bid commits to no blobs, or the payload is +/// already verified (which required them), or every column this node +/// custodies for the block is stored. Columns are verified on arrival, so +/// holding them is the evidence `is_data_available` asks for. +async fn get_payload_attestation_data( + Query(query): Query, + State(store): State, + Extension(custody): Extension, + Extension(sync_status): Extension, + headers: HeaderMap, +) -> Response { + if sync_status.get() == SyncStatus::Syncing { + return ApiError::ServiceUnavailable("node is syncing").into_response(); + } + let data = match payload_attestation_data(&store, &custody.0, query.slot) { + Ok(Some(data)) => data, + Ok(None) => return StatusCode::NO_CONTENT.into_response(), + Err(err) => return err.into_response(), + }; + let accept = headers.get(header::ACCEPT).and_then(|v| v.to_str().ok()); + let response = match Encoding::from_accept(accept) { + Encoding::Ssz => ssz_response(data.to_ssz()), + Encoding::Json => crate::json_response(serde_json::json!({ + "version": ForkName::Gloas.as_str(), + "data": data, + })), + }; + with_consensus_version(response, ForkName::Gloas) +} + +/// The number of blob commitments in a gloas block's bid. +fn bid_commitment_count(block: &SignedBeaconBlock) -> Option { + match block { + SignedBeaconBlock::Gloas(inner) => Some( + inner + .message + .body + .signed_execution_payload_bid + .message + .blob_kzg_commitments + .len(), + ), + _ => None, + } +} + +/// The block of `slot` on the head's chain, if there is one. +fn canonical_block_at(store: &Store, slot: Slot) -> Result, ApiError> { + let (head_slot, head_root) = store + .beacon_head() + .ok_or(ApiError::Internal("no head block"))?; + if slot > head_slot { + return Ok(None); + } + if slot == head_slot { + return Ok(Some(head_root)); + } + let (_, state) = head(store)?; + // The state records the latest block at or before the slot, which is the + // block of a slot only when that slot was not skipped. + let Ok(root) = get_block_root_at_slot(&state, slot) else { + return Ok(None); + }; + Ok(store + .block_slot_and_state_root(&root) + .filter(|(block_slot, _)| *block_slot == slot) + .map(|_| root)) +} + +fn payload_attestation_data( + store: &Store, + custody_columns: &[u64], + slot: Slot, +) -> Result, ApiError> { + let config = store.config(); + if compute_epoch_at_slot(slot) < config.gloas_fork_epoch { + return Err(ApiError::BadRequest("slot is before the gloas fork")); + } + let Some(root) = canonical_block_at(store, slot)? else { + return Ok(None); + }; + let block = store + .get_signed_block(&root) + .map_err(|_| ApiError::Internal("store read failed"))?; + let Some(commitment_count) = block.as_ref().and_then(bid_commitment_count) else { + return Ok(None); + }; + + let due_ms = slot + .saturating_mul(config.slot_duration_ms) + .saturating_add(get_payload_due_ms(&config)); + let payload_present = store + .beacon_envelope_seen_ms(&root) + .is_some_and(|seen_ms| seen_ms < due_ms); + + let blob_data_available = commitment_count == 0 || store.has_verified_payload(&root) || { + let present = store + .data_column_indices_for(slot, &root) + .map_err(|_| ApiError::Internal("store read failed"))?; + custody_columns + .iter() + .all(|column| present.contains(column)) + }; + + Ok(Some(PayloadAttestationData { + beacon_block_root: root, + slot, + payload_present, + blob_data_available, + })) +} + +// --------------------------------------------------------------------------- +// Pool +// --------------------------------------------------------------------------- + +/// One rejected message, in the Beacon API's `IndexedErrorMessage` shape: its +/// position in the submitted array, and why. +#[derive(Debug, Serialize)] +struct Failure { + index: usize, + message: String, +} + +/// `POST /eth/v1/beacon/pool/payload_attestations`. +/// +/// Each message goes through the checks gossip applies to a peer's +/// (`validate_payload_attestation_message_gossip`: the current slot, a known +/// block at the slot, committee membership, the signature), since a vote that +/// fails them is one every peer would score this node down for relaying. +/// A valid one is pooled for block production, gossiped, and handed to the +/// chain actor by the publish path (gossip never delivers a node its own +/// messages), the others are reported by position and the rest still go out. +async fn post_pool_payload_attestations( + State(store): State, + Extension(p2p): Extension, + Extension(pool): Extension, + headers: HeaderMap, + body: Bytes, +) -> Response { + if let Err(err) = require_gloas_or_absent(&headers) { + return err.into_response(); + } + let Ok(messages) = serde_json::from_slice::>(&body) else { + return ApiError::BadRequest("invalid request body").into_response(); + }; + + let now_ms = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|elapsed| elapsed.as_millis() as u64) + .unwrap_or(0); + // Not P2P's seen cache: that holds what peers sent, and says nothing about + // what this node's own validators submit. The pool's own dedup covers a + // repeat. + let seen = SeenPayloadAttestations::new(std::num::NonZeroUsize::MIN); + let mut failures = Vec::new(); + for (index, message) in messages.into_iter().enumerate() { + let slot = message.data.slot; + let validator = message.validator_index; + let verdict = match cheap_checks(&seen, &store, &message, now_ms) { + Ok(()) => stateful_checks(&store, &message), + Err(outcome) => outcome, + }; + if verdict != Outcome::Accept { + let (outcome, reason) = verdict.labels(); + warn!(%slot, validator, outcome, reason, "Refused a submitted payload attestation"); + failures.push(Failure { + index, + message: format!("{outcome}: {reason}"), + }); + continue; + } + let first = pool + .lock() + .expect("payload attestation pool lock poisoned") + .insert(message.clone()); + if !first { + debug!(%slot, validator, "Payload attestation already pooled; not republishing"); + continue; + } + match p2p.publish_payload_attestation_message(message) { + Ok(()) => debug!(%slot, validator, "Accepted payload attestation for gossip"), + Err(_) => failures.push(Failure { + index, + message: "the network actor is not running".to_string(), + }), + } + } + + if failures.is_empty() { + return StatusCode::OK.into_response(); + } + let body = serde_json::json!({ + "code": 400, + "message": "some payload attestations failed validation and were not published", + "failures": failures, + }); + let mut response = crate::json_response(body); + *response.status_mut() = StatusCode::BAD_REQUEST; + response +} + +/// `Eth-Consensus-Version` is optional here, and must name gloas when given: +/// the container exists from that fork alone. +fn require_gloas_or_absent(headers: &HeaderMap) -> Result<(), ApiError> { + let Some(value) = headers.get("eth-consensus-version") else { + return Ok(()); + }; + match value.to_str().ok().and_then(ForkName::parse) { + Some(ForkName::Gloas) => Ok(()), + _ => Err(ApiError::BadRequest( + "Eth-Consensus-Version must name gloas", + )), + } +} + +#[derive(Debug, Deserialize)] +struct PoolQuery { + slot: Option, +} + +/// `GET /eth/v1/beacon/pool/payload_attestations?slot=`: the votes this node +/// holds as `PayloadAttestation`s, those of `slot` only when given. +/// +/// The pool keeps the unaggregated messages (block production combines the +/// ones it needs), but the specification's response is the aggregate: one per +/// distinct `PayloadAttestationData`, its bitvector over the slot's committee +/// and one aggregated signature. Aggregated here with the logic block +/// production uses, against the head state, which can read the committee of +/// the slot it is in, the one before and the next one. A slot outside that +/// window has no readable committee, so its votes are left out rather than +/// listed unaggregated; the pool only ever accepts the current slot's votes, +/// so in practice that is a vote the head has since left far behind. +async fn get_pool_payload_attestations( + Query(query): Query, + State(store): State, + Extension(pool): Extension, +) -> Response { + let held = pool + .lock() + .expect("payload attestation pool lock poisoned") + .all(query.slot); + let aggregated = tokio::task::spawn_blocking(move || aggregate_pool(&store, held)).await; + let data = match aggregated { + Ok(Ok(data)) => data, + Ok(Err(err)) => return err.into_response(), + Err(_) => return ApiError::Internal("aggregating the pool failed").into_response(), + }; + with_consensus_version( + crate::json_response(serde_json::json!({ + "version": ForkName::Gloas.as_str(), + "data": data, + })), + ForkName::Gloas, + ) +} + +/// Group `held` by slot and aggregate each slot's votes against the head state. +fn aggregate_pool( + store: &Store, + held: Vec, +) -> Result, ApiError> { + let config = store.config(); + let (_, state) = head(store)?; + let mut by_slot: BTreeMap> = BTreeMap::new(); + for message in held { + by_slot.entry(message.data.slot).or_default().push(message); + } + Ok(by_slot + .into_iter() + .flat_map(|(slot, messages)| { + aggregate_payload_attestations(&state, slot, messages, &config) + }) + .map(|(_, attestation)| attestation) + .collect()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::test_utils::{RecordingNetwork, gloas_beacon_block}; + use axum::{body::Body, http::Request}; + use ethlambda_state_transition::beacon::{ + constants::DOMAIN_PTC_ATTESTER, + helpers::{ + accessors::get_domain, + misc::compute_signing_root, + test_state::{sign_for, with_signing_validators_at}, + }, + }; + use ethlambda_storage::{ForkCheckpoints, backend::InMemoryBackend}; + use ethlambda_types::beacon::{config::Config, primitives::KzgCommitment}; + use ethlambda_types::checkpoint::Checkpoint; + use ethlambda_types::primitives::HashTreeRoot as _; + use http_body_util::BodyExt as _; + use libssz::SszDecode; + use tower::ServiceExt as _; + + const COUNT: usize = preset::PTC_WINDOW_LENGTH; + + fn now_secs() -> u64 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_secs() + } + + /// A gloas state with every `ptc_window` entry naming its own window index + /// as the whole committee, so the validator a slot's committee holds is + /// readable straight off the window layout. + fn gloas_state() -> BeaconState { + let mut state = with_signing_validators_at(ForkName::Gloas, COUNT); + let BeaconState::Gloas(inner) = &mut state else { + unreachable!("built as gloas") + }; + // Distinct roots per slot, so a `dependent_root` read from the wrong + // slot shows up. + for (slot, root) in inner.block_roots.iter_mut().enumerate() { + *root = H256::repeat_byte(slot as u8 + 1); + } + for index in 0..preset::PTC_WINDOW_LENGTH { + inner.ptc_window[index] = vec![index as u64; preset::PTC_SIZE] + .try_into() + .expect("built at exactly PTC_SIZE"); + } + state + } + + /// The window index (and so the validator, see [`gloas_state`]) of the + /// `offset`th slot of `epoch`, for a state in `state_epoch`. + fn window_index(state_epoch: Epoch, epoch: Epoch, offset: u64) -> u64 { + (epoch + 1 - state_epoch) * preset::SLOTS_PER_EPOCH + offset + } + + /// A store whose head is a gloas block at `state`'s slot with + /// `commitments` blob commitments in its bid, on a clock that has that + /// slot running now (the gossip checks only take the current slot). + fn store_with_head(state: BeaconState, config: Config, commitments: usize) -> (Store, H256) { + let slot = state.slot(); + let mut block = gloas_beacon_block(slot, H256::ZERO, H256::ZERO, H256::repeat_byte(1)); + let SignedBeaconBlock::Gloas(inner) = &mut block else { + unreachable!("built as gloas") + }; + inner + .message + .body + .signed_execution_payload_bid + .message + .blob_kzg_commitments = vec![KzgCommitment::default(); commitments] + .try_into() + .expect("a few commitments fit"); + let root = block.message_hash_tree_root(); + let slot_secs = config.slot_duration_ms / 1000; + let genesis = now_secs() - slot * slot_secs - 1; + let mut store = Store::init_beacon( + Arc::new(InMemoryBackend::default()), + genesis, + config, + root, + Checkpoint { root, slot }, + slot, + ); + store.insert_signed_block(root, block).unwrap(); + store.insert_state(root, state).unwrap(); + store + .update_checkpoints(ForkCheckpoints::head_only(root)) + .unwrap(); + (store, root) + } + + fn gloas_config() -> Config { + Config::mainnet().with_fork_epoch(ForkName::Gloas, 0) + } + + struct Reply { + status: StatusCode, + headers: HeaderMap, + body: Bytes, + } + + impl Reply { + fn json(&self) -> serde_json::Value { + serde_json::from_slice(&self.body).unwrap() + } + } + + async fn send( + store: Store, + pool: SharedPayloadAttestationPool, + network: Arc, + custody: Vec, + sync: SyncStatusController, + request: Request, + ) -> Reply { + let p2p: RpcToP2PRef = network; + let response = routes() + .with_state(store) + .layer(Extension(p2p)) + .layer(Extension(pool)) + .layer(Extension(CustodyColumns(custody))) + .layer(Extension(sync)) + .oneshot(request) + .await + .unwrap(); + let status = response.status(); + let headers = response.headers().clone(); + let body = response.into_body().collect().await.unwrap().to_bytes(); + Reply { + status, + headers, + body, + } + } + + async fn request(store: Store, request: Request) -> Reply { + send( + store, + Default::default(), + Default::default(), + Vec::new(), + Default::default(), + request, + ) + .await + } + + fn duties_request(epoch: u64, body: &str) -> Request { + Request::post(format!("/eth/v1/validator/duties/ptc/{epoch}")) + .body(Body::from(body.to_string())) + .unwrap() + } + + // ----- duties ---------------------------------------------------------- + + #[tokio::test] + async fn duties_are_read_from_the_window_for_the_previous_current_and_next_epoch() { + let state = gloas_state(); + let state_epoch = compute_epoch_at_slot(state.slot()); + let head_slot = state.slot(); + let (store, head_root) = store_with_head(state, gloas_config(), 0); + for epoch in [state_epoch - 1, state_epoch, state_epoch + 1] { + let validator = window_index(state_epoch, epoch, 3); + let reply = request( + store.clone(), + duties_request(epoch, &format!(r#"["{validator}","9999"]"#)), + ) + .await; + assert_eq!(reply.status, StatusCode::OK); + let json = reply.json(); + // The unknown validator has no duty; the other sits in its slot. + assert_eq!(json["data"].as_array().unwrap().len(), 1); + assert_eq!(json["data"][0]["validator_index"], validator.to_string()); + assert_eq!( + json["data"][0]["slot"], + (compute_start_slot_at_epoch(epoch) + 3).to_string() + ); + assert!( + json["data"][0]["pubkey"] + .as_str() + .unwrap() + .starts_with("0x") + ); + assert_eq!(json["execution_optimistic"], false); + // The block before the previous epoch's start; the head itself + // when that slot is not before the head's. + let dependent_slot = + compute_start_slot_at_epoch(epoch.saturating_sub(1)).saturating_sub(1); + let expected = if dependent_slot >= head_slot { + head_root + } else { + H256::repeat_byte(dependent_slot as u8 + 1) + }; + assert_eq!(json["dependent_root"], format!("{expected:?}")); + } + } + + #[tokio::test] + async fn a_validator_with_several_seats_gets_its_first_slot_once() { + let mut state = gloas_state(); + let state_epoch = compute_epoch_at_slot(state.slot()); + let BeaconState::Gloas(inner) = &mut state else { + unreachable!("built as gloas") + }; + let early = window_index(state_epoch, state_epoch, 5) as usize; + let late = window_index(state_epoch, state_epoch, 9) as usize; + inner.ptc_window[early][1] = 77; + inner.ptc_window[late][0] = 77; + inner.ptc_window[late][1] = 77; + let (store, _) = store_with_head(state, gloas_config(), 0); + + let reply = request(store, duties_request(state_epoch, r#"["77"]"#)).await; + + let json = reply.json(); + assert_eq!(json["data"].as_array().unwrap().len(), 1); + assert_eq!( + json["data"][0]["slot"], + (compute_start_slot_at_epoch(state_epoch) + 5).to_string() + ); + } + + #[tokio::test] + async fn malformed_bodies_and_far_epochs_are_a_400() { + let state = gloas_state(); + let state_epoch = compute_epoch_at_slot(state.slot()); + let (store, _) = store_with_head(state, gloas_config(), 0); + for body in ["[]", "not json", r#"["x"]"#, r#"[1]"#, r#"{"a":1}"#] { + let reply = request(store.clone(), duties_request(state_epoch, body)).await; + assert_eq!(reply.status, StatusCode::BAD_REQUEST, "{body}"); + } + let reply = request(store.clone(), duties_request(state_epoch + 2, r#"["1"]"#)).await; + assert_eq!(reply.status, StatusCode::BAD_REQUEST); + let reply = request(store, duties_request(u64::MAX, r#"["1"]"#)).await; + assert_eq!(reply.status, StatusCode::BAD_REQUEST); + } + + #[tokio::test] + async fn a_pre_gloas_epoch_is_answered_with_no_duties() { + let state = gloas_state(); + let state_epoch = compute_epoch_at_slot(state.slot()); + // Gloas starts after the head's epoch, so the head's own epoch has no + // committee even though the state happens to be gloas-shaped. + let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, state_epoch + 1); + let (store, _) = store_with_head(state, config, 0); + + let reply = request(store, duties_request(state_epoch, r#"["1","2"]"#)).await; + + assert_eq!(reply.status, StatusCode::OK); + assert_eq!(reply.json()["data"], serde_json::json!([])); + assert!(reply.json()["dependent_root"].is_string()); + } + + /// While the head is still fulu, the first gloas epoch is answered from a + /// copy of the head state advanced across the upgrade. + #[tokio::test] + async fn the_first_gloas_epoch_is_computed_from_a_fulu_head() { + use ethlambda_state_transition::beacon::helpers::fulu::initialize_proposer_lookahead; + let mut state = with_signing_validators_at(ForkName::Fulu, COUNT); + let lookahead = initialize_proposer_lookahead(&state).unwrap(); + let BeaconState::Fulu(fulu) = &mut state else { + unreachable!("built as fulu") + }; + fulu.proposer_lookahead = lookahead.try_into().unwrap(); + let head_epoch = compute_epoch_at_slot(state.slot()); + let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, head_epoch + 1); + let (store, _root) = crate::test_utils::beacon_store_with_config(state, config); + let everyone = + serde_json::to_string(&(0..COUNT as u64).map(|i| i.to_string()).collect::>()) + .unwrap(); + + let reply = request(store.clone(), duties_request(head_epoch + 1, &everyone)).await; + + assert_eq!(reply.status, StatusCode::OK, "{:?}", reply.body); + let duties = reply.json()["data"].as_array().unwrap().clone(); + assert!( + !duties.is_empty(), + "the committee seats some of the registry" + ); + let first_slot = compute_start_slot_at_epoch(head_epoch + 1); + for duty in &duties { + let slot: u64 = duty["slot"].as_str().unwrap().parse().unwrap(); + assert!((first_slot..first_slot + preset::SLOTS_PER_EPOCH).contains(&slot)); + } + // The fulu epoch itself has none. + let reply = request(store, duties_request(head_epoch, &everyone)).await; + assert_eq!(reply.json()["data"], serde_json::json!([])); + } + + // ----- payload attestation data ---------------------------------------- + + fn data_request(slot: u64, accept: Option<&str>) -> Request { + let mut request = Request::get(format!( + "/eth/v1/validator/payload_attestation_data?slot={slot}" + )); + if let Some(accept) = accept { + request = request.header("accept", accept); + } + request.body(Body::empty()).unwrap() + } + + fn due_ms(store: &Store, slot: u64) -> u64 { + let config = store.config(); + slot * config.slot_duration_ms + get_payload_due_ms(&config) + } + + #[tokio::test] + async fn the_data_names_the_head_block_and_defaults_to_nothing_seen() { + let state = gloas_state(); + let slot = state.slot(); + let (store, root) = store_with_head(state, gloas_config(), 0); + + let reply = request(store, data_request(slot, None)).await; + + assert_eq!(reply.status, StatusCode::OK); + assert_eq!(reply.headers["eth-consensus-version"], "gloas"); + let json = reply.json(); + assert_eq!(json["version"], "gloas"); + assert_eq!(json["data"]["beacon_block_root"], format!("{root:?}")); + assert_eq!(json["data"]["slot"], slot.to_string()); + assert_eq!(json["data"]["payload_present"], false); + // No commitments, so there is no blob data to wait for. + assert_eq!(json["data"]["blob_data_available"], true); + } + + #[tokio::test] + async fn payload_present_needs_an_envelope_seen_before_the_payload_deadline() { + let slot = gloas_state().slot(); + let present_when_seen_at = |offset: i64| async move { + let (mut store, root) = store_with_head(gloas_state(), gloas_config(), 0); + let seen = (due_ms(&store, slot) as i64 + offset) as u64; + store.insert_beacon_envelope_seen(root, slot, seen); + request(store, data_request(slot, None)).await.json()["data"]["payload_present"].clone() + }; + + assert_eq!(present_when_seen_at(-1).await, true); + // The deadline itself is already late. + assert_eq!(present_when_seen_at(0).await, false); + assert_eq!(present_when_seen_at(1_000).await, false); + } + + #[tokio::test] + async fn blob_availability_follows_the_custody_columns_unless_the_payload_is_verified() { + let slot = gloas_state().slot(); + let ask = |store: Store| async move { + send( + store, + Default::default(), + Default::default(), + vec![0, 1], + Default::default(), + data_request(slot, None), + ) + .await + .json()["data"]["blob_data_available"] + .clone() + }; + let (store, root) = store_with_head(gloas_state(), gloas_config(), 1); + + assert_eq!(ask(store.clone()).await, false); + store + .put_data_column_sidecar(slot, &root, 0, vec![1]) + .unwrap(); + assert_eq!(ask(store.clone()).await, false); + store + .put_data_column_sidecar(slot, &root, 1, vec![1]) + .unwrap(); + assert_eq!(ask(store.clone()).await, true); + + // A verified payload had its data, whatever columns are stored. + let (mut store, root) = store_with_head(gloas_state(), gloas_config(), 1); + let envelope = crate::test_utils::gloas_envelope(root, slot); + store.insert_verified_payload(slot, &envelope); + assert_eq!(ask(store).await, true); + } + + #[tokio::test] + async fn a_slot_without_a_block_is_a_204_and_a_pre_gloas_slot_a_400() { + let state = gloas_state(); + let slot = state.slot(); + let (store, _) = store_with_head(state, gloas_config(), 0); + for missing in [slot + 1, slot - 1] { + let reply = request(store.clone(), data_request(missing, None)).await; + assert_eq!(reply.status, StatusCode::NO_CONTENT, "slot {missing}"); + assert!(reply.body.is_empty()); + } + + let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, 5); + let (store, _) = store_with_head(gloas_state(), config, 0); + let reply = request(store, data_request(slot, None)).await; + assert_eq!(reply.status, StatusCode::BAD_REQUEST); + } + + #[tokio::test] + async fn the_data_is_served_as_ssz_on_request_and_refused_while_syncing() { + let state = gloas_state(); + let slot = state.slot(); + let (store, root) = store_with_head(state, gloas_config(), 0); + + let reply = request( + store.clone(), + data_request(slot, Some("application/octet-stream")), + ) + .await; + assert_eq!(reply.status, StatusCode::OK); + assert_eq!(reply.headers["eth-consensus-version"], "gloas"); + assert_eq!(reply.headers["content-type"], "application/octet-stream"); + let data = PayloadAttestationData::from_ssz_bytes(&reply.body).unwrap(); + assert_eq!(data.beacon_block_root, root); + assert_eq!(data.slot, slot); + + let reply = send( + store, + Default::default(), + Default::default(), + Vec::new(), + SyncStatusController::new(SyncStatus::Syncing), + data_request(slot, None), + ) + .await; + assert_eq!(reply.status, StatusCode::SERVICE_UNAVAILABLE); + } + + // ----- pool ------------------------------------------------------------ + + /// A vote by `validator` on the head block at the head's slot, signed under + /// the state's PTC domain unless `sign` is false. + fn vote( + state: &BeaconState, + root: H256, + validator: u64, + sign: bool, + ) -> PayloadAttestationMessage { + let data = PayloadAttestationData { + beacon_block_root: root, + slot: state.slot(), + payload_present: true, + blob_data_available: true, + }; + let domain = get_domain( + state, + DOMAIN_PTC_ATTESTER, + Some(compute_epoch_at_slot(data.slot)), + ); + let signing_root = compute_signing_root(data.hash_tree_root(), domain); + PayloadAttestationMessage { + validator_index: validator, + data, + signature: if sign { + sign_for(validator as usize, signing_root) + } else { + Default::default() + }, + } + } + + fn submit(messages: &[PayloadAttestationMessage], version: Option<&str>) -> Request { + let mut request = Request::post("/eth/v1/beacon/pool/payload_attestations"); + if let Some(version) = version { + request = request.header("eth-consensus-version", version); + } + request + .body(Body::from(serde_json::to_vec(messages).unwrap())) + .unwrap() + } + + async fn submit_to( + store: Store, + messages: &[PayloadAttestationMessage], + version: Option<&str>, + ) -> (Reply, SharedPayloadAttestationPool, Arc) { + let pool = SharedPayloadAttestationPool::default(); + let network = Arc::new(RecordingNetwork::default()); + let reply = send( + store, + pool.clone(), + network.clone(), + Vec::new(), + Default::default(), + submit(messages, version), + ) + .await; + (reply, pool, network) + } + + #[tokio::test] + async fn a_valid_vote_is_pooled_and_published_once() { + let state = gloas_state(); + let state_epoch = compute_epoch_at_slot(state.slot()); + let member = window_index(state_epoch, state_epoch, 0); + let (store, root) = store_with_head(state.clone(), gloas_config(), 0); + let message = vote(&state, root, member, true); + + let (reply, pool, network) = + submit_to(store, &[message.clone(), message.clone()], Some("gloas")).await; + + assert_eq!(reply.status, StatusCode::OK, "{:?}", reply.body); + assert_eq!(pool.lock().unwrap().all(None), vec![message.clone()]); + // The repeat is neither re-pooled nor re-gossiped. + assert_eq!(*network.payload_attestations.lock().unwrap(), vec![message]); + } + + #[tokio::test] + async fn a_bad_vote_is_reported_by_position_and_the_rest_still_go_out() { + let state = gloas_state(); + let state_epoch = compute_epoch_at_slot(state.slot()); + let member = window_index(state_epoch, state_epoch, 0); + let (store, root) = store_with_head(state.clone(), gloas_config(), 0); + let good = vote(&state, root, member, true); + let unsigned = vote(&state, root, member, false); + // Signed, but its sender sits in no committee of the slot. + let outsider = vote(&state, root, 5, true); + let mut unknown_block = vote(&state, root, member, true); + unknown_block.data.beacon_block_root = H256::repeat_byte(0xee); + + let (reply, pool, network) = submit_to( + store, + &[unsigned, good.clone(), outsider, unknown_block], + Some("gloas"), + ) + .await; + + assert_eq!(reply.status, StatusCode::BAD_REQUEST); + let json = reply.json(); + assert_eq!(json["code"], 400); + let positions: Vec = json["failures"] + .as_array() + .unwrap() + .iter() + .map(|failure| failure["index"].as_u64().unwrap()) + .collect(); + assert_eq!(positions, vec![0, 2, 3]); + assert_eq!(pool.lock().unwrap().all(None), vec![good.clone()]); + assert_eq!(*network.payload_attestations.lock().unwrap(), vec![good]); + } + + #[tokio::test] + async fn a_vote_for_another_slot_is_refused() { + let state = gloas_state(); + let state_epoch = compute_epoch_at_slot(state.slot()); + let member = window_index(state_epoch, state_epoch, 0); + let (store, root) = store_with_head(state.clone(), gloas_config(), 0); + let mut stale = vote(&state, root, member, true); + stale.data.slot -= 1; + + let (reply, pool, _) = submit_to(store, &[stale], None).await; + + assert_eq!(reply.status, StatusCode::BAD_REQUEST); + assert!(pool.lock().unwrap().all(None).is_empty()); + } + + #[tokio::test] + async fn the_consensus_version_must_be_gloas_when_given() { + let state = gloas_state(); + let (store, root) = store_with_head(state.clone(), gloas_config(), 0); + let message = vote(&state, root, 0, false); + for version in ["electra", "fulu", "nonsense"] { + let (reply, pool, _) = + submit_to(store.clone(), std::slice::from_ref(&message), Some(version)).await; + assert_eq!(reply.status, StatusCode::BAD_REQUEST, "{version}"); + assert!(pool.lock().unwrap().all(None).is_empty()); + } + let reply = request( + store, + Request::post("/eth/v1/beacon/pool/payload_attestations") + .body(Body::from("not json")) + .unwrap(), + ) + .await; + assert_eq!(reply.status, StatusCode::BAD_REQUEST); + } + + #[tokio::test] + async fn the_pool_is_listed_as_aggregates_by_slot() { + let state = gloas_state(); + let state_epoch = compute_epoch_at_slot(state.slot()); + let member = window_index(state_epoch, state_epoch, 0); + let (store, root) = store_with_head(state.clone(), gloas_config(), 0); + let pool = SharedPayloadAttestationPool::default(); + let message = vote(&state, root, member, true); + + // The previous slot's committee is its own window entry, which names + // the validator `slot % SLOTS_PER_EPOCH` here. + let mut earlier_state = state.clone(); + let BeaconState::Gloas(inner) = &mut earlier_state else { + unreachable!("built as gloas") + }; + inner.slot -= 1; + let earlier_member = earlier_state.slot() % preset::SLOTS_PER_EPOCH; + let older = vote(&earlier_state, root, earlier_member, true); + + // Not in the head slot's committee: nothing to aggregate it into. + let outsider = vote(&state, root, member + 1, true); + for held in [&older, &message, &outsider] { + pool.lock().unwrap().insert(held.clone()); + } + let list = |uri: String| { + let store = store.clone(); + let pool = pool.clone(); + async move { + send( + store, + pool, + Default::default(), + Vec::new(), + Default::default(), + Request::get(uri).body(Body::empty()).unwrap(), + ) + .await + } + }; + + let all = list("/eth/v1/beacon/pool/payload_attestations".to_string()).await; + assert_eq!(all.status, StatusCode::OK); + assert_eq!(all.headers["eth-consensus-version"], "gloas"); + assert_eq!(all.json()["version"], "gloas"); + let data = all.json()["data"].as_array().unwrap().clone(); + assert_eq!( + data.len(), + 2, + "one aggregate per slot, the outsider's dropped" + ); + // Oldest slot first, each an aggregate and not a message. + assert_eq!(data[0]["data"]["slot"], older.data.slot.to_string()); + assert_eq!(data[1]["data"]["slot"], message.data.slot.to_string()); + for aggregate in &data { + assert!(aggregate.get("validator_index").is_none()); + // The single seat-holder fills every position of the committee. + let bits = aggregate["aggregation_bits"].as_str().unwrap(); + assert_eq!(bits, format!("0x{}", "ff".repeat(preset::PTC_SIZE / 8))); + } + + // It is the aggregate block production would pack, which only keeps + // one that verifies against the state. + let expected = ethlambda_state_transition::beacon::gloas_block_production::aggregate_payload_attestations( + &state, + message.data.slot, + [message.clone()], + &gloas_config(), + ); + assert_eq!(expected.len(), 1); + assert_eq!(data[1], serde_json::to_value(&expected[0].1).unwrap()); + + let slot = message.data.slot; + let one = list(format!( + "/eth/v1/beacon/pool/payload_attestations?slot={slot}" + )) + .await; + assert_eq!(one.json()["data"].as_array().unwrap().len(), 1); + assert_eq!(one.json()["data"][0]["data"]["slot"], slot.to_string()); + } +} diff --git a/crates/net/rpc/src/beacon/validator.rs b/crates/net/rpc/src/beacon/validator.rs index 2d827de7..d1bca404 100644 --- a/crates/net/rpc/src/beacon/validator.rs +++ b/crates/net/rpc/src/beacon/validator.rs @@ -21,6 +21,8 @@ use ethlambda_types::{ BeaconState, shared::{AttestationData, Checkpoint}, }, + fork::ForkName, + fork_choice::PayloadStatus, preset, primitives::{BlsPubkey, CommitteeIndex, Epoch, ExecutionAddress, Slot, ValidatorIndex}, signing::{compute_epoch_at_slot, compute_start_slot_at_epoch}, @@ -31,12 +33,12 @@ use serde::{Deserialize, Serialize}; use ethlambda_network_api::RpcToP2PRef; use ethlambda_state_transition::beacon::{ - fork_choice::checkpoint_state, + fork_choice::{checkpoint_state, get_head_node}, gossip::attestation::compute_subnet_for_attestation, helpers::accessors::{CommitteeCacheExt as _, get_block_root_at_slot}, }; -use crate::beacon::{ApiError, refuse_validator_duties_from_gloas}; +use crate::beacon::ApiError; pub(crate) fn routes() -> Router { Router::new() @@ -190,7 +192,7 @@ struct ProposerDuty { /// `GET /eth/v1/validator/duties/proposer/{epoch}`. /// -/// Read from fulu's `proposer_lookahead`, which the state keeps for its own +/// Read from the `proposer_lookahead` fulu introduced and gloas keeps, which the state keeps for its own /// epoch and the next `MIN_SEED_LOOKAHEAD` epochs, so any epoch in that window /// is answered without advancing a state. Any other epoch is refused. /// @@ -206,17 +208,18 @@ async fn get_proposer_duties(Path(epoch): Path, State(store): State Result { let epoch = parse_epoch(epoch)?; - refuse_validator_duties_from_gloas( - &store.config(), - epoch, - "gloas proposer duties are not supported", - )?; let (head_root, state) = head(store)?; - let BeaconState::Fulu(fulu) = state.as_ref() else { - return Err(ApiError::BadRequest( - "proposer duties are served from fulu's proposer lookahead only", - )); + // Gloas keeps fulu's lookahead as it is, and `upgrade_to_gloas` carries it + // over, so a fulu head already holds the first gloas epoch's proposers. + let proposer_lookahead = match state.as_ref() { + BeaconState::Fulu(fulu) => &fulu.proposer_lookahead, + BeaconState::Gloas(gloas) => &gloas.proposer_lookahead, + _ => { + return Err(ApiError::BadRequest( + "proposer duties are served from the proposer lookahead of fulu and gloas only", + )); + } }; let state_epoch = compute_epoch_at_slot(state.slot()); let offset = epoch @@ -228,7 +231,7 @@ fn proposer_duties(store: &Store, epoch: &str) -> Result()) .collect::, _>>() .map_err(|_| ApiError::BadRequest("invalid validator index"))?; - refuse_validator_duties_from_gloas( - &store.config(), - epoch, - "gloas attester duties are not supported", - )?; let (head_root, state) = head(store)?; let state_epoch = compute_epoch_at_slot(state.slot()); @@ -363,11 +361,11 @@ fn attester_duties( #[derive(Debug, Deserialize)] struct AttestationDataQuery { slot: Slot, - /// Required by the endpoint, and ignored: from electra on the committee - /// travels outside `AttestationData`, whose `index` is always zero, so - /// every committee of a slot attests to the same data. + /// Ignored, and optional since gloas's beacon-APIs dropped it: from electra + /// on the committee travels outside `AttestationData`, whose `index` no + /// longer names it, so every committee of a slot attests to the same data. #[allow(dead_code)] - committee_index: CommitteeIndex, + committee_index: Option, } /// `GET /eth/v1/validator/attestation_data?slot&committee_index`. @@ -384,7 +382,8 @@ struct AttestationDataQuery { /// - `target` is `slot`'s epoch and its boundary block: the head itself when /// no block has filled the boundary slot since, else the root the state /// recorded there. -/// - `index` is zero, as electra requires. +/// - `index` is zero, as electra requires. At a gloas slot it is the +/// payload-present flag instead; see [`payload_present_index`]. /// /// A slot before the head's, or more than one slot past the wall clock, is /// refused: neither is a slot a validator is asked to attest to. @@ -399,11 +398,6 @@ async fn get_attestation_data( } fn attestation_data(store: &Store, slot: Slot) -> Result { - refuse_validator_duties_from_gloas( - &store.config(), - compute_epoch_at_slot(slot), - "gloas attestation data is not supported", - )?; let (head_root, state) = head(store)?; if slot < state.slot() { return Err(ApiError::BadRequest("slot is before the head block")); @@ -430,15 +424,56 @@ fn attestation_data(store: &Store, slot: Slot) -> Result Result { + if head_state.slot() == slot { + return Ok(0); + } + if !matches!(head_state, BeaconState::Gloas(_)) { + return Ok(1); + } + let status = match store.head_payload_status() { + Some(status) => status, + None => { + get_head_node(store, &store.config()) + .map_err(|_| ApiError::Internal("fork choice head lookup failed"))? + .payload_status + } + }; + Ok(u64::from(status == PayloadStatus::Full)) +} + #[cfg(test)] mod tests { use super::*; @@ -451,6 +486,7 @@ mod tests { fulu::{get_beacon_proposer_indices, initialize_proposer_lookahead}, test_state::with_signing_validators_at, }; + use ethlambda_types::beacon::config::Config; use ethlambda_types::beacon::fork::ForkName; use http_body_util::BodyExt as _; use tower::ServiceExt as _; @@ -848,52 +884,113 @@ mod tests { } #[tokio::test] - async fn attestation_data_is_answered_up_to_the_last_fulu_slot_and_refused_after() { - let (status, _) = - with_gloas_next_epoch(|_, head_slot| attestation_data_request(head_slot)).await; - assert_eq!(status, StatusCode::OK); - - let (status, json) = with_gloas_next_epoch(|head_epoch, _| { - attestation_data_request(compute_start_slot_at_epoch(head_epoch + 1)) + async fn a_fulu_head_answers_the_first_gloas_epoch_for_every_duty() { + let (status, _) = with_gloas_next_epoch(|head_epoch, _| { + get_request(format!( + "/eth/v1/validator/duties/proposer/{}", + head_epoch + 1 + )) }) .await; - assert_eq!(status, StatusCode::BAD_REQUEST); - assert_eq!(json["message"], "gloas attestation data is not supported"); - } + assert_eq!(status, StatusCode::OK); - #[tokio::test] - async fn proposer_duties_are_answered_for_the_last_fulu_epoch_and_refused_after() { let (status, _) = with_gloas_next_epoch(|head_epoch, _| { - get_request(format!("/eth/v1/validator/duties/proposer/{head_epoch}")) + Request::post(format!( + "/eth/v1/validator/duties/attester/{}", + head_epoch + 1 + )) + .header("content-type", "application/json") + .body(Body::from("[\"0\"]")) + .unwrap() }) .await; assert_eq!(status, StatusCode::OK); + // The head is a fulu block, which counts as FULL at the boundary. let (status, json) = with_gloas_next_epoch(|head_epoch, _| { - get_request(format!( - "/eth/v1/validator/duties/proposer/{}", - head_epoch + 1 - )) + attestation_data_request(compute_start_slot_at_epoch(head_epoch + 1)) }) .await; - assert_eq!(status, StatusCode::BAD_REQUEST); - assert_eq!(json["message"], "gloas proposer duties are not supported"); + assert_eq!(status, StatusCode::OK); + assert_eq!(json["data"]["index"], "1"); } #[tokio::test] - async fn attester_duties_are_answered_for_the_last_fulu_epoch_and_refused_after() { - let attester = |epoch: Epoch| { - Request::post(format!("/eth/v1/validator/duties/attester/{epoch}")) - .header("content-type", "application/json") - .body(Body::from("[\"0\"]")) - .unwrap() + async fn the_gloas_proposer_duties_match_the_lookahead_of_a_gloas_state() { + let mut state = with_signing_validators_at(ForkName::Gloas, COUNT); + let lookahead = initialize_proposer_lookahead(&state).unwrap(); + let BeaconState::Gloas(gloas) = &mut state else { + unreachable!("built as gloas") }; - let (status, _) = with_gloas_next_epoch(|head_epoch, _| attester(head_epoch)).await; - assert_eq!(status, StatusCode::OK); + gloas.proposer_lookahead = lookahead.clone().try_into().unwrap(); + let epoch = compute_epoch_at_slot(state.slot()); + let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, 0); + let (store, _) = crate::test_utils::beacon_store_with_config(state, config); + let request = get_request(format!("/eth/v1/validator/duties/proposer/{epoch}")); + let response = routes().with_state(store).oneshot(request).await.unwrap(); + assert_eq!(response.status(), StatusCode::OK); + let body = response.into_body().collect().await.unwrap().to_bytes(); + let json: serde_json::Value = serde_json::from_slice(&body).unwrap(); + let proposers: Vec = json["data"] + .as_array() + .unwrap() + .iter() + .map(|duty| duty["validator_index"].as_str().unwrap().parse().unwrap()) + .collect(); + assert_eq!(proposers, lookahead[..preset::SLOTS_PER_EPOCH as usize]); + } + + /// A store whose head is a gloas state `slots_past_boundary` into its + /// epoch, under a schedule with gloas from epoch 0, and the payload status + /// fork choice recorded for that head. + fn gloas_head( + slots_past_boundary: u64, + recorded: Option, + ) -> (Store, BeaconState) { + let mut state = with_signing_validators_at(ForkName::Gloas, COUNT); + let BeaconState::Gloas(gloas) = &mut state else { + unreachable!("built as gloas") + }; + gloas.slot += slots_past_boundary; + let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, 0); + let (store, head_root) = crate::test_utils::beacon_store_with_config(state.clone(), config); + if let Some(status) = recorded { + store.set_head_payload_status(head_root, status); + } + (store, state) + } - let (status, json) = with_gloas_next_epoch(|head_epoch, _| attester(head_epoch + 1)).await; - assert_eq!(status, StatusCode::BAD_REQUEST); - assert_eq!(json["message"], "gloas attester duties are not supported"); + async fn gloas_index(store: Store, slot: Slot) -> (StatusCode, serde_json::Value) { + // No `committee_index`: gloas's request omits it. + let request = get_request(format!("/eth/v1/validator/attestation_data?slot={slot}")); + let response = routes().with_state(store).oneshot(request).await.unwrap(); + let status = response.status(); + let body = response.into_body().collect().await.unwrap().to_bytes(); + (status, serde_json::from_slice(&body).unwrap()) + } + + #[tokio::test] + async fn a_gloas_vote_in_the_head_blocks_own_slot_has_index_zero() { + let (store, state) = gloas_head(1, Some(PayloadStatus::Full)); + let (status, json) = gloas_index(store, state.slot()).await; + assert_eq!(status, StatusCode::OK, "{json}"); + assert_eq!(json["data"]["index"], "0"); + } + + #[tokio::test] + async fn a_gloas_vote_for_an_earlier_full_head_has_index_one() { + let (store, state) = gloas_head(1, Some(PayloadStatus::Full)); + let (status, json) = gloas_index(store, state.slot() + 1).await; + assert_eq!(status, StatusCode::OK, "{json}"); + assert_eq!(json["data"]["index"], "1"); + } + + #[tokio::test] + async fn a_gloas_vote_for_an_earlier_empty_head_has_index_zero() { + let (store, state) = gloas_head(1, Some(PayloadStatus::Empty)); + let (status, json) = gloas_index(store, state.slot() + 1).await; + assert_eq!(status, StatusCode::OK, "{json}"); + assert_eq!(json["data"]["index"], "0"); } #[tokio::test] diff --git a/crates/net/rpc/src/beacon/validator_client_tests.rs b/crates/net/rpc/src/beacon/validator_client_tests.rs index 16663d75..93196049 100644 --- a/crates/net/rpc/src/beacon/validator_client_tests.rs +++ b/crates/net/rpc/src/beacon/validator_client_tests.rs @@ -18,6 +18,7 @@ use ethlambda_state_transition::beacon::helpers::{ fulu::initialize_proposer_lookahead, test_state::{sign_for, with_signing_validators_at}, }; +use ethlambda_state_transition::beacon::payload_attestation_pool::SharedPayloadAttestationPool; use ethlambda_types::{ beacon::{ constants::DOMAIN_BEACON_ATTESTER, @@ -32,7 +33,7 @@ use ethlambda_types::{ use ethlambda_validator::{ Error, beacon_node::{ - BeaconNodeApi, BlockRequest, + AggregateKind, BeaconNodeApi, BlockRequest, SignedAggregates, dto::{ AttestationDataOutDto, CommitteeSubscriptionDto, ProposerPreparationDto, SingleAttestationDto, encode_hex, @@ -41,6 +42,8 @@ use ethlambda_validator::{ }, }; +use ethlambda_storage::Store; + use crate::test_utils::{RecordingNetwork, beacon_store_at}; const COUNT: usize = 64; @@ -76,7 +79,21 @@ async fn serve_with_engine( fulu.current_sync_committee = sync_committee.clone(); fulu.next_sync_committee = sync_committee; - let (mut store, _root) = beacon_store_at(state.clone()); + let (store, _root) = beacon_store_at(state.clone()); + let (client, network, _pool) = spawn_server(store, engine).await; + (client, state, network) +} + +/// Serve `store` through the router the real server builds, over a socket on an +/// ephemeral port, and connect this crate's client to it. +async fn spawn_server( + mut store: Store, + engine: Option, +) -> ( + HttpBeaconNode, + Arc, + SharedPayloadAttestationPool, +) { // The store's clock, which the aggregate conditions read the current epoch // from, as the chain actor's ticks would have set it. let now_ms = std::time::SystemTime::now() @@ -86,10 +103,13 @@ async fn serve_with_engine( store.set_time_ms(now_ms).unwrap(); let network = Arc::new(RecordingNetwork::default()); let p2p: RpcToP2PRef = network.clone(); + let payload_pool = SharedPayloadAttestationPool::default(); let router = crate::build_beacon_api_router(store, "ethlambda/test", "peer".into()) .layer(Extension(SyncStatusController::new(SyncStatus::Synced))) .layer(Extension(p2p)) .layer(Extension(SharedAttestationPool::default())) + .layer(Extension(payload_pool.clone())) + .layer(Extension(crate::CustodyColumns(Vec::new()))) .layer(Extension(crate::beacon::validator::FeeRecipients::default())) .layer(Extension(engine)); @@ -98,7 +118,7 @@ async fn serve_with_engine( tokio::spawn(async move { axum::serve(listener, router).await }); let client = HttpBeaconNode::new(format!("http://{address}")).unwrap(); - (client, state, network) + (client, network, payload_pool) } /// One slot of an attester's work, in the order `ethlambda validator` does it. @@ -157,7 +177,7 @@ async fn the_validator_client_can_attest_through_this_node() { .unwrap(); // The attestation data is checked by the client itself before it signs. - let data = client.attestation_data(slot).await.unwrap(); + let data = client.attestation_data(slot, ForkName::Fulu).await.unwrap(); let domain = get_domain(&state, DOMAIN_BEACON_ATTESTER, Some(data.target.epoch)); let signing_root = compute_signing_root(data.hash_tree_root(), domain); let data_dto = AttestationDataOutDto::from(&data); @@ -197,14 +217,17 @@ async fn the_validator_client_can_attest_through_this_node() { .iter() .filter(|attestation| attestation.committee_index == duty.committee_index) .count(); + let AggregateKind::Electra(attestation) = aggregate.attestation else { + panic!("a fulu slot's aggregate is electra's container"); + }; let bits = (0..duty.committee_length as usize) - .filter(|&i| aggregate.attestation.aggregation_bits.get(i).unwrap()) + .filter(|&i| attestation.aggregation_bits.get(i).unwrap()) .count(); assert_eq!(bits, voters); - let signed = signed_aggregate(&state, duty.validator_index, aggregate.attestation); + let signed = signed_aggregate(&state, duty.validator_index, attestation); client - .publish_aggregates(ForkName::Fulu, &[signed]) + .publish_aggregates(ForkName::Fulu, &SignedAggregates::Electra(vec![signed])) .await .unwrap(); assert_eq!(network.aggregates.lock().unwrap().len(), 1); @@ -250,6 +273,7 @@ async fn block_production_without_an_execution_client_is_retryable() { let (client, state, _) = serve().await; let request = BlockRequest { slot: state.slot() + 1, + fork: ForkName::Fulu, proposer_index: 0, randao_reveal: Default::default(), graffiti: Default::default(), @@ -260,19 +284,63 @@ async fn block_production_without_an_execution_client_is_retryable() { } /// A stand-in execution client: `forkchoiceUpdated` with attributes answers a -/// payload id, and `getPayloadV5` a payload that extends the requested head -/// with exactly the requested attributes, which is all a real one's payload -/// has to agree with for the block to verify. +/// payload id, and `getPayload` a payload that extends the requested head with +/// exactly the requested attributes, which is all a real one's payload has to +/// agree with for the block to verify. Answers both generations: V3/V5 for +/// fulu and V4/V6 (`PayloadAttributesV4`, `ExecutionPayloadV4`) for gloas. async fn fake_execution_client() -> ethlambda_engine::EngineClient { + fake_execution_client_with_blob(None).await +} + +/// [`fake_execution_client`] whose gloas payloads carry `blob`, with its real +/// commitment and cell proofs, in the blobs bundle. +async fn fake_execution_client_with_blob(blob: Option>) -> ethlambda_engine::EngineClient { use axum::{Json, routing::post}; + use ethlambda_state_transition::beacon::kzg::{ + blob_to_kzg_commitment, compute_cells_and_kzg_proofs, + }; use std::sync::Mutex; + let bundle = match &blob { + None => serde_json::json!({ "commitments": [], "proofs": [], "blobs": [] }), + Some(blob) => { + let commitment = blob_to_kzg_commitment(blob).unwrap(); + let (_, proofs) = compute_cells_and_kzg_proofs(blob).unwrap(); + serde_json::json!({ + "commitments": [encode_hex(&commitment.0)], + "proofs": proofs.iter().map(|proof| encode_hex(&proof.0)).collect::>(), + "blobs": [encode_hex(blob)], + }) + } + }; let requested: Arc>> = Arc::default(); let handler = move |Json(request): Json| { let requested = requested.clone(); + let bundle = bundle.clone(); async move { + let payload = |attributes: &serde_json::Value, gas_limit: serde_json::Value| { + serde_json::json!({ + "parentHash": attributes["parentHash"], + "feeRecipient": attributes["suggestedFeeRecipient"], + "stateRoot": format!("0x{}", "00".repeat(32)), + "receiptsRoot": format!("0x{}", "00".repeat(32)), + "logsBloom": format!("0x{}", "00".repeat(256)), + "prevRandao": attributes["prevRandao"], + "blockNumber": "0x1", + "gasLimit": gas_limit, + "gasUsed": "0x0", + "timestamp": attributes["timestamp"], + "extraData": "0x", + "baseFeePerGas": "0x7", + "blockHash": format!("0x{}", "ee".repeat(32)), + "transactions": [], + "withdrawals": attributes["withdrawals"], + "blobGasUsed": "0x0", + "excessBlobGas": "0x0", + }) + }; let result = match request["method"].as_str() { - Some("engine_forkchoiceUpdatedV3") => { + Some("engine_forkchoiceUpdatedV3" | "engine_forkchoiceUpdatedV4") => { let mut attributes = request["params"][1].clone(); attributes["parentHash"] = request["params"][0]["headBlockHash"].clone(); *requested.lock().unwrap() = Some(attributes); @@ -284,31 +352,27 @@ async fn fake_execution_client() -> ethlambda_engine::EngineClient { Some("engine_getPayloadV5") => { let attributes = requested.lock().unwrap().clone().unwrap(); serde_json::json!({ - "executionPayload": { - "parentHash": attributes["parentHash"], - "feeRecipient": attributes["suggestedFeeRecipient"], - "stateRoot": format!("0x{}", "00".repeat(32)), - "receiptsRoot": format!("0x{}", "00".repeat(32)), - "logsBloom": format!("0x{}", "00".repeat(256)), - "prevRandao": attributes["prevRandao"], - "blockNumber": "0x1", - "gasLimit": "0x1c9c380", - "gasUsed": "0x0", - "timestamp": attributes["timestamp"], - "extraData": "0x", - "baseFeePerGas": "0x7", - "blockHash": format!("0x{}", "ee".repeat(32)), - "transactions": [], - "withdrawals": attributes["withdrawals"], - "blobGasUsed": "0x0", - "excessBlobGas": "0x0", - }, + "executionPayload": payload(&attributes, "0x1c9c380".into()), "blockValue": "0x2a", "blobsBundle": { "commitments": [], "proofs": [], "blobs": [] }, "shouldOverrideBuilder": false, "executionRequests": [], }) } + Some("engine_getPayloadV6") => { + let attributes = requested.lock().unwrap().clone().unwrap(); + let mut execution_payload = + payload(&attributes, attributes["targetGasLimit"].clone()); + execution_payload["blockAccessList"] = "0xc0".into(); + execution_payload["slotNumber"] = attributes["slotNumber"].clone(); + serde_json::json!({ + "executionPayload": execution_payload, + "blockValue": "0x2a", + "blobsBundle": bundle, + "shouldOverrideBuilder": false, + "executionRequests": [], + }) + } _ => serde_json::Value::Null, }; Json(serde_json::json!({ "jsonrpc": "2.0", "id": request["id"], "result": result })) @@ -352,20 +416,542 @@ async fn the_validator_client_can_propose_through_this_node() { let request = BlockRequest { slot, + fork: ForkName::Fulu, proposer_index: proposer, randao_reveal, graffiti: Default::default(), }; let produced = client.produce_block(&request).await.unwrap(); - assert_eq!(produced.block().slot, slot); - assert_eq!(produced.block().proposer_index, proposer); + assert_eq!(produced.slot(), slot); + assert_eq!(produced.proposer_index(), proposer); let block_domain = get_domain(&advanced, DOMAIN_BEACON_PROPOSER, Some(epoch)); let signature = sign_for( proposer as usize, - compute_signing_root(produced.block().hash_tree_root(), block_domain), + compute_signing_root(produced.block_root(), block_domain), ); let body = produced.into_signed_ssz(signature); client.publish_block(ForkName::Fulu, &body).await.unwrap(); assert_eq!(network.blocks.lock().unwrap().len(), 1); } + +// --------------------------------------------------------------------------- +// Gloas +// --------------------------------------------------------------------------- + +mod gloas { + use std::time::Duration; + + use ethlambda_state_transition::beacon::{ + block_production::advance_to_slot, + bls, + constants::{DOMAIN_BEACON_BUILDER, DOMAIN_BEACON_PROPOSER, DOMAIN_PTC_ATTESTER}, + fork_choice::PayloadStatus, + gloas_block_production::test_support::{config, parent_state, post_state}, + helpers::{ + accessors::get_beacon_proposer_index, + gloas::{compute_ptc, get_ptc}, + test_state::secret_key_for, + }, + }; + use ethlambda_types::beacon::{ + constants::BUILDER_INDEX_SELF_BUILD, + containers::{SignedAggregateAndProof, SignedBeaconBlock, gloas as gloas_containers}, + primitives::Bytes32, + }; + use ethlambda_validator::{ + beacon_node::dto::{ProposerDutyDto, PtcDutyDto}, + keys::ValidatorStore, + payload_attestation::PayloadAttestationService, + proposal::ProposalService, + signing::SigningContext, + }; + use tokio::sync::RwLock; + + use super::*; + use crate::test_utils::{beacon_store_with_head_block, gloas_beacon_block}; + + /// A gloas head state at slot 32 under [`config`] (gloas from epoch 0), + /// with the proposer lookahead and sync committee a real registry gives it + /// and every `ptc_window` entry computed, so the committees the node reads + /// for the previous, current and next epoch are the real ones. + fn head_state() -> BeaconState { + let mut state = parent_state(); + let epoch = compute_epoch_at_slot(state.slot()); + let first_slot = compute_start_slot_at_epoch(epoch - 1); + let cache = + ethlambda_state_transition::beacon::helpers::accessors::CommitteeCache::default(); + let window: Vec<_> = (first_slot..first_slot + preset::PTC_WINDOW_LENGTH as u64) + .map(|slot| compute_ptc(&state, slot, &cache).unwrap()) + .collect(); + let BeaconState::Gloas(inner) = &mut state else { + unreachable!("built as gloas") + }; + for (index, committee) in window.into_iter().enumerate() { + inner.ptc_window[index] = committee; + } + // The fork the schedule puts the state in, so a signature made from + // this client's schedule verifies against the state's own domain: the + // builders leave `fork` zeroed, which no real state has. + let schedule = config(); + inner.fork = ethlambda_types::beacon::containers::shared::Fork { + previous_version: schedule.fork_version(ForkName::Fulu), + current_version: schedule.fork_version(ForkName::Gloas), + epoch: schedule.gloas_fork_epoch, + }; + state + } + + struct Served { + client: HttpBeaconNode, + state: BeaconState, + network: Arc, + payload_pool: SharedPayloadAttestationPool, + store: Store, + head_root: ethlambda_types::primitives::H256, + context: Arc, + } + + /// The gloas head state served over HTTP, its head block a gloas block of + /// the state's slot, on a clock where `clock_slot` is running now. + async fn serve_gloas( + clock_slot: u64, + engine: Option, + ) -> Served { + let state = head_state(); + let block = gloas_beacon_block( + state.slot(), + Default::default(), + Default::default(), + Bytes32::repeat_byte(1), + ); + // The root a block built on this state names as its parent: the + // state's latest header, its state root filled in as the slot + // transition does. + let mut header = state.latest_block_header().clone(); + header.state_root = state.hash_tree_root(); + let head_root = header.hash_tree_root(); + let store = + beacon_store_with_head_block(state.clone(), config(), block, head_root, clock_slot); + let (client, network, payload_pool) = spawn_server(store.clone(), engine).await; + let genesis = client.genesis().await.expect("genesis"); + let spec = client + .spec() + .await + .expect("the client reads this node's spec"); + let context = Arc::new(SigningContext { + config: spec, + genesis_validators_root: genesis.genesis_validators_root, + }); + Served { + client, + state, + network, + payload_pool, + store, + head_root, + context, + } + } + + /// A key store holding the test registry's secret for each of `indices`. + fn keys_for(state: &BeaconState, indices: &[u64]) -> RwLock { + let mut keys = ValidatorStore::new(); + for &index in indices { + let pubkey = keys + .insert_secret("test", &secret_key_for(index as usize).to_bytes()) + .expect("the test secret is a valid key"); + assert_eq!( + pubkey, + state.validator(index).unwrap().pubkey, + "the test registry's key for validator {index}" + ); + } + RwLock::new(keys) + } + + /// One slot of a gloas attester's work: the data comes back without a + /// committee index and names the payload signal, the gloas header goes out + /// with the votes, and the aggregate is the gloas container. + #[tokio::test] + async fn the_validator_client_can_attest_and_aggregate_at_a_gloas_slot() { + let Served { + client, + state, + network, + store, + head_root, + .. + } = serve_gloas(32, None).await; + let slot = state.slot(); + let epoch = compute_epoch_at_slot(slot); + assert_eq!( + client.spec().await.unwrap().fork_at_epoch(epoch), + ForkName::Gloas + ); + + let indices: Vec = (0..COUNT as u64).collect(); + let attesters = client.attester_duties(epoch, &indices).await.unwrap(); + assert_eq!(attesters.duties.len(), COUNT); + + // The attested block is the one of the requested slot: the payload + // signal is 0. + let data = client + .attestation_data(slot, ForkName::Gloas) + .await + .unwrap(); + assert_eq!(data.slot, slot); + assert_eq!(data.index, 0, "a same-slot vote carries no payload signal"); + // A vote for the next slot attests to the head block, now a slot + // old: its signal is whether fork choice holds that block's payload + // as full, and the client passes it on unchanged because it is signed. + let later = client + .attestation_data(slot + 1, ForkName::Gloas) + .await + .unwrap(); + assert_eq!(later.beacon_block_root, head_root); + assert_eq!(later.index, 0, "no envelope was seen for the head block"); + store.set_head_payload_status(head_root, PayloadStatus::Full); + let later = client + .attestation_data(slot + 1, ForkName::Gloas) + .await + .unwrap(); + assert_eq!(later.index, 1); + let domain = get_domain(&state, DOMAIN_BEACON_ATTESTER, Some(data.target.epoch)); + let signing_root = compute_signing_root(data.hash_tree_root(), domain); + let data_dto = AttestationDataOutDto::from(&data); + let attestations: Vec = attesters + .duties + .iter() + .filter(|duty| duty.slot == slot) + .map(|duty| SingleAttestationDto { + committee_index: duty.committee_index, + attester_index: duty.validator_index, + data: data_dto.clone(), + signature: encode_hex(&sign_for(duty.validator_index as usize, signing_root).0), + }) + .collect(); + assert!(!attestations.is_empty(), "someone attests at the head slot"); + + let accepted = client + .submit_attestations(&attestations, "gloas") + .await + .unwrap(); + assert_eq!(accepted, attestations.len()); + { + let published = network.published.lock().unwrap(); + assert_eq!(published.len(), attestations.len()); + for (_, attestation) in published.iter() { + assert_eq!(attestation.data, data); + } + } + + // Aggregation, as the first committee's aggregator would. + let duty = attesters + .duties + .iter() + .find(|duty| duty.slot == slot) + .expect("someone attests at the head slot"); + let aggregate = client + .aggregate_attestation(slot, data.hash_tree_root(), duty.committee_index) + .await + .unwrap(); + assert_eq!(aggregate.fork, ForkName::Gloas); + let AggregateKind::Gloas(attestation) = aggregate.attestation else { + panic!("a gloas slot's aggregate is gloas's container"); + }; + let voters = attestations + .iter() + .filter(|attestation| attestation.committee_index == duty.committee_index) + .count(); + let bits = (0..duty.committee_length as usize) + .filter(|&i| attestation.aggregation_bits.get(i).unwrap_or(false)) + .count(); + assert_eq!(bits, voters); + + let signed = signed_gloas_aggregate(&state, duty.validator_index, attestation); + client + .publish_aggregates( + ForkName::Gloas, + &SignedAggregates::Gloas(vec![signed.clone()]), + ) + .await + .unwrap(); + let recorded = network.aggregates.lock().unwrap(); + assert_eq!(recorded.len(), 1); + assert_eq!(recorded[0], SignedAggregateAndProof::Gloas(signed)); + } + + /// [`super::signed_aggregate`] for gloas, whose `Attestation` roots + /// differently: the proof is signed over the gloas container. + fn signed_gloas_aggregate( + state: &BeaconState, + aggregator: u64, + aggregate: gloas_containers::Attestation, + ) -> gloas_containers::SignedAggregateAndProof { + use ethlambda_types::beacon::constants::{ + DOMAIN_AGGREGATE_AND_PROOF, DOMAIN_SELECTION_PROOF, + }; + let slot = aggregate.data.slot; + let epoch = compute_epoch_at_slot(slot); + let selection_domain = get_domain(state, DOMAIN_SELECTION_PROOF, Some(epoch)); + let selection_proof = sign_for( + aggregator as usize, + compute_signing_root(slot.hash_tree_root(), selection_domain), + ); + let message = gloas_containers::AggregateAndProof { + aggregator_index: aggregator, + aggregate, + selection_proof, + }; + let domain = get_domain(state, DOMAIN_AGGREGATE_AND_PROOF, Some(epoch)); + let signature = sign_for( + aggregator as usize, + compute_signing_root(message.hash_tree_root(), domain), + ); + gloas_containers::SignedAggregateAndProof { message, signature } + } + + /// The committee's duties are the state's, no vote exists for a slot with + /// no block, and the client's own votes (signed with its own key store and + /// fork schedule) are accepted, gossiped and pooled. + #[tokio::test] + async fn the_validator_client_can_serve_on_the_payload_timeliness_committee() { + let Served { + client, + state, + network, + payload_pool, + head_root, + context, + .. + } = serve_gloas(32, None).await; + let slot = state.slot(); + let epoch = compute_epoch_at_slot(slot); + let config = config(); + + // The duties are the first seat each validator holds in the epoch's + // committees. + let mut expected = std::collections::BTreeMap::new(); + for epoch_slot in compute_start_slot_at_epoch(epoch)..compute_start_slot_at_epoch(epoch + 1) + { + for member in get_ptc(&state, epoch_slot, &config).unwrap().iter() { + expected.entry(*member).or_insert(epoch_slot); + } + } + let indices: Vec = (0..COUNT as u64).collect(); + let duties = client.ptc_duties(epoch, &indices).await.unwrap(); + let served: std::collections::BTreeMap = duties + .duties + .iter() + .map(|duty| (duty.validator_index, duty.slot)) + .collect(); + assert_eq!(served, expected); + for duty in &duties.duties { + assert_eq!( + duty.pubkey, + encode_hex(&state.validator(duty.validator_index).unwrap().pubkey.0) + ); + } + // An epoch before gloas has no committee; this schedule has none. + let next = client.ptc_duties(epoch + 1, &indices).await.unwrap(); + assert!( + !next.duties.is_empty(), + "the next epoch's window is readable" + ); + + // No block of the next slot, so no vote to cast; the head's own slot + // has one, naming the head block. + assert!( + client + .payload_attestation_data(slot + 1) + .await + .unwrap() + .is_none() + ); + let data = client + .payload_attestation_data(slot) + .await + .unwrap() + .expect("the head block's slot has data"); + assert_eq!(data.beacon_block_root, head_root); + assert_eq!(data.slot, slot); + assert!(!data.payload_present, "no envelope was seen"); + assert!(data.blob_data_available, "the bid commits to no blobs"); + + // The real service: members of the slot's committee, signing with the + // client's own schedule. + let members: Vec = duties + .duties + .iter() + .filter(|duty| duty.slot == slot) + .map(|duty| duty.validator_index) + .take(3) + .collect(); + assert!( + !members.is_empty(), + "someone sits on the head slot's committee" + ); + let keys = keys_for(&state, &members); + let ptc_duties: Vec = duties + .duties + .iter() + .filter(|duty| members.contains(&duty.validator_index)) + .cloned() + .collect(); + let service = PayloadAttestationService::new(Arc::new(client), context); + let accepted = service.attest(slot, &ptc_duties, &keys).await.unwrap(); + assert_eq!(accepted, members.len()); + + // Recorded for gossip, pooled, and signed so that it verifies under the + // state's own DOMAIN_PTC_ATTESTER. + let recorded = network.payload_attestations.lock().unwrap().clone(); + assert_eq!(recorded.len(), members.len()); + let domain = get_domain(&state, DOMAIN_PTC_ATTESTER, Some(epoch)); + let signing_root = compute_signing_root(data.hash_tree_root(), domain); + for message in &recorded { + assert!(members.contains(&message.validator_index)); + assert_eq!(message.data, data); + let pubkey = state.validator(message.validator_index).unwrap().pubkey; + assert!(bls::verify(&pubkey, signing_root, &message.signature)); + } + let pooled = payload_pool.lock().unwrap().all(Some(slot)); + assert_eq!(pooled.len(), members.len()); + for message in &recorded { + assert!(pooled.contains(message)); + } + } + + /// Insert the first block the node publishes into `store`, with its + /// post-state, as the chain actor would on importing it: nothing in this + /// harness does, and the envelope endpoint waits for the block. + fn import_published_block( + mut store: Store, + network: Arc, + parent: BeaconState, + ) { + tokio::spawn(async move { + loop { + let published = network.blocks.lock().unwrap().first().cloned(); + if let Some(SignedBeaconBlock::Gloas(signed)) = published { + let root = signed.message.hash_tree_root(); + let post = post_state(&parent, &signed.message); + store + .insert_signed_block(root, SignedBeaconBlock::Gloas(signed)) + .unwrap(); + store.insert_state(root, post).unwrap(); + return; + } + tokio::time::sleep(Duration::from_millis(10)).await; + } + }); + } + + /// A proposer's slot at gloas, through `ProposalService`: produceBlockV4 + /// with the payload, the block signed and published, the envelope signed + /// and published. What the network was handed must verify against the + /// state, with the proposer's key. + async fn propose_through_the_service(blob: Option>) { + let engine = fake_execution_client_with_blob(blob.clone()).await; + let Served { + client, + state, + network, + store, + context, + .. + } = serve_gloas(33, Some(engine)).await; + let slot = state.slot() + 1; + let epoch = compute_epoch_at_slot(slot); + let advanced = advance_to_slot(&state, slot, &config()).unwrap(); + let proposer = get_beacon_proposer_index(&advanced).unwrap(); + let keys = keys_for(&state, &[proposer]); + let pubkey = state.validator(proposer).unwrap().pubkey; + import_published_block(store, network.clone(), advanced.clone()); + + let service = + ProposalService::new(Arc::new(client), context, Bytes32::repeat_byte(0xab), None); + let duty = ProposerDutyDto { + pubkey: encode_hex(&pubkey.0), + validator_index: proposer, + slot, + }; + service + .propose(slot, &duty, &keys) + .await + .expect("the proposal goes through this node"); + + // The block: gloas, the proposer's, and signed under the state's + // proposer domain. + let blocks = network.blocks.lock().unwrap().clone(); + assert_eq!(blocks.len(), 1); + let SignedBeaconBlock::Gloas(signed_block) = &blocks[0] else { + panic!("a gloas slot publishes a gloas block"); + }; + let block = &signed_block.message; + assert_eq!(block.slot, slot); + assert_eq!(block.proposer_index, proposer); + let block_root = block.hash_tree_root(); + let block_domain = get_domain(&advanced, DOMAIN_BEACON_PROPOSER, Some(epoch)); + assert!(bls::verify( + &pubkey, + compute_signing_root(block_root, block_domain), + &signed_block.signature + )); + + // The envelope: for this block, a self-build, matching the bid it + // reveals, and signed with the proposer's key under the builder domain. + let envelopes = network.envelopes.lock().unwrap().clone(); + assert_eq!(envelopes.len(), 1, "the envelope reached the network"); + let (signed_envelope, sidecars) = &envelopes[0]; + let envelope = &signed_envelope.message; + let bid = &block.body.signed_execution_payload_bid.message; + assert_eq!(bid.builder_index, BUILDER_INDEX_SELF_BUILD); + assert_eq!(envelope.builder_index, BUILDER_INDEX_SELF_BUILD); + assert_eq!(envelope.beacon_block_root, block_root); + assert_eq!(envelope.payload.block_hash, bid.block_hash); + assert_eq!(envelope.payload.parent_hash, bid.parent_block_hash); + assert_eq!(envelope.payload.gas_limit, bid.gas_limit); + assert_eq!(envelope.payload.slot_number, slot); + assert_eq!( + envelope.execution_requests.hash_tree_root(), + bid.execution_requests_root + ); + let builder_domain = get_domain(&advanced, DOMAIN_BEACON_BUILDER, Some(epoch)); + assert!(bls::verify( + &pubkey, + compute_signing_root(envelope.hash_tree_root(), builder_domain), + &signed_envelope.signature + )); + + // With a blob: the bid commits to it and every column goes out with the + // envelope (`Eth-Blob-Data-Included: true` carried the blobs). + match blob { + None => { + assert!(bid.blob_kzg_commitments.is_empty()); + assert!(sidecars.is_empty()); + } + Some(blob) => { + let expected = + ethlambda_state_transition::beacon::kzg::blob_to_kzg_commitment(&blob).unwrap(); + assert_eq!(bid.blob_kzg_commitments.len(), 1); + assert_eq!(bid.blob_kzg_commitments[0], expected); + assert_eq!(sidecars.len(), preset::NUMBER_OF_COLUMNS); + } + } + } + + #[tokio::test] + async fn the_validator_client_can_propose_and_reveal_a_gloas_block() { + propose_through_the_service(None).await; + } + + #[tokio::test] + async fn the_validator_client_can_propose_a_gloas_block_with_a_blob() { + let mut blob = vec![0u8; preset::BYTES_PER_BLOB]; + for (i, element) in blob.chunks_mut(32).enumerate() { + element[31] = (i % 100) as u8; + } + propose_through_the_service(Some(blob)).await; + } +} diff --git a/crates/net/rpc/src/lib.rs b/crates/net/rpc/src/lib.rs index 7958cbc7..e03ae786 100644 --- a/crates/net/rpc/src/lib.rs +++ b/crates/net/rpc/src/lib.rs @@ -3,7 +3,9 @@ use std::net::{IpAddr, SocketAddr}; use axum::{Extension, Router}; use ethlambda_blockchain::{EventBus, SyncStatusController}; use ethlambda_network_api::RpcToP2PRef; -use ethlambda_state_transition::beacon::attestation_pool::SharedAttestationPool; +use ethlambda_state_transition::beacon::{ + attestation_pool::SharedAttestationPool, payload_attestation_pool::SharedPayloadAttestationPool, +}; use ethlambda_storage::Store; use ethlambda_types::aggregator::AggregatorController; use tokio_util::sync::CancellationToken; @@ -184,6 +186,11 @@ pub fn build_beacon_api_router(store: Store, version: &'static str, peer_id: Str .with_state(store) } +/// The data columns this node custodies, as the chain actor's availability +/// gate reads them. Empty on a node with nothing to custody. +#[derive(Clone, Debug, Default)] +pub struct CustodyColumns(pub Vec); + /// What the Beacon API's validator endpoints reach beyond the store. pub struct BeaconApiHandles { /// Through which the pool, aggregate and block endpoints gossip what a @@ -192,6 +199,14 @@ pub struct BeaconApiHandles { /// Filled by the attestation pool endpoint and the aggregator subnets, /// read by the aggregate endpoint and block production. pub attestation_pool: SharedAttestationPool, + /// Filled by gossip and the payload attestation pool endpoint, read by + /// block production and the pool's GET. + pub payload_attestation_pool: SharedPayloadAttestationPool, + /// The columns this node custodies: what `payload_attestation_data` checks + /// a block's blob availability against, and what block production tells + /// the execution client it samples for when asking it to build a gloas + /// payload. + pub custody_columns: CustodyColumns, /// The execution client block production builds payloads with; `None` /// makes it answer 503. pub engine: Option, @@ -215,6 +230,8 @@ pub async fn start_beacon_rpc_server( .layer(Extension(sync_status)) .layer(Extension(handles.p2p)) .layer(Extension(handles.attestation_pool)) + .layer(Extension(handles.payload_attestation_pool)) + .layer(Extension(handles.custody_columns)) .layer(Extension(beacon::validator::FeeRecipients::default())) .layer(Extension(handles.engine)); start_http_servers(config, Some(api_router), shutdown).await @@ -417,6 +434,15 @@ pub(crate) mod test_utils { pub(crate) subscriptions: std::sync::Mutex>, pub(crate) blocks: std::sync::Mutex>, + pub(crate) envelopes: std::sync::Mutex< + Vec<( + ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope, + Vec, + )>, + >, + pub(crate) payload_attestations: std::sync::Mutex< + Vec, + >, } impl ethlambda_network_api::RpcToP2P for RecordingNetwork { @@ -455,6 +481,25 @@ pub(crate) mod test_utils { self.blocks.lock().unwrap().push(block); Ok(()) } + + fn publish_execution_payload_envelope( + &self, + envelope: Box< + ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope, + >, + sidecars: Vec, + ) -> Result<(), spawned_concurrency::error::ActorError> { + self.envelopes.lock().unwrap().push((*envelope, sidecars)); + Ok(()) + } + + fn publish_payload_attestation_message( + &self, + message: ethlambda_types::beacon::containers::gloas::PayloadAttestationMessage, + ) -> Result<(), spawned_concurrency::error::ActorError> { + self.payload_attestations.lock().unwrap().push(message); + Ok(()) + } } /// [`beacon_fixture`]'s chain with three gloas blocks on top of its head. @@ -604,6 +649,51 @@ pub(crate) mod test_utils { (store, root) } + /// A beacon store whose head is `block` (a block at `state`'s slot) with + /// `state` as its post-state, on a clock whose slot `clock_slot` began a + /// second ago. + /// + /// For endpoints that read the head block itself (the payload attestation + /// data names it) or that only accept the current slot (gossip-checked + /// votes), which [`beacon_store_at`]'s phase0 anchor on mainnet's real + /// clock cannot serve. + /// + /// The block is stored under `root`, which the caller names because a test + /// state's `latest_block_header` need not commit to the block built to + /// stand in for it, and a block built on the state names that header's root + /// as its parent. + pub(crate) fn beacon_store_with_head_block( + state: BeaconState, + config: Config, + block: SignedBeaconBlock, + root: H256, + clock_slot: u64, + ) -> Store { + let slot = state.slot(); + let slot_secs = config.slot_duration_ms / 1000; + let now_secs = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("the clock is after the epoch") + .as_secs(); + let genesis = now_secs - clock_slot * slot_secs - 1; + let mut store = Store::init_beacon( + Arc::new(InMemoryBackend::default()), + genesis, + config, + root, + Checkpoint { root, slot }, + slot, + ); + store + .insert_signed_block(root, block) + .expect("insert head block"); + store.insert_state(root, state).expect("insert head state"); + store + .update_checkpoints(ForkCheckpoints::head_only(root)) + .expect("make the block the head"); + store + } + /// Build a beacon store anchored at `anchor_slot`, with a real child block /// at `anchor_slot + 1` whose import moves the head for real. /// diff --git a/crates/storage/src/store.rs b/crates/storage/src/store.rs index 49078d61..025998d6 100644 --- a/crates/storage/src/store.rs +++ b/crates/storage/src/store.rs @@ -784,6 +784,15 @@ pub(crate) struct BeaconScratch { /// specification's `block_payload_statuses`; an absent root reads as /// `NOT_VALIDATED`. pub(crate) block_payload_statuses: HashMap, + /// Gloas: beacon root to `(slot, milliseconds past genesis the block's + /// envelope reached this node)`. What the payload timeliness committee's + /// `payload_present` is decided from: the specification's vote is "an + /// envelope was seen before `get_payload_due_ms()` into the slot", and + /// nothing else this store keeps records *when*. A cache like + /// `el_block_hashes`, not chain history, and pruned with it by + /// `prune_beacon_el_block_hashes`; a restart forgets it, which can only + /// turn a late `payload_present` into `false` for the one slot in flight. + pub(crate) envelope_seen_ms: HashMap, } /// Encode a LiveChain key (slot, root) to bytes. @@ -3944,8 +3953,34 @@ impl Store { .insert(root, (slot, block_hash)); } + /// Records when `root`'s envelope reached this node, in milliseconds past + /// genesis. The earliest arrival wins: a later copy says nothing about + /// whether the envelope was on time. + pub fn insert_beacon_envelope_seen(&mut self, root: H256, slot: u64, seen_ms: u64) { + self.beacon + .lock() + .unwrap() + .envelope_seen_ms + .entry(root) + .and_modify(|(_, held)| *held = (*held).min(seen_ms)) + .or_insert((slot, seen_ms)); + } + + /// When `root`'s envelope reached this node, in milliseconds past genesis, + /// if it has. + pub fn beacon_envelope_seen_ms(&self, root: &H256) -> Option { + self.beacon + .lock() + .unwrap() + .envelope_seen_ms + .get(root) + .map(|(_, seen_ms)| *seen_ms) + } + /// Drops cached hashes strictly below `finalized_slot`, always keeping - /// `keep`. + /// `keep`. The envelope arrival times go with them, below the same bound + /// (they are only ever read for a slot in flight, so `keep` needs no + /// exemption there). /// /// Strictly below, not at or below: the justified and head blocks /// `forkchoiceUpdated` reads are at or above that slot, so this bound keeps @@ -3967,6 +4002,11 @@ impl Store { .unwrap() .el_block_hashes .retain(|root, (slot, _hash)| *slot >= finalized_slot || *root == keep); + self.beacon + .lock() + .unwrap() + .envelope_seen_ms + .retain(|_, (slot, _)| *slot >= finalized_slot); } /// Returns `root`'s unrealized justification, if this store has computed @@ -7740,6 +7780,24 @@ mod tests { ); } + #[test] + fn envelope_arrival_keeps_the_earliest_and_is_pruned_with_the_hash_cache() { + let mut store = Store::test_store(); + let old = H256::repeat_byte(1); + let fresh = H256::repeat_byte(2); + store.insert_beacon_envelope_seen(old, 4, 100); + store.insert_beacon_envelope_seen(fresh, 6, 900); + store.insert_beacon_envelope_seen(fresh, 6, 2_000); + store.insert_beacon_envelope_seen(fresh, 6, 500); + + assert_eq!(store.beacon_envelope_seen_ms(&fresh), Some(500)); + assert_eq!(store.beacon_envelope_seen_ms(&H256::repeat_byte(9)), None); + + store.prune_beacon_el_block_hashes(5, H256::repeat_byte(7)); + assert_eq!(store.beacon_envelope_seen_ms(&old), None); + assert_eq!(store.beacon_envelope_seen_ms(&fresh), Some(500)); + } + #[test] fn el_block_hashes_prune_strictly_below_the_finalized_slot() { let mut store = Store::test_store(); diff --git a/crates/validator/src/aggregation.rs b/crates/validator/src/aggregation.rs index 493ae187..3a0a32dc 100644 --- a/crates/validator/src/aggregation.rs +++ b/crates/validator/src/aggregation.rs @@ -35,7 +35,8 @@ use std::collections::BTreeMap; use std::sync::Arc; -use ethlambda_types::beacon::containers::electra::{AggregateAndProof, SignedAggregateAndProof}; +use ethlambda_types::beacon::containers::electra; +use ethlambda_types::beacon::containers::gloas; use ethlambda_types::beacon::containers::shared::AttestationData; use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::primitives::{HashTreeRoot as _, Slot}; @@ -43,12 +44,19 @@ use tokio::sync::RwLock; use tracing::{info, warn}; use crate::aggregation_selection::selection_for; -use crate::beacon_node::BeaconNodeApi; use crate::beacon_node::dto::{AttesterDutyDto, parse_pubkey}; +use crate::beacon_node::{AggregateKind, BeaconNodeApi, SignedAggregates}; use crate::error::{Error, Result}; use crate::keys::ValidatorStore; use crate::signing::SigningContext; +/// An `AggregateAndProof` of either fork, held between building it and +/// signing it. +enum Wrapper { + Electra(electra::AggregateAndProof), + Gloas(gloas::AggregateAndProof), +} + pub struct AggregationService { beacon_node: Arc, context: Arc, @@ -155,7 +163,8 @@ impl AggregationService { let signed = { let store = store.read().await; - let mut signed = Vec::with_capacity(selected.len()); + let mut electra_signed = Vec::new(); + let mut gloas_signed = Vec::new(); for (duty, selection_proof) in &selected { let Some(aggregate) = aggregates.get(&duty.committee_index) else { continue; @@ -168,15 +177,31 @@ impl AggregationService { } }; - let message = AggregateAndProof { - aggregator_index: duty.validator_index, - aggregate: aggregate.attestation.clone(), - selection_proof: *selection_proof, + // The wrapper is the fork's own container, and its root is taken + // from it: gloas's `Attestation` hashes differently from + // electra's, so a root computed over the wrong one would sign a + // message no node can verify. + let (root, wrapper) = match &aggregate.attestation { + AggregateKind::Electra(attestation) => { + let message = electra::AggregateAndProof { + aggregator_index: duty.validator_index, + aggregate: attestation.clone(), + selection_proof: *selection_proof, + }; + (message.hash_tree_root(), Wrapper::Electra(message)) + } + AggregateKind::Gloas(attestation) => { + let message = gloas::AggregateAndProof { + aggregator_index: duty.validator_index, + aggregate: attestation.clone(), + selection_proof: *selection_proof, + }; + (message.hash_tree_root(), Wrapper::Gloas(message)) + } }; // Signed over the whole wrapper, not over the aggregate: that // is what binds this validator's index and its selection proof // to the votes it is republishing. - let root = message.hash_tree_root(); let signature = match self .context .sign_aggregate_and_proof(&store, &pubkey, root, slot) @@ -187,9 +212,22 @@ impl AggregationService { continue; } }; - signed.push(SignedAggregateAndProof { message, signature }); + match wrapper { + Wrapper::Electra(message) => { + electra_signed.push(electra::SignedAggregateAndProof { message, signature }) + } + Wrapper::Gloas(message) => { + gloas_signed.push(gloas::SignedAggregateAndProof { message, signature }) + } + } + } + // Every aggregate in a batch is one fork (see `batch_fork`), so only + // one of the two lists is ever filled. + if fork >= ForkName::Gloas { + SignedAggregates::Gloas(gloas_signed) + } else { + SignedAggregates::Electra(electra_signed) } - signed }; if signed.is_empty() { @@ -491,4 +529,80 @@ mod tests { assert_eq!(published, 0); assert!(node.aggregate_requests().is_empty()); } + + fn gloas_context() -> Arc { + Arc::new(SigningContext { + config: Config::mainnet().with_fork_epoch(ForkName::Gloas, 100), + genesis_validators_root: Root::ZERO, + }) + } + + /// A slot in the gloas era at which this key is selected under the gloas + /// fork version, found rather than written down for the reason `slots` is. + fn gloas_selected_slot(pubkey: &BlsPubkey, store: &ValidatorStore) -> Slot { + let context = gloas_context(); + (3_200..3_600) + .find(|slot| { + let proof = context + .sign_selection_proof(store, pubkey, *slot) + .expect("signs"); + is_aggregator(128, &proof) + }) + .expect("some slot selects this key") + } + + /// At a gloas slot the aggregate is gloas's container, and the wrapper is + /// signed over the gloas `AggregateAndProof`'s root, not electra's: the two + /// attestations hash differently, so the wrong root signs a message no node + /// can verify. + #[tokio::test] + async fn a_gloas_aggregate_is_wrapped_signed_and_published_as_gloas() { + use blst::min_pk::{PublicKey, Signature}; + + let (store, pubkey) = store(); + let slot = { + let guard = store.read().await; + gloas_selected_slot(&pubkey, &guard) + }; + let node = Arc::new(MockBeaconNode::new().with_gloas_aggregate(data(slot))); + + let published = AggregationService::new(node.clone(), gloas_context()) + .aggregate(slot, &data(slot), &[duty(&pubkey, 1, slot, 2)], &store) + .await + .expect("aggregates"); + assert_eq!(published, 1); + + assert!( + node.published_aggregates().is_empty(), + "not the electra form" + ); + let (fork, list) = node.published_gloas_aggregates().remove(0); + assert_eq!(fork, ForkName::Gloas); + let entry = &list[0]; + assert_eq!(entry.message.aggregator_index, 1); + + let root = + gloas_context().aggregate_and_proof_signing_root(entry.message.hash_tree_root(), slot); + let pk = PublicKey::from_bytes(&pubkey.0).expect("valid pubkey"); + let sig = Signature::from_bytes(&entry.signature.0).expect("valid signature"); + assert_eq!( + sig.verify( + true, + root.as_slice(), + b"BLS_SIG_BLS12381G2_XMD:SHA-256_SSWU_RO_POP_", + &[], + &pk, + true + ), + blst::BLST_ERROR::BLST_SUCCESS + ); + + // The same wrapper built from electra's container hashes differently. + let as_electra = electra::AggregateAndProof { + aggregator_index: 1, + aggregate: electra::Attestation::try_from(&entry.message.aggregate).expect("converts"), + selection_proof: entry.message.selection_proof, + }; + assert_ne!(as_electra.hash_tree_root(), entry.message.hash_tree_root()); + } } diff --git a/crates/validator/src/attestation.rs b/crates/validator/src/attestation.rs index c274f2bf..a5984e59 100644 --- a/crates/validator/src/attestation.rs +++ b/crates/validator/src/attestation.rs @@ -4,6 +4,7 @@ use std::sync::Arc; use ethlambda_types::beacon::containers::shared::AttestationData; use ethlambda_types::beacon::primitives::Slot; +use ethlambda_types::beacon::signing::compute_epoch_at_slot; use tokio::sync::RwLock; use tracing::{error, info, warn}; @@ -105,7 +106,15 @@ impl AttestationService { }); } - let data = self.beacon_node.attestation_data(slot).await?; + // The fork of the slot, by this client's own schedule, tells the node + // which form of the query to answer: from gloas the `committee_index` + // is omitted and the answer's `index` is the payload signal. It is + // signed as received, never reset to zero. + let slot_fork = self + .context + .config + .fork_at_epoch(compute_epoch_at_slot(slot)); + let data = self.beacon_node.attestation_data(slot, slot_fork).await?; // Checked again here, having already been checked by the // implementation this call went through (see the contract on @@ -651,4 +660,94 @@ mod tests { Some("altair") ); } + + fn gloas_context() -> Arc { + Arc::new(SigningContext { + config: Config::mainnet() + .with_fork_epoch(ethlambda_types::beacon::fork::ForkName::Gloas, 10), + genesis_validators_root: Root::ZERO, + }) + } + + /// From gloas `data.index` is the payload signal, so a node answering 1 + /// must see it submitted as 1 and signed as 1: resetting it to zero, as + /// the electra rule would suggest, signs a vote the node never made. + #[tokio::test] + async fn a_gloas_index_of_one_is_submitted_and_signed_unchanged() { + use blst::min_pk::{PublicKey, Signature}; + use ethlambda_types::beacon::fork::ForkName; + + let mut store = ValidatorStore::new(); + let pubkey = store.insert_secret("test", &secret()).expect("inserts"); + let pubkey_hex = encode_hex(&pubkey.0); + let store = RwLock::new(store); + + let slot = 10 * preset::SLOTS_PER_EPOCH + 3; + let node = Arc::new( + MockBeaconNode::new() + .with_attestation_data(slot) + .with_attestation_index(1), + ); + let attested = AttestationService::new(node.clone(), gloas_context()) + .attest(slot, &[duty(&pubkey_hex, 1337, slot, 3)], &store) + .await + .expect("attests"); + + assert_eq!(attested.data.expect("data").index, 1); + let submitted = node.submitted(); + assert_eq!(submitted.len(), 1); + assert_eq!(submitted[0].data.index, 1, "the index must pass through"); + assert_eq!(node.last_submitted_fork_name().as_deref(), Some("gloas")); + assert_eq!(node.attestation_data_forks(), vec![ForkName::Gloas]); + + // The signature is over the data with index 1, not over a zeroed copy. + let mut signed_over = node.attestation_data.expect("set"); + assert_eq!(signed_over.index, 1); + let root = gloas_context().attestation_signing_root(&signed_over); + let signature_bytes: [u8; 96] = + hex::decode(submitted[0].signature.trim_start_matches("0x")) + .expect("hex") + .try_into() + .expect("96 bytes"); + let pk = PublicKey::from_bytes(&pubkey.0).expect("valid pubkey"); + let sig = Signature::from_bytes(&signature_bytes).expect("valid signature"); + let verify = |root: Root| { + sig.verify( + true, + root.as_slice(), + b"BLS_SIG_BLS12381G2_XMD:SHA-256_SSWU_RO_POP_", + &[], + &pk, + true, + ) + }; + assert_eq!(verify(root), blst::BLST_ERROR::BLST_SUCCESS); + + signed_over.index = 0; + assert_ne!( + verify(gloas_context().attestation_signing_root(&signed_over)), + blst::BLST_ERROR::BLST_SUCCESS, + "a vote for index 0 is a different message" + ); + } + + /// Slots before gloas are asked about under their own fork, so the HTTP + /// client keeps sending `committee_index=0` there. + #[tokio::test] + async fn a_pre_gloas_slot_is_asked_about_under_its_own_fork() { + use ethlambda_types::beacon::fork::ForkName; + + let mut store = ValidatorStore::new(); + let pubkey = store.insert_secret("test", &secret()).expect("inserts"); + let pubkey_hex = encode_hex(&pubkey.0); + let store = RwLock::new(store); + + let slot = 9 * preset::SLOTS_PER_EPOCH + 3; + let node = Arc::new(MockBeaconNode::new().with_attestation_data(slot)); + AttestationService::new(node.clone(), gloas_context()) + .attest(slot, &[duty(&pubkey_hex, 1337, slot, 3)], &store) + .await + .expect("attests"); + assert_eq!(node.attestation_data_forks(), vec![ForkName::Phase0]); + } } diff --git a/crates/validator/src/beacon_node/block_contents.rs b/crates/validator/src/beacon_node/block_contents.rs index 7dcb4019..c4d2806c 100644 --- a/crates/validator/src/beacon_node/block_contents.rs +++ b/crates/validator/src/beacon_node/block_contents.rs @@ -42,7 +42,18 @@ //! | Fork | Produced | Published | //! |---|---|---| //! | Deneb, Electra, Fulu | `BlockContents` | `SignedBlockContents` | -//! | Gloas and later | bare block; blobs move to the payload envelope | bare signed block | +//! | Gloas and later | `GloasBlockContents` (block and self-built envelope) or a bare block | bare signed block, then the envelope separately | +//! +//! # Gloas +//! +//! The block no longer carries the payload (EIP-7732): it commits to a bid, and +//! the payload travels in an `ExecutionPayloadEnvelope` the builder reveals +//! afterwards. With `include_payload=true` the node hands back the block and the +//! envelope for a self-built payload together as [`GloasBlockContents`], and +//! says so in `Eth-Execution-Payload-Included`; without it the body is a bare +//! block and the envelope is fetched separately. Publication is split the same +//! way: the bare signed block first, then the signed envelope, with the blobs +//! and cell proofs moving into the envelope's contents. //! //! Deneb shares that envelope but not the block inside it: its //! `BeaconBlockBody` has twelve fields where electra's has thirteen, so the two @@ -76,6 +87,7 @@ use ethlambda_types::beacon::containers::deneb::Blob; use ethlambda_types::beacon::containers::electra::{BeaconBlock, SignedBeaconBlock}; +use ethlambda_types::beacon::containers::gloas; use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::preset; use ethlambda_types::beacon::primitives::{BlsSignature, KzgProof}; @@ -142,6 +154,64 @@ pub struct FuluSignedBlockContents { pub blobs: Blobs, } +/// `produceBlockV4`'s body for gloas when the node includes the payload: the +/// block, the envelope revealing the self-built payload it commits to, and the +/// blob material that goes out with the envelope. Not a consensus container; see +/// the module doc. +#[derive(Debug, Clone, PartialEq, SszEncode, SszDecode)] +pub struct GloasBlockContents { + pub block: gloas::BeaconBlock, + pub execution_payload_envelope: gloas::ExecutionPayloadEnvelope, + pub kzg_proofs: CellKzgProofs, + pub blobs: Blobs, +} + +/// `publishExecutionPayloadEnvelope`'s body for gloas when the client supplies +/// the blobs (`Eth-Blob-Data-Included: true`). +#[derive(Debug, Clone, PartialEq, SszEncode, SszDecode)] +pub struct SignedExecutionPayloadEnvelopeContents { + pub signed_execution_payload_envelope: gloas::SignedExecutionPayloadEnvelope, + pub kzg_proofs: CellKzgProofs, + pub blobs: Blobs, +} + +/// The envelope and blob material that came with a gloas block, kept apart from +/// the block because it is published separately and after it. +#[derive(Debug, Clone, PartialEq)] +pub struct GloasPayload { + pub envelope: gloas::ExecutionPayloadEnvelope, + pub kzg_proofs: CellKzgProofs, + pub blobs: Blobs, +} + +impl GloasPayload { + /// Attach `signature` and encode the body for `Eth-Blob-Data-Included: true`. + pub fn into_signed_contents_ssz(self, signature: BlsSignature) -> Vec { + SignedExecutionPayloadEnvelopeContents { + signed_execution_payload_envelope: gloas::SignedExecutionPayloadEnvelope { + message: self.envelope, + signature, + }, + kzg_proofs: self.kzg_proofs, + blobs: self.blobs, + } + .to_ssz() + } +} + +/// Encode a bare signed envelope, the body for `Eth-Blob-Data-Included: false`, +/// where the node attaches the blobs it cached at production time. +pub fn signed_envelope_ssz( + envelope: gloas::ExecutionPayloadEnvelope, + signature: BlsSignature, +) -> Vec { + gloas::SignedExecutionPayloadEnvelope { + message: envelope, + signature, + } + .to_ssz() +} + /// A block a beacon node produced, with whatever travelled alongside it. /// /// The proofs and blobs are kept in the shape they arrived in rather than @@ -180,6 +250,12 @@ pub enum Contents { kzg_proofs: CellKzgProofs, blobs: Blobs, }, + /// Gloas: a bare block, and the self-built payload's envelope when the + /// node included it. + Gloas { + block: gloas::BeaconBlock, + payload: Option>, + }, } impl ProducedBlock { @@ -230,16 +306,18 @@ impl ProducedBlock { }, }) } - // ePBS (EIP-7732) replaces the embedded payload this client - // decodes here with a builder's bid, a shape neither - // `Contents` variant models yet, so this is refused the same - // way a pre-electra fork is rather than mis-decoded. + // Gloas needs one more fact than the fork: whether the node + // included the payload, which only the response header says. See + // `Self::from_gloas_ssz`. + ForkName::Gloas => Err(Error::InconsistentResponse( + "a gloas block cannot be decoded without Eth-Execution-Payload-Included" + .to_string(), + )), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix | ForkName::Capella | ForkName::Deneb - | ForkName::Gloas | ForkName::Lean => Err(Error::InconsistentResponse(format!( "beacon node produced a block for fork {}, which this client does not propose \ under; electra is the earliest supported", @@ -248,21 +326,121 @@ impl ProducedBlock { } } - /// The block itself, whichever wrapper it came in. - pub fn block(&self) -> &BeaconBlock { + /// Decode a gloas `produceBlockV4` body. `payload_included` is the node's + /// `Eth-Execution-Payload-Included`, which selects between + /// [`GloasBlockContents`] and a bare block: SSZ carries no tag, so a wrong + /// guess would decode garbage or fail in a way that names neither. + pub fn from_gloas_ssz(payload_included: bool, bytes: &[u8]) -> crate::error::Result { + let decode = |what: &str, err: libssz::DecodeError| { + Error::Decode(format!( + "produced gloas block: {what} did not decode: {err:?}" + )) + }; + let (block, payload) = if payload_included { + let contents = GloasBlockContents::from_ssz_bytes(bytes) + .map_err(|err| decode("block contents", err))?; + ( + contents.block, + Some(Box::new(GloasPayload { + envelope: contents.execution_payload_envelope, + kzg_proofs: contents.kzg_proofs, + blobs: contents.blobs, + })), + ) + } else { + let block = gloas::BeaconBlock::from_ssz_bytes(bytes) + .map_err(|err| decode("bare block", err))?; + (block, None) + }; + Ok(Self { + fork: ForkName::Gloas, + contents: Contents::Gloas { block, payload }, + }) + } + + /// The block's slot, whichever fork's block it is. + pub fn slot(&self) -> ethlambda_types::beacon::primitives::Slot { + match &self.contents { + Contents::WithBlobProofs { block, .. } | Contents::WithCellProofs { block, .. } => { + block.slot + } + Contents::Gloas { block, .. } => block.slot, + } + } + + /// The proposer the block names. + pub fn proposer_index(&self) -> ethlambda_types::beacon::primitives::ValidatorIndex { + match &self.contents { + Contents::WithBlobProofs { block, .. } | Contents::WithCellProofs { block, .. } => { + block.proposer_index + } + Contents::Gloas { block, .. } => block.proposer_index, + } + } + + /// The block's `hash_tree_root`, taken from the decoded container of + /// whichever fork it is, which is what gets signed. + pub fn block_root(&self) -> ethlambda_types::beacon::primitives::Root { + use ethlambda_types::beacon::primitives::HashTreeRoot as _; + match &self.contents { + Contents::WithBlobProofs { block, .. } | Contents::WithCellProofs { block, .. } => { + block.hash_tree_root() + } + Contents::Gloas { block, .. } => block.hash_tree_root(), + } + } + + /// Where the block pays its execution-layer rewards: the payload's fee + /// recipient before gloas, the bid's from it on. + pub fn fee_recipient(&self) -> ethlambda_types::beacon::primitives::ExecutionAddress { match &self.contents { Contents::WithBlobProofs { block, .. } | Contents::WithCellProofs { block, .. } => { + block.body.execution_payload.fee_recipient + } + Contents::Gloas { block, .. } => { block + .body + .signed_execution_payload_bid + .message + .fee_recipient } } } + /// The builder the gloas bid names, `None` before gloas. The self-build + /// sentinel means this client's own proposer key signs the envelope. + pub fn builder_index(&self) -> Option { + match &self.contents { + Contents::Gloas { block, .. } => Some( + block + .body + .signed_execution_payload_bid + .message + .builder_index, + ), + _ => None, + } + } + + /// Take the envelope and blob material out of a gloas block, leaving the + /// block to be signed and published alone. `None` for every earlier fork + /// and for a gloas block the node returned without its payload. + pub fn take_gloas_payload(&mut self) -> Option { + match &mut self.contents { + Contents::Gloas { payload, .. } => payload.take().map(|payload| *payload), + _ => None, + } + } + /// How many blobs came with the block. For logging. pub fn blob_count(&self) -> usize { match &self.contents { Contents::WithBlobProofs { blobs, .. } | Contents::WithCellProofs { blobs, .. } => { blobs.len() } + Contents::Gloas { payload, .. } => { + payload.as_ref().map_or(0, |payload| payload.blobs.len()) + } } } @@ -299,6 +477,14 @@ impl ProducedBlock { blobs, } .to_ssz(), + // Bare: the envelope and blobs are published on their own, after + // this body, and are expected to have been taken out already by + // `take_gloas_payload`. + Contents::Gloas { block, .. } => gloas::SignedBeaconBlock { + message: block, + signature, + } + .to_ssz(), } } } @@ -372,6 +558,66 @@ pub fn empty_block_for( } } +/// The smallest gloas block for `slot` and `proposer_index`, whose bid names +/// `builder_index`. +#[cfg(test)] +pub fn empty_gloas_block_for( + slot: ethlambda_types::beacon::primitives::Slot, + proposer_index: ethlambda_types::beacon::primitives::ValidatorIndex, + builder_index: gloas::BuilderIndex, +) -> gloas::BeaconBlock { + let mut block = gloas::BeaconBlock { + slot, + proposer_index, + ..Default::default() + }; + block + .body + .signed_execution_payload_bid + .message + .builder_index = builder_index; + block +} + +/// A gloas envelope for the block whose root is `beacon_block_root`, with an +/// otherwise zeroed payload. +#[cfg(test)] +pub fn empty_gloas_envelope_for( + beacon_block_root: ethlambda_types::beacon::primitives::Root, + builder_index: gloas::BuilderIndex, +) -> gloas::ExecutionPayloadEnvelope { + let payload = gloas::ExecutionPayload { + parent_hash: Default::default(), + fee_recipient: Default::default(), + state_root: Default::default(), + receipts_root: Default::default(), + logs_bloom: vec![0u8; preset::BYTES_PER_LOGS_BLOOM] + .try_into() + .expect("a logs bloom is BYTES_PER_LOGS_BLOOM long by construction"), + prev_randao: Default::default(), + block_number: 0, + gas_limit: 0, + gas_used: 0, + timestamp: 0, + extra_data: Default::default(), + base_fee_per_gas: Default::default(), + block_hash: Default::default(), + transactions: Default::default(), + withdrawals: Default::default(), + blob_gas_used: 0, + excess_blob_gas: 0, + block_access_list: Default::default(), + slot_number: 0, + }; + gloas::ExecutionPayloadEnvelope { + payload, + execution_requests: Default::default(), + builder_index, + beacon_block_root, + parent_beacon_block_root: Default::default(), + } +} + /// A zeroed blob. `Blob` deliberately has no `Default`, because zeroing 128 KiB /// by accident is exactly the mistake that derive would invite. #[cfg(test)] @@ -410,8 +656,8 @@ mod tests { let bytes = contents_bytes(2, 2); let produced = ProducedBlock::from_ssz(ForkName::Electra, &bytes).expect("decodes"); - assert_eq!(produced.block().slot, 1234); - assert_eq!(produced.block().proposer_index, 7); + assert_eq!(produced.slot(), 1234); + assert_eq!(produced.proposer_index(), 7); assert_eq!(produced.blob_count(), 2); assert_eq!(produced.fork, ForkName::Electra); assert!(matches!(produced.contents, Contents::WithBlobProofs { .. })); @@ -427,7 +673,7 @@ mod tests { let bytes = contents_bytes(1, 1); let produced = ProducedBlock::from_ssz(ForkName::Electra, &bytes).expect("decodes"); - assert_eq!(produced.block().hash_tree_root(), expected); + assert_eq!(produced.block_root(), expected); let signed = produced.into_signed_ssz(BlsSignature([9; 96])); let decoded = SignedBlockContents::from_ssz_bytes(&signed).expect("decodes"); @@ -536,4 +782,121 @@ mod tests { .expect_err("must fail"); assert!(matches!(err, Error::Decode(_)), "got {err:?}"); } + + fn gloas_contents_bytes( + builder_index: u64, + proofs: usize, + blobs: usize, + ) -> (Vec, ethlambda_types::beacon::primitives::Root) { + let block = empty_gloas_block_for(77, 5, builder_index); + let root = block.hash_tree_root(); + let bytes = GloasBlockContents { + block, + execution_payload_envelope: empty_gloas_envelope_for(root, builder_index), + kzg_proofs: vec![KzgProof([3; 48]); proofs] + .try_into() + .expect("in bounds"), + blobs: vec![blob(); blobs].try_into().expect("in bounds"), + } + .to_ssz(); + (bytes, root) + } + + /// With the payload included the body is the contents container, and the + /// envelope, proofs and blobs come out in one piece for the second + /// publication. + #[test] + fn a_gloas_block_with_its_payload_decodes_and_hands_the_payload_over() { + let (bytes, root) = gloas_contents_bytes(u64::MAX, 3, 2); + let mut produced = ProducedBlock::from_gloas_ssz(true, &bytes).expect("decodes"); + + assert_eq!(produced.fork, ForkName::Gloas); + assert_eq!(produced.slot(), 77); + assert_eq!(produced.proposer_index(), 5); + assert_eq!(produced.block_root(), root); + assert_eq!(produced.blob_count(), 2); + assert_eq!(produced.builder_index(), Some(u64::MAX)); + + let payload = produced + .take_gloas_payload() + .expect("the payload came along"); + assert_eq!(payload.envelope.beacon_block_root, root); + assert_eq!(payload.kzg_proofs.len(), 3); + assert_eq!(payload.blobs.len(), 2); + assert!(produced.take_gloas_payload().is_none(), "taken once"); + + // The block published afterwards is bare, and its root is untouched. + let signed = produced.into_signed_ssz(BlsSignature([4; 96])); + let decoded = + gloas::SignedBeaconBlock::from_ssz_bytes(&signed).expect("a bare gloas block"); + assert_eq!(decoded.message.hash_tree_root(), root); + assert_eq!(decoded.signature, BlsSignature([4; 96])); + } + + #[test] + fn a_bare_gloas_block_has_no_payload_to_hand_over() { + let block = empty_gloas_block_for(78, 6, 1); + let mut produced = ProducedBlock::from_gloas_ssz(false, &block.to_ssz()).expect("decodes"); + assert_eq!(produced.slot(), 78); + assert_eq!(produced.blob_count(), 0); + assert!(produced.take_gloas_payload().is_none()); + } + + /// The header selects the container, so a wrong value is a decode error and + /// not a block that silently differs. + #[test] + fn the_included_header_selects_the_container() { + let (bytes, _) = gloas_contents_bytes(u64::MAX, 0, 0); + let err = + ProducedBlock::from_gloas_ssz(false, &bytes).expect_err("contents are not a block"); + assert!(matches!(err, Error::Decode(_)), "got {err:?}"); + } + + #[test] + fn a_gloas_body_cannot_be_decoded_without_the_included_header() { + let (bytes, _) = gloas_contents_bytes(u64::MAX, 0, 0); + let err = ProducedBlock::from_ssz(ForkName::Gloas, &bytes).expect_err("must refuse"); + assert!(matches!(err, Error::InconsistentResponse(_)), "got {err:?}"); + } + + /// From gloas the fee recipient lives in the bid, not in a payload the + /// block no longer carries. + #[test] + fn a_gloas_blocks_fee_recipient_is_the_bids() { + let mut block = empty_gloas_block_for(1, 1, u64::MAX); + block + .body + .signed_execution_payload_bid + .message + .fee_recipient = ethlambda_types::beacon::primitives::H160([0xcd; 20]); + let produced = ProducedBlock::from_gloas_ssz(false, &block.to_ssz()).expect("decodes"); + assert_eq!( + produced.fee_recipient(), + ethlambda_types::beacon::primitives::H160([0xcd; 20]) + ); + } + + /// An envelope contents body is the signed envelope, then proofs, then + /// blobs, and decodes back to what was put in. + #[test] + fn the_envelope_contents_round_trip() { + let (bytes, root) = gloas_contents_bytes(u64::MAX, 2, 1); + let mut produced = ProducedBlock::from_gloas_ssz(true, &bytes).expect("decodes"); + let payload = produced.take_gloas_payload().expect("payload"); + let signed = payload.into_signed_contents_ssz(BlsSignature([8; 96])); + let decoded = SignedExecutionPayloadEnvelopeContents::from_ssz_bytes(&signed).expect("ok"); + assert_eq!( + decoded + .signed_execution_payload_envelope + .message + .beacon_block_root, + root + ); + assert_eq!( + decoded.signed_execution_payload_envelope.signature, + BlsSignature([8; 96]) + ); + assert_eq!(decoded.kzg_proofs.len(), 2); + assert_eq!(decoded.blobs.len(), 1); + } } diff --git a/crates/validator/src/beacon_node/dto.rs b/crates/validator/src/beacon_node/dto.rs index df5708f4..e62c24e3 100644 --- a/crates/validator/src/beacon_node/dto.rs +++ b/crates/validator/src/beacon_node/dto.rs @@ -205,6 +205,18 @@ pub struct ProposerDutyDto { pub slot: Slot, } +/// One entry of `POST /eth/v1/validator/duties/ptc/{epoch}`: the same three +/// fields a proposer duty has, since a committee member too is named by key, +/// index and the one slot it votes in. +#[derive(Debug, Clone, Deserialize)] +pub struct PtcDutyDto { + pub pubkey: String, + #[serde(with = "quoted_u64")] + pub validator_index: ValidatorIndex, + #[serde(with = "quoted_u64")] + pub slot: Slot, +} + #[derive(Debug, Clone, Deserialize)] pub struct ValidatorEntryDto { #[serde(with = "quoted_u64")] @@ -229,6 +241,10 @@ pub struct SingleAttestationDto { pub signature: String, } +/// `index` is whatever the beacon node answered, never rewritten: it is part of +/// the signed message. It is zero from electra, and from gloas it carries the +/// payload signal (1 when the attested block's payload was seen as full), which +/// is why it must not be normalised on the way out. #[derive(Debug, Clone, Serialize)] pub struct AttestationDataOutDto { #[serde(with = "quoted_u64")] @@ -476,6 +492,27 @@ pub fn config_from_spec_response(value: &serde_json::Value) -> Result { if let Some(bps) = spec_u64(value, "AGGREGATE_DUE_BPS")? { config.aggregate_due_bps = bps; } + // The gloas offsets and the payload deadlines. Absent keeps the + // specification's defaults already in `Config` (2500, 5000, 5000, 7500). + for (key, field) in [ + ( + "ATTESTATION_DUE_BPS_GLOAS", + &mut config.attestation_due_bps_gloas, + ), + ( + "AGGREGATE_DUE_BPS_GLOAS", + &mut config.aggregate_due_bps_gloas, + ), + ("PAYLOAD_DUE_BPS", &mut config.payload_due_bps), + ( + "PAYLOAD_ATTESTATION_DUE_BPS", + &mut config.payload_attestation_due_bps, + ), + ] { + if let Some(bps) = spec_u64(value, key)? { + *field = bps; + } + } if let Some(version) = spec_version(value, "GENESIS_FORK_VERSION")? { config = config.with_fork_version(ForkName::Phase0, version); } @@ -910,6 +947,27 @@ mod tests { assert_eq!(config.aggregate_due_bps, 5_000); } + #[test] + fn the_gloas_offsets_are_read_and_default_to_the_specification() { + let response = serde_json::json!({ + "ATTESTATION_DUE_BPS_GLOAS": "2000", + "AGGREGATE_DUE_BPS_GLOAS": "4000", + "PAYLOAD_DUE_BPS": "5500", + "PAYLOAD_ATTESTATION_DUE_BPS": "8000", + }); + let config = config_from_spec_response(&response).expect("builds"); + assert_eq!(config.attestation_due_bps_gloas, 2_000); + assert_eq!(config.aggregate_due_bps_gloas, 4_000); + assert_eq!(config.payload_due_bps, 5_500); + assert_eq!(config.payload_attestation_due_bps, 8_000); + + let config = config_from_spec_response(&serde_json::json!({})).expect("builds"); + assert_eq!(config.attestation_due_bps_gloas, 2_500); + assert_eq!(config.aggregate_due_bps_gloas, 5_000); + assert_eq!(config.payload_due_bps, 5_000); + assert_eq!(config.payload_attestation_due_bps, 7_500); + } + /// A node that has not moved to the basis-point form keeps the defaults, /// which are the right answer for every network that predates it. #[test] @@ -965,4 +1023,39 @@ mod tests { let err = config_from_spec_response(&response).expect_err("must reject"); assert!(matches!(err, Error::Decode(_)), "got {err:?}"); } + + /// A gloas aggregate parses straight into the gloas container, bitfields + /// included, and serialises back to the JSON the electra DTO produces for + /// the same vote, which is what the endpoint takes. + #[test] + fn a_gloas_aggregate_round_trips_through_its_json() { + use ethlambda_types::beacon::containers::gloas; + + let electra_attestation = Attestation { + aggregation_bits: AggregationBits::from_ssz_bytes(&[0x3f]).expect("bits"), + data: AttestationData { + slot: 5, + index: 1, + beacon_block_root: Root::repeat_byte(3), + source: Checkpoint { + epoch: 0, + root: Root::ZERO, + }, + target: Checkpoint { + epoch: 0, + root: Root::repeat_byte(4), + }, + }, + signature: BlsSignature([9; 96]), + committee_bits: CommitteeBits::from_ssz_bytes(&[0x01, 0, 0, 0, 0, 0, 0, 0]) + .expect("bits"), + }; + let json = serde_json::to_value(AttestationOutDto::from(&electra_attestation)) + .expect("serialises"); + + let parsed: gloas::Attestation = serde_json::from_value(json.clone()).expect("parses"); + assert_eq!(parsed, gloas::Attestation::from(&electra_attestation)); + assert_eq!(parsed.data.index, 1, "the payload signal survives parsing"); + assert_eq!(serde_json::to_value(&parsed).expect("serialises"), json); + } } diff --git a/crates/validator/src/beacon_node/fallback.rs b/crates/validator/src/beacon_node/fallback.rs index c1d3eb35..c5eb5c8d 100644 --- a/crates/validator/src/beacon_node/fallback.rs +++ b/crates/validator/src/beacon_node/fallback.rs @@ -9,7 +9,7 @@ use std::pin::Pin; use async_trait::async_trait; use ethlambda_types::beacon::config::Config; -use ethlambda_types::beacon::containers::electra::SignedAggregateAndProof; +use ethlambda_types::beacon::containers::gloas; use ethlambda_types::beacon::containers::shared::AttestationData; use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::primitives::{BlsPubkey, Epoch, Root, Slot, ValidatorIndex}; @@ -21,7 +21,7 @@ use crate::beacon_node::dto::{ }; use crate::beacon_node::{ AggregateAttestation, AttesterDuties, BeaconNodeApi, BlockRequest, Genesis, ProposerDuties, - Published, ValidatorEntry, + PtcDuties, Published, SignedAggregates, ValidatorEntry, }; use crate::error::{Error, Result}; @@ -223,8 +223,8 @@ impl BeaconNodeApi for FallbackBeaconNode { .await } - async fn attestation_data(&self, slot: Slot) -> Result { - self.try_each("attestation_data", |node| node.attestation_data(slot)) + async fn attestation_data(&self, slot: Slot, fork: ForkName) -> Result { + self.try_each("attestation_data", |node| node.attestation_data(slot, fork)) .await } @@ -251,6 +251,61 @@ impl BeaconNodeApi for FallbackBeaconNode { .await } + /// Fails over like any query: the envelope the node cached is only on the + /// node that produced the block, and one that did not answers 404. + async fn execution_payload_envelope( + &self, + slot: Slot, + block_root: Root, + ) -> Result { + self.try_each("execution_payload_envelope", |node| { + node.execution_payload_envelope(slot, block_root) + }) + .await + } + + /// First node that accepts it wins, for the reason [`Self::publish_block`] + /// is: the accepting node gossips it. + async fn publish_execution_payload_envelope( + &self, + body: &[u8], + blob_data_included: bool, + ) -> Result<()> { + self.try_each("publish_execution_payload_envelope", |node| { + node.publish_execution_payload_envelope(body, blob_data_included) + }) + .await + } + + async fn ptc_duties(&self, epoch: Epoch, indices: &[ValidatorIndex]) -> Result { + self.try_each("ptc_duties", |node| node.ptc_duties(epoch, indices)) + .await + } + + /// A 204 is an answer, so the first node that knows no block ends the + /// walk, even though another might have one. That is the right trade: PTC + /// votes are due at three quarters of the slot, and a node that has seen + /// nothing by then is one this client should not be voting through. + async fn payload_attestation_data( + &self, + slot: Slot, + ) -> Result> { + self.try_each("payload_attestation_data", |node| { + node.payload_attestation_data(slot) + }) + .await + } + + async fn submit_payload_attestations( + &self, + messages: &[gloas::PayloadAttestationMessage], + ) -> Result { + self.try_each("submit_payload_attestations", |node| { + node.submit_payload_attestations(messages) + }) + .await + } + async fn submit_attestations( &self, attestations: &[SingleAttestationDto], @@ -280,7 +335,7 @@ impl BeaconNodeApi for FallbackBeaconNode { async fn publish_aggregates( &self, fork: ForkName, - aggregates: &[SignedAggregateAndProof], + aggregates: &SignedAggregates, ) -> Result<()> { self.try_each("publish_aggregates", |node| { node.publish_aggregates(fork, aggregates) @@ -354,7 +409,7 @@ mod tests { let fallback = FallbackBeaconNode::new(vec![stale, fresh]); let data = fallback - .attestation_data(96) + .attestation_data(96, ForkName::Electra) .await .expect("the second node answers correctly"); @@ -374,7 +429,7 @@ mod tests { ]); let err = fallback - .attestation_data(96) + .attestation_data(96, ForkName::Electra) .await .expect_err("no node answered about the requested slot"); @@ -477,6 +532,7 @@ mod tests { use ethlambda_types::beacon::primitives::{BlsSignature, Bytes32}; BlockRequest { slot, + fork: ForkName::Electra, proposer_index, randao_reveal: BlsSignature([0; 96]), graffiti: Bytes32::default(), @@ -497,7 +553,7 @@ mod tests { .produce_block(&request(96, 7)) .await .expect("the second node answers correctly"); - assert_eq!(block.block().slot, 96); + assert_eq!(block.slot(), 96); } /// A node on a different fork computes a different proposer. Signing its @@ -513,7 +569,7 @@ mod tests { .produce_block(&request(96, 7)) .await .expect("the second node names the expected proposer"); - assert_eq!(block.block().proposer_index, 7); + assert_eq!(block.proposer_index(), 7); } #[tokio::test] diff --git a/crates/validator/src/beacon_node/http.rs b/crates/validator/src/beacon_node/http.rs index dee38b01..71a845fb 100644 --- a/crates/validator/src/beacon_node/http.rs +++ b/crates/validator/src/beacon_node/http.rs @@ -24,7 +24,8 @@ use std::time::Duration; use async_trait::async_trait; use ethlambda_types::beacon::config::Config; -use ethlambda_types::beacon::containers::electra::{Attestation, SignedAggregateAndProof}; +use ethlambda_types::beacon::containers::electra::Attestation; +use ethlambda_types::beacon::containers::gloas; use ethlambda_types::beacon::containers::shared::AttestationData; use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::primitives::{BlsPubkey, Epoch, Root, Slot, ValidatorIndex}; @@ -32,16 +33,19 @@ use reqwest::{Client, StatusCode}; use serde::Serialize; use tracing::{debug, warn}; +use libssz::SszDecode as _; + use crate::beacon_node::block_contents::ProducedBlock; use crate::beacon_node::dto::{ AttestationDataDto, AttestationDto, AttesterDutyDto, CommitteeSubscriptionDto, DataResponse, DutiesResponse, GenesisDto, IndexedErrorResponse, ProposerDutyDto, ProposerPreparationDto, - SignedAggregateAndProofOutDto, SingleAttestationDto, SyncingDto, ValidatorEntryDto, + PtcDutyDto, SignedAggregateAndProofOutDto, SingleAttestationDto, SyncingDto, ValidatorEntryDto, VersionedResponse, config_from_spec_response, encode_hex, parse_pubkey, parse_root, }; use crate::beacon_node::{ - AggregateAttestation, AttesterDuties, BeaconNodeApi, BlockRequest, Genesis, ProposerDuties, - Published, ValidatorEntry, validate_attestation_data, validate_produced_block, + AggregateAttestation, AggregateKind, AttesterDuties, BeaconNodeApi, BlockRequest, Genesis, + ProposerDuties, PtcDuties, Published, SignedAggregates, ValidatorEntry, + validate_attestation_data, validate_produced_block, }; use crate::error::{BeaconNodeFailure, Error, Result}; @@ -54,6 +58,23 @@ use crate::error::{BeaconNodeFailure, Error, Result}; /// a stalled request could then outlive the slot it was serving. const REQUEST_TIMEOUT: Duration = Duration::from_secs(8); +/// What an SSZ answer carries besides its bytes: the fork, which only a header +/// can say, and the two booleans the block endpoints use to say what the bytes +/// are. +struct SszResponse { + fork: ForkName, + /// `Eth-Execution-Payload-Blinded`, sent by `produceBlockV3` only. + blinded: Option, + /// `Eth-Execution-Payload-Included`, sent by `produceBlockV4` only. + payload_included: Option, + body: Vec, +} + +/// The body `produceBlockV4` requires. Self-build only, so no minimum bid, no +/// boost and no builders: the node decodes it, as the specification demands, +/// and builds locally. +const BUILDER_CONFIG: &str = r#"{"min_bid":"0","builder_boost_factor":"0","builders":[]}"#; + /// A [`BeaconNodeApi`] backed by one beacon node's standard REST Beacon API. pub struct HttpBeaconNode { base_url: String, @@ -170,24 +191,43 @@ impl HttpBeaconNode { /// root that produces. The comparison is case-insensitive: the schema's /// enum is lowercase but nothing in the specification says a client must /// match it that way. - /// - /// Returns the blinded flag as an `Option`, `None` when the header is - /// absent, because only one endpoint sends it and only that endpoint's - /// caller knows whether its absence is an error. - async fn get_ssz(&self, path: &str) -> Result<(ForkName, Option, Vec)> { + async fn get_ssz(&self, path: &str) -> Result { let url = format!("{}{path}", self.base_url); debug!(%url, "Beacon API GET (ssz)"); - let response = self + let request = self .client .get(&url) + .header(reqwest::header::ACCEPT, "application/octet-stream"); + Self::read_ssz(url, request).await + } + + /// POST a JSON body and read the answer as SSZ, the way [`Self::get_ssz`] + /// does. For `produceBlockV4`, which is a POST because it carries the + /// builder configuration, and answers in SSZ like its predecessor. + async fn post_json_for_ssz( + &self, + path: &str, + json_body: &'static str, + consensus_version: &str, + ) -> Result { + let url = format!("{}{path}", self.base_url); + debug!(%url, "Beacon API POST (json in, ssz out)"); + let request = self + .client + .post(&url) .header(reqwest::header::ACCEPT, "application/octet-stream") - .send() - .await - .map_err(|err| Error::BeaconNode { - url: url.clone(), - failure: BeaconNodeFailure::classify(&err), - detail: err.to_string(), - })?; + .header(reqwest::header::CONTENT_TYPE, "application/json") + .header("Eth-Consensus-Version", consensus_version) + .body(json_body); + Self::read_ssz(url, request).await + } + + async fn read_ssz(url: String, request: reqwest::RequestBuilder) -> Result { + let response = request.send().await.map_err(|err| Error::BeaconNode { + url: url.clone(), + failure: BeaconNodeFailure::classify(&err), + detail: err.to_string(), + })?; let status = response.status(); if status == StatusCode::SERVICE_UNAVAILABLE { @@ -233,26 +273,111 @@ impl HttpBeaconNode { )) })?; - // Reported, not judged. Only `produceBlockV3` sends this header, so - // requiring it here would break every other caller of this helper; - // whether its absence matters is the caller's question. - let blinded = match header("eth-execution-payload-blinded") { - Some(value) if value.eq_ignore_ascii_case("true") => Some(true), - Some(value) if value.eq_ignore_ascii_case("false") => Some(false), - Some(value) => { - return Err(Error::InconsistentResponse(format!( - "Eth-Execution-Payload-Blinded is {value}, expected true or false" - ))); - } - None => None, - }; + // Reported, not judged. Each is sent by one endpoint only, so requiring + // it here would break every other caller of this helper; whether its + // absence matters is the caller's question. + let blinded = Self::bool_header(header("eth-execution-payload-blinded"), "Blinded")?; + let payload_included = + Self::bool_header(header("eth-execution-payload-included"), "Included")?; let body = response.bytes().await.map_err(|err| Error::BeaconNode { url, failure: BeaconNodeFailure::classify(&err), detail: err.to_string(), })?; - Ok((fork, blinded, body.to_vec())) + Ok(SszResponse { + fork, + blinded, + payload_included, + body: body.to_vec(), + }) + } + + /// A `true`/`false` header, `None` when absent, an error when it is + /// anything else. + fn bool_header(value: Option, name: &str) -> Result> { + match value { + Some(value) if value.eq_ignore_ascii_case("true") => Ok(Some(true)), + Some(value) if value.eq_ignore_ascii_case("false") => Ok(Some(false)), + Some(value) => Err(Error::InconsistentResponse(format!( + "Eth-Execution-Payload-{name} is {value}, expected true or false" + ))), + None => Ok(None), + } + } + + /// A GET whose success may be a 204, which is an answer ("nothing to + /// report") and not a failure. + async fn get_optional(&self, path: &str) -> Result> { + let url = format!("{}{path}", self.base_url); + debug!(%url, "Beacon API GET"); + let response = self + .client + .get(&url) + .send() + .await + .map_err(|err| Error::BeaconNode { + url: url.clone(), + failure: BeaconNodeFailure::classify(&err), + detail: err.to_string(), + })?; + if response.status() == StatusCode::NO_CONTENT { + return Ok(None); + } + Self::decode(response).await.map(Some) + } + + /// `produceBlockV4`: a POST carrying the builder configuration, asking for + /// the self-built payload to be included. + /// + /// `include_payload=true` saves the proposal a round trip, which matters + /// because the whole proposal, envelope included, has to fit before the + /// attesters' deadline. The node says whether it complied in + /// `Eth-Execution-Payload-Included`, and a missing header is an error for + /// the reason the blinded flag's absence is on the v3 path: it selects which + /// container the bytes are. + async fn produce_gloas_block(&self, request: &BlockRequest) -> Result { + let path = format!( + "/eth/v4/validator/blocks/{}?randao_reveal={}&graffiti={}&include_payload=true", + request.slot, + encode_hex(&request.randao_reveal.0), + encode_hex(&request.graffiti.0), + ); + let response = self + .post_json_for_ssz(&path, BUILDER_CONFIG, request.fork.as_str()) + .await?; + if response.fork != ForkName::Gloas { + return Err(Error::InconsistentResponse(format!( + "asked for a gloas block for slot {}, node answered with a {} one", + request.slot, + response.fork.as_str() + ))); + } + let included = response.payload_included.ok_or_else(|| { + Error::InconsistentResponse( + "response carries no Eth-Execution-Payload-Included, so whether the body is \ + block contents or a bare block is unknown" + .to_string(), + ) + })?; + let block = ProducedBlock::from_gloas_ssz(included, &response.body)?; + // Enforced here rather than at the call site so failover works: see the + // contract on `BeaconNodeApi::produce_block`. + validate_produced_block(request, &block)?; + Ok(block) + } + + /// The path for one slot's attestation data. + /// + /// Gloas omits `committee_index`: the parameter is optional there, and the + /// answer's `index` is the payload signal rather than a committee, so + /// asking about committee 0 would only suggest the old meaning. + fn attestation_data_path(slot: Slot, fork: ForkName) -> String { + if fork >= ForkName::Gloas { + format!("/eth/v1/validator/attestation_data?slot={slot}") + } else { + format!("/eth/v1/validator/attestation_data?slot={slot}&committee_index=0") + } } /// The query string for one aggregate. @@ -438,12 +563,9 @@ impl BeaconNodeApi for HttpBeaconNode { }) } - async fn attestation_data(&self, slot: Slot) -> Result { - let response: DataResponse = self - .get(&format!( - "/eth/v1/validator/attestation_data?slot={slot}&committee_index=0" - )) - .await?; + async fn attestation_data(&self, slot: Slot, fork: ForkName) -> Result { + let response: DataResponse = + self.get(&Self::attestation_data_path(slot, fork)).await?; let data = AttestationData::try_from(&response.data)?; // Enforced here rather than at the call site so that failover works: // see the contract on `BeaconNodeApi::attestation_data`. An `Err` here @@ -473,13 +595,21 @@ impl BeaconNodeApi for HttpBeaconNode { /// burn the slot's proposal guard entry for a block that can never be /// sent. async fn produce_block(&self, request: &BlockRequest) -> Result { + if request.fork >= ForkName::Gloas { + return self.produce_gloas_block(request).await; + } let path = format!( "/eth/v3/validator/blocks/{}?randao_reveal={}&graffiti={}&builder_boost_factor=0", request.slot, encode_hex(&request.randao_reveal.0), encode_hex(&request.graffiti.0), ); - let (fork, blinded, body) = self.get_ssz(&path).await?; + let SszResponse { + fork, + blinded, + body, + .. + } = self.get_ssz(&path).await?; // Required on this endpoint, so its absence is an error rather than a // default. The specification marks it required precisely because it // selects which container the bytes are, and a blinded body is a @@ -540,6 +670,136 @@ impl BeaconNodeApi for HttpBeaconNode { }) } + async fn execution_payload_envelope( + &self, + slot: Slot, + block_root: Root, + ) -> Result { + let path = format!( + "/eth/v1/validator/execution_payload_envelopes/{slot}/{}", + encode_hex(&block_root.0) + ); + let response = self.get_ssz(&path).await?; + if response.fork != ForkName::Gloas { + return Err(Error::InconsistentResponse(format!( + "node answered an envelope request with a {} body", + response.fork.as_str() + ))); + } + let envelope = gloas::ExecutionPayloadEnvelope::from_ssz_bytes(&response.body) + .map_err(|err| Error::Decode(format!("execution payload envelope: {err:?}")))?; + // The envelope is signed as it stands, so one for another block must + // never reach the signing path. Enforced here so failover can act on it. + if envelope.beacon_block_root != block_root { + return Err(Error::InconsistentResponse(format!( + "asked for the envelope of block {}, node answered with one for block {}", + encode_hex(&block_root.0), + encode_hex(&envelope.beacon_block_root.0) + ))); + } + Ok(envelope) + } + + async fn publish_execution_payload_envelope( + &self, + body: &[u8], + blob_data_included: bool, + ) -> Result<()> { + let url = format!( + "{}/eth/v1/beacon/execution_payload_envelopes", + self.base_url + ); + let response = self + .client + .post(&url) + .header("Eth-Consensus-Version", ForkName::Gloas.as_str()) + .header("Eth-Blob-Data-Included", blob_data_included.to_string()) + .header(reqwest::header::CONTENT_TYPE, "application/octet-stream") + .body(body.to_vec()) + .send() + .await + .map_err(|err| Error::BeaconNode { + url: url.clone(), + failure: BeaconNodeFailure::classify(&err), + detail: err.to_string(), + })?; + let status = response.status(); + if status.is_success() { + return Ok(()); + } + let body = response.text().await.unwrap_or_default(); + Err(Error::BeaconNodeStatus { + status: status.as_u16(), + body, + }) + } + + async fn ptc_duties(&self, epoch: Epoch, indices: &[ValidatorIndex]) -> Result { + let body: Vec = indices.iter().map(|index| index.to_string()).collect(); + let response: DutiesResponse> = self + .post( + &format!("/eth/v1/validator/duties/ptc/{epoch}"), + &body, + None, + ) + .await?; + Ok(PtcDuties { + dependent_root: parse_root(&response.dependent_root)?, + duties: response.data, + }) + } + + async fn payload_attestation_data( + &self, + slot: Slot, + ) -> Result> { + let response: Option> = self + .get_optional(&format!( + "/eth/v1/validator/payload_attestation_data?slot={slot}" + )) + .await?; + let Some(response) = response else { + return Ok(None); + }; + // Enforced here for the reason the other fetches' checks are: this is + // signable material, and a node answering about another slot must be + // failed over from rather than signed for. + if response.data.slot != slot { + return Err(Error::InconsistentResponse(format!( + "requested payload attestation data for slot {slot}, node answered for slot {}", + response.data.slot + ))); + } + Ok(Some(response.data)) + } + + /// The same partial-success reading as attestations: see + /// [`Self::handle_pool_submission`]. + async fn submit_payload_attestations( + &self, + messages: &[gloas::PayloadAttestationMessage], + ) -> Result { + let url = format!("{}/eth/v1/beacon/pool/payload_attestations", self.base_url); + let response = self + .client + .post(&url) + .header("Eth-Consensus-Version", ForkName::Gloas.as_str()) + .json(messages) + .send() + .await + .map_err(|err| Error::BeaconNode { + url: url.clone(), + failure: BeaconNodeFailure::classify(&err), + detail: err.to_string(), + })?; + let status = response.status(); + if status.is_success() { + return Ok(messages.len()); + } + let body = response.text().await.unwrap_or_default(); + Self::handle_pool_submission(status, body, messages.len()) + } + /// Distinct from `post_no_content`: this endpoint's 400s are not always /// total failures. See [`Self::handle_pool_submission`]. async fn submit_attestations( @@ -583,7 +843,10 @@ impl BeaconNodeApi for HttpBeaconNode { // JSON, not SSZ, and not by preference: Lighthouse answers this endpoint // in JSON whatever `Accept` says. See `AttestationDto` for why that is // safe here when it would not be for a block. - let response: VersionedResponse = self.get(&path).await?; + // + // The body is read as untyped JSON first, because the layout of `data` + // depends on the `version` beside it. + let response: VersionedResponse = self.get(&path).await?; let fork = ForkName::parse(&response.version.to_ascii_lowercase()).ok_or_else(|| { Error::InconsistentResponse(format!( "aggregate names fork {}, which this client does not know", @@ -595,18 +858,14 @@ impl BeaconNodeApi for HttpBeaconNode { // Electra widened `Attestation` with `committee_bits`, so an earlier // fork's bytes are a different shape and this client has no container // for them. Named explicitly rather than `fork < ForkName::Electra`, - // so a fork added after fulu is not silently waved through as - // electra-shaped. - match fork { - // Gloas's own attestation shape is not this client's electra one - // (EIP-7549 continues to change under EIP-7688), so it is - // refused by name too rather than mis-decoded. + // so a fork added after gloas is not silently waved through as + // gloas-shaped. + let attestation = match fork { ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix | ForkName::Capella | ForkName::Deneb - | ForkName::Gloas | ForkName::Lean => { return Err(Error::InconsistentResponse(format!( "node produced a {} aggregate, which this client does not publish; electra \ @@ -614,19 +873,29 @@ impl BeaconNodeApi for HttpBeaconNode { fork.as_str() ))); } - ForkName::Electra | ForkName::Fulu => {} - } - let attestation = Attestation::try_from(&response.data)?; + ForkName::Electra | ForkName::Fulu => { + let dto: AttestationDto = serde_json::from_value(response.data) + .map_err(|err| Error::Decode(format!("electra aggregate: {err}")))?; + AggregateKind::Electra(Attestation::try_from(&dto)?) + } + // Gloas's attestation is its own container, with its own hash tree + // root, so it is decoded as one and signed over as one. + ForkName::Gloas => { + let attestation: gloas::Attestation = serde_json::from_value(response.data) + .map_err(|err| Error::Decode(format!("gloas aggregate: {err}")))?; + AggregateKind::Gloas(attestation) + } + }; // The same contract the other two fetches carry, enforced here so a // node answering about the wrong slot is failed over from rather than // wrapped in a signature. `committee_index` is deliberately not checked // against `committee_bits`: which committees an aggregate covers is the // node's answer to the question, and electra's gossip rules already // require exactly one. - if attestation.data.slot != slot { + if attestation.data().slot != slot { return Err(Error::InconsistentResponse(format!( "requested an aggregate for slot {slot}, node answered for slot {}", - attestation.data.slot + attestation.data().slot ))); } Ok(AggregateAttestation { fork, attestation }) @@ -637,18 +906,23 @@ impl BeaconNodeApi for HttpBeaconNode { async fn publish_aggregates( &self, fork: ForkName, - aggregates: &[SignedAggregateAndProof], + aggregates: &SignedAggregates, ) -> Result<()> { - let body: Vec = aggregates - .iter() - .map(SignedAggregateAndProofOutDto::from) - .collect(); - self.post_no_content( - "/eth/v2/validator/aggregate_and_proofs", - &body, - Some(fork.as_str()), - ) - .await + const PATH: &str = "/eth/v2/validator/aggregate_and_proofs"; + match aggregates { + SignedAggregates::Electra(list) => { + let body: Vec = list + .iter() + .map(SignedAggregateAndProofOutDto::from) + .collect(); + self.post_no_content(PATH, &body, Some(fork.as_str())).await + } + // The gloas container serialises as the endpoint's JSON itself + // (hex bitfields, quoted integers), so there is no second DTO. + SignedAggregates::Gloas(list) => { + self.post_no_content(PATH, list, Some(fork.as_str())).await + } + } } async fn prepare_beacon_proposer(&self, preparations: &[ProposerPreparationDto]) -> Result<()> { @@ -772,4 +1046,40 @@ mod tests { assert!(path.contains(parameter), "{parameter} missing from {path}"); } } + + /// Before gloas the query still names committee 0, exactly as it always + /// has; from gloas it is omitted. + #[test] + fn the_committee_index_is_omitted_from_gloas_on() { + assert_eq!( + HttpBeaconNode::attestation_data_path(96, ForkName::Fulu), + "/eth/v1/validator/attestation_data?slot=96&committee_index=0" + ); + assert_eq!( + HttpBeaconNode::attestation_data_path(96, ForkName::Gloas), + "/eth/v1/validator/attestation_data?slot=96" + ); + } + + /// The body `produceBlockV4` requires: self-build only, so no builders. + #[test] + fn the_builder_config_is_the_documented_self_build_body() { + let value: serde_json::Value = serde_json::from_str(BUILDER_CONFIG).expect("valid json"); + assert_eq!(value["min_bid"], "0"); + assert_eq!(value["builder_boost_factor"], "0"); + assert_eq!(value["builders"], serde_json::json!([])); + } + + #[test] + fn a_boolean_header_is_read_strictly() { + assert_eq!( + HttpBeaconNode::bool_header(Some("True".into()), "Included").expect("ok"), + Some(true) + ); + assert_eq!( + HttpBeaconNode::bool_header(None, "Included").expect("ok"), + None + ); + HttpBeaconNode::bool_header(Some("yes".into()), "Included").expect_err("not a boolean"); + } } diff --git a/crates/validator/src/beacon_node/mock.rs b/crates/validator/src/beacon_node/mock.rs index 9e137030..34d01f57 100644 --- a/crates/validator/src/beacon_node/mock.rs +++ b/crates/validator/src/beacon_node/mock.rs @@ -7,22 +7,38 @@ use std::sync::Mutex; use async_trait::async_trait; use ethlambda_types::beacon::config::Config; use ethlambda_types::beacon::containers::electra::{Attestation, SignedAggregateAndProof}; +use ethlambda_types::beacon::containers::gloas; use ethlambda_types::beacon::containers::shared::AttestationData; use ethlambda_types::beacon::containers::shared::Checkpoint; use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::primitives::{BlsPubkey, Epoch, Root, Slot, ValidatorIndex}; -use crate::beacon_node::block_contents::{Contents, ProducedBlock, empty_block_for}; +use crate::beacon_node::block_contents::{ + Contents, GloasPayload, ProducedBlock, empty_block_for, empty_gloas_block_for, + empty_gloas_envelope_for, +}; use crate::beacon_node::dto::{ - AttesterDutyDto, CommitteeSubscriptionDto, ProposerDutyDto, ProposerPreparationDto, + AttesterDutyDto, CommitteeSubscriptionDto, ProposerDutyDto, ProposerPreparationDto, PtcDutyDto, SingleAttestationDto, }; use crate::beacon_node::{ - AggregateAttestation, AttesterDuties, BeaconNodeApi, BlockRequest, Genesis, ProposerDuties, - Published, ValidatorEntry, + AggregateAttestation, AggregateKind, AttesterDuties, BeaconNodeApi, BlockRequest, Genesis, + ProposerDuties, PtcDuties, Published, SignedAggregates, ValidatorEntry, }; use crate::error::{BeaconNodeFailure, Error, Result}; +/// What the mock produces at a gloas slot. +#[derive(Debug, Clone, Copy)] +pub struct GloasBlockSpec { + pub slot: Slot, + pub proposer_index: ValidatorIndex, + /// The builder the bid names; `BUILDER_INDEX_SELF_BUILD` for a self-build. + pub builder_index: u64, + /// Whether `produce_block` returns the envelope with the block, as + /// `include_payload=true` does. + pub include_payload: bool, +} + /// A `BeaconNodeApi` driven entirely by fields and methods a test controls, /// rather than by a network or a real beacon node. #[derive(Default)] @@ -49,6 +65,15 @@ pub struct MockBeaconNode { /// The slot and proposer the mock will produce a block for. `None` makes /// `produce_block` fail, the way an unset `attestation_data` does. pub produces_block: Option<(Slot, ValidatorIndex)>, + /// What the mock produces when asked for a gloas block. + pub produces_gloas_block: Option, + /// When set, the envelope the mock serves is for this root instead of the + /// produced block's, so a test can present a wrong one. + pub envelope_root_override: Option, + /// Every envelope body published, with `Eth-Blob-Data-Included`. + pub published_envelopes: Mutex, bool)>>, + /// Every `execution_payload_envelope` request, as (slot, block root). + pub envelope_requests: Mutex>, /// What `publish_block` reports. Defaults to `Imported`; a test that cares /// about the 202 path sets it. pub publish_outcome: Option, @@ -71,6 +96,18 @@ pub struct MockBeaconNode { /// When set, `aggregate_attestation` answers with an aggregate over this /// data. `None` makes it fail the way a node with nothing to fold does. pub aggregate: Option, + /// The fork the mock names for its aggregate. Defaults to electra. + pub aggregate_fork: Option, + /// Every published gloas aggregate batch. + pub published_gloas_aggregates: Mutex)>>, + pub ptc: Mutex>, + pub ptc_calls: Mutex, + /// What `payload_attestation_data` answers with. `None` is the 204. + pub payload_attestation_data: Option, + pub payload_attestation_data_calls: Mutex, + pub submitted_payload_attestations: Mutex>, + /// The fork each `attestation_data` call named, in call order. + pub attestation_data_forks: Mutex>, pub duties_calls: Mutex, pub validator_indices_calls: Mutex, pub proposer_duties_calls: Mutex, @@ -201,6 +238,117 @@ impl MockBeaconNode { self } + /// Answer `produce_block` at a gloas slot with a bare gloas block whose + /// bid names `builder_index`, with its self-built envelope attached when + /// `include_payload` is set. + pub fn with_gloas_block( + mut self, + slot: Slot, + proposer_index: ValidatorIndex, + builder_index: u64, + include_payload: bool, + ) -> Self { + self.produces_gloas_block = Some(GloasBlockSpec { + slot, + proposer_index, + builder_index, + include_payload, + }); + self + } + + /// Serve an envelope for `root` whatever block was produced. + pub fn with_envelope_for(mut self, root: Root) -> Self { + self.envelope_root_override = Some(root); + self + } + + /// Every envelope body published, with `Eth-Blob-Data-Included`. + pub fn published_envelopes(&self) -> Vec<(Vec, bool)> { + self.published_envelopes.lock().expect("lock").clone() + } + + /// Every envelope fetch, as (slot, block root). + pub fn envelope_requests(&self) -> Vec<(Slot, Root)> { + self.envelope_requests.lock().expect("lock").clone() + } + + /// Answer `aggregate_attestation` with a gloas aggregate over `data`. + pub fn with_gloas_aggregate(mut self, data: AttestationData) -> Self { + self.aggregate = Some(data); + self.aggregate_fork = Some(ForkName::Gloas); + self + } + + /// Every gloas aggregate batch published so far. + pub fn published_gloas_aggregates( + &self, + ) -> Vec<(ForkName, Vec)> { + self.published_gloas_aggregates + .lock() + .expect("lock") + .clone() + } + + /// Give `attestation_data` answers a nonzero `index`, the way a gloas node + /// signals a full payload. + pub fn with_attestation_index(mut self, index: u64) -> Self { + if let Some(data) = self.attestation_data.as_mut() { + data.index = index; + } + self + } + + /// The forks `attestation_data` was asked under, in call order. + pub fn attestation_data_forks(&self) -> Vec { + self.attestation_data_forks.lock().expect("lock").clone() + } + + pub fn with_ptc_duties( + self, + epoch: Epoch, + dependent_root: Root, + duties: Vec, + ) -> Self { + self.set_ptc_duties(epoch, dependent_root, duties); + self + } + + /// Replace the PTC duties stored for `epoch`, the way `set_duties` does. + pub fn set_ptc_duties(&self, epoch: Epoch, dependent_root: Root, duties: Vec) { + let mut stored = self.ptc.lock().expect("lock"); + let entry = PtcDuties { + dependent_root, + duties, + }; + match stored.iter_mut().find(|(held, _)| *held == epoch) { + Some((_, existing)) => *existing = entry, + None => stored.push((epoch, entry)), + } + } + + pub fn ptc_call_count(&self) -> usize { + *self.ptc_calls.lock().expect("lock") + } + + /// Answer `payload_attestation_data` with `data`. + pub fn with_payload_attestation_data(mut self, data: gloas::PayloadAttestationData) -> Self { + self.payload_attestation_data = Some(data); + self + } + + pub fn payload_attestation_data_call_count(&self) -> usize { + *self.payload_attestation_data_calls.lock().expect("lock") + } + + /// Every payload attestation message submitted so far. + pub fn submitted_payload_attestations(&self) -> Vec { + self.submitted_payload_attestations + .lock() + .expect("lock") + .clone() + } + pub fn with_publish_outcome(mut self, outcome: Published) -> Self { self.publish_outcome = Some(outcome); self @@ -377,9 +525,10 @@ impl BeaconNodeApi for MockBeaconNode { }) } - async fn attestation_data(&self, slot: Slot) -> Result { + async fn attestation_data(&self, slot: Slot, fork: ForkName) -> Result { self.guard("attestation_data")?; *self.attestation_data_calls.lock().expect("lock") += 1; + self.attestation_data_forks.lock().expect("lock").push(fork); let data = self.attestation_data.ok_or_else(|| Error::BeaconNode { url: "mock".to_string(), failure: BeaconNodeFailure::Request, @@ -405,6 +554,28 @@ impl BeaconNodeApi for MockBeaconNode { .lock() .expect("lock") .push(request.clone()); + if request.fork >= ForkName::Gloas { + let spec = self.produces_gloas_block.ok_or_else(|| Error::BeaconNode { + url: "mock".to_string(), + failure: BeaconNodeFailure::Request, + detail: "no gloas block set".into(), + })?; + let block = empty_gloas_block_for(spec.slot, spec.proposer_index, spec.builder_index); + let payload = spec.include_payload.then(|| { + use ethlambda_types::beacon::primitives::HashTreeRoot as _; + Box::new(GloasPayload { + envelope: empty_gloas_envelope_for(block.hash_tree_root(), spec.builder_index), + kzg_proofs: Default::default(), + blobs: Default::default(), + }) + }); + let block = ProducedBlock { + fork: ForkName::Gloas, + contents: Contents::Gloas { block, payload }, + }; + crate::beacon_node::validate_produced_block(request, &block)?; + return Ok(block); + } let (slot, proposer_index) = self.produces_block.ok_or_else(|| Error::BeaconNode { url: "mock".to_string(), failure: BeaconNodeFailure::Request, @@ -434,6 +605,92 @@ impl BeaconNodeApi for MockBeaconNode { Ok(self.publish_outcome.unwrap_or(Published::Imported)) } + async fn execution_payload_envelope( + &self, + slot: Slot, + block_root: Root, + ) -> Result { + self.guard("execution_payload_envelope")?; + self.envelope_requests + .lock() + .expect("lock") + .push((slot, block_root)); + let spec = self + .produces_gloas_block + .ok_or_else(|| Error::BeaconNodeStatus { + status: 404, + body: "no envelope cached".to_string(), + })?; + let root = self.envelope_root_override.unwrap_or(block_root); + let envelope = empty_gloas_envelope_for(root, spec.builder_index); + // Honours the contract the real implementation does. + if envelope.beacon_block_root != block_root { + return Err(Error::InconsistentResponse( + "the envelope is for another block".to_string(), + )); + } + Ok(envelope) + } + + async fn publish_execution_payload_envelope( + &self, + body: &[u8], + blob_data_included: bool, + ) -> Result<()> { + self.guard("publish_execution_payload_envelope")?; + self.published_envelopes + .lock() + .expect("lock") + .push((body.to_vec(), blob_data_included)); + Ok(()) + } + + async fn ptc_duties(&self, epoch: Epoch, _indices: &[ValidatorIndex]) -> Result { + self.guard("ptc_duties")?; + *self.ptc_calls.lock().expect("lock") += 1; + self.ptc + .lock() + .expect("lock") + .iter() + .find(|(stored, _)| *stored == epoch) + .map(|(_, duties)| duties.clone()) + .ok_or_else(|| Error::BeaconNode { + url: "mock".to_string(), + failure: BeaconNodeFailure::Request, + detail: format!("no ptc duties for epoch {epoch}"), + }) + } + + async fn payload_attestation_data( + &self, + slot: Slot, + ) -> Result> { + self.guard("payload_attestation_data")?; + *self.payload_attestation_data_calls.lock().expect("lock") += 1; + let Some(data) = self.payload_attestation_data else { + return Ok(None); + }; + if data.slot != slot { + return Err(Error::InconsistentResponse(format!( + "requested payload attestation data for slot {slot}, node answered for slot {}", + data.slot + ))); + } + Ok(Some(data)) + } + + async fn submit_payload_attestations( + &self, + messages: &[gloas::PayloadAttestationMessage], + ) -> Result { + self.guard("submit_payload_attestations")?; + self.submitted_payload_attestations + .lock() + .expect("lock") + .extend_from_slice(messages); + Ok(messages.len()) + } + async fn submit_attestations( &self, attestations: &[SingleAttestationDto], @@ -476,27 +733,46 @@ impl BeaconNodeApi for MockBeaconNode { data.slot ))); } + if self.aggregate_fork == Some(ForkName::Gloas) { + return Ok(AggregateAttestation { + fork: ForkName::Gloas, + attestation: AggregateKind::Gloas(gloas::Attestation { + aggregation_bits: Default::default(), + data, + signature: Default::default(), + committee_bits: Default::default(), + }), + }); + } Ok(AggregateAttestation { fork: ForkName::Electra, - attestation: Attestation { + attestation: AggregateKind::Electra(Attestation { aggregation_bits: Default::default(), data, signature: Default::default(), committee_bits: Default::default(), - }, + }), }) } async fn publish_aggregates( &self, fork: ForkName, - aggregates: &[SignedAggregateAndProof], + aggregates: &SignedAggregates, ) -> Result<()> { self.guard("publish_aggregates")?; - self.published_aggregates - .lock() - .expect("lock") - .push((fork, aggregates.to_vec())); + match aggregates { + SignedAggregates::Electra(list) => self + .published_aggregates + .lock() + .expect("lock") + .push((fork, list.clone())), + SignedAggregates::Gloas(list) => self + .published_gloas_aggregates + .lock() + .expect("lock") + .push((fork, list.clone())), + } Ok(()) } diff --git a/crates/validator/src/beacon_node/mod.rs b/crates/validator/src/beacon_node/mod.rs index 06d68e29..f14824c9 100644 --- a/crates/validator/src/beacon_node/mod.rs +++ b/crates/validator/src/beacon_node/mod.rs @@ -16,11 +16,12 @@ use ethlambda_types::beacon::signing::compute_epoch_at_slot; use crate::beacon_node::block_contents::ProducedBlock; use crate::beacon_node::dto::{ - AttesterDutyDto, CommitteeSubscriptionDto, ProposerDutyDto, ProposerPreparationDto, + AttesterDutyDto, CommitteeSubscriptionDto, ProposerDutyDto, ProposerPreparationDto, PtcDutyDto, SingleAttestationDto, }; use crate::error::Result; -use ethlambda_types::beacon::containers::electra::{Attestation, SignedAggregateAndProof}; +use ethlambda_types::beacon::containers::electra; +use ethlambda_types::beacon::containers::gloas; pub mod block_contents; pub mod dto; @@ -94,6 +95,12 @@ pub struct Duties { /// means here. pub type AttesterDuties = Duties; +/// One epoch's payload timeliness committee duties for the indices asked +/// about. See [`Duties`] for what `dependent_root` means; for these it is the +/// same kind of root the attester schedule uses, since committee membership is +/// derived from the same shuffling. +pub type PtcDuties = Duties; + /// Every proposer for one epoch, not only this client's. The endpoint takes no /// validator list, so the caller filters. See [`Duties`] for what /// `dependent_root` means here, and why it is the more fragile of the two. @@ -111,6 +118,10 @@ pub type ProposerDuties = Duties; #[derive(Debug, Clone)] pub struct BlockRequest { pub slot: Slot, + /// The fork `slot` is in, by this client's own schedule. Selects the + /// endpoint version: `produceBlockV3` before gloas, `produceBlockV4` from + /// it on. + pub fork: ForkName, /// The validator this client believes proposes `slot`. pub proposer_index: ValidatorIndex, /// This proposer's reveal for the slot's epoch, which the node needs @@ -150,17 +161,19 @@ pub enum Published { /// else is one this client's key can only sign uselessly, while still burning /// the guard's record for that slot. pub fn validate_produced_block(request: &BlockRequest, block: &ProducedBlock) -> Result<()> { - let block = block.block(); - if block.slot != request.slot { + if block.slot() != request.slot { return Err(crate::error::Error::InconsistentResponse(format!( "requested a block for slot {}, node produced one for slot {}", - request.slot, block.slot + request.slot, + block.slot() ))); } - if block.proposer_index != request.proposer_index { + if block.proposer_index() != request.proposer_index { return Err(crate::error::Error::InconsistentResponse(format!( "block for slot {} names proposer {}, expected {}", - request.slot, block.proposer_index, request.proposer_index + request.slot, + block.proposer_index(), + request.proposer_index ))); } Ok(()) @@ -174,7 +187,49 @@ pub fn validate_produced_block(request: &BlockRequest, block: &ProducedBlock) -> #[derive(Debug, Clone)] pub struct AggregateAttestation { pub fork: ForkName, - pub attestation: Attestation, + pub attestation: AggregateKind, +} + +/// The attestation container an aggregate is, by fork. +/// +/// Gloas's `Attestation` is not electra's: its `aggregation_bits` is the +/// unbounded progressive bitlist, so its hash tree root differs and an +/// `AggregateAndProof` wrapping it must be signed over the gloas root. +#[derive(Debug, Clone)] +pub enum AggregateKind { + /// Electra and fulu share one layout. + Electra(electra::Attestation), + Gloas(gloas::Attestation), +} + +impl AggregateKind { + /// The data the aggregate votes on, whichever layout carries it. + pub fn data(&self) -> &AttestationData { + match self { + Self::Electra(attestation) => &attestation.data, + Self::Gloas(attestation) => &attestation.data, + } + } +} + +/// A batch of signed aggregates, all of one fork. +#[derive(Debug, Clone)] +pub enum SignedAggregates { + Electra(Vec), + Gloas(Vec), +} + +impl SignedAggregates { + pub fn len(&self) -> usize { + match self { + Self::Electra(list) => list.len(), + Self::Gloas(list) => list.len(), + } + } + + pub fn is_empty(&self) -> bool { + self.len() == 0 + } } /// A validator's index and status, as resolved from its public key. @@ -252,11 +307,15 @@ pub trait BeaconNodeApi: Send + Sync { /// treats the two the same way will act on a stale proposer schedule. async fn proposer_duties(&self, epoch: Epoch) -> Result; - /// Produce the attestation data for `slot`. + /// Produce the attestation data for `slot`, which is in `fork`. /// - /// No committee index: the specification deprecated that parameter, from - /// Electra on `AttestationData.index` must be zero, and beacon nodes ignore - /// whatever is supplied. Keeping it out means a caller cannot get it wrong. + /// No committee index is ever a parameter here. Before gloas the query + /// still carries `committee_index=0`, which the specification deprecated + /// from electra on and nodes ignore, so a caller cannot get it wrong. From + /// gloas it is omitted, because the answer's `index` is no longer a + /// committee but the payload signal (0 for a same-slot block, 1 when the + /// attested block's payload was seen as full), and the caller must pass it + /// through unchanged: it is signed. /// /// # Contract: the answer is for `slot`, or this is an `Err` /// @@ -273,10 +332,14 @@ pub trait BeaconNodeApi: Send + Sync { /// returned, so a node stuck on a stale head is seen as a success every /// slot and the next node is never consulted. Enforced here, a wrong /// answer is an `Err` that failover treats like any other and moves past. - async fn attestation_data(&self, slot: Slot) -> Result; + async fn attestation_data(&self, slot: Slot, fork: ForkName) -> Result; /// Ask the node to build a block for `request.slot`. /// + /// From gloas the node is asked to include the self-built payload's + /// envelope in the answer, which saves a round trip inside the proposal's + /// budget. + /// /// Returns the block and whatever travelled with it, decoded from SSZ. /// JSON is not used here: see [`block_contents`] for why a block, alone /// among everything this client handles, cannot go through the JSON path. @@ -308,6 +371,56 @@ pub trait BeaconNodeApi: Send + Sync { /// proposal, whether or not the first one succeeded. async fn publish_block(&self, fork: ForkName, body: &[u8]) -> Result; + /// The envelope the node cached for the self-built payload of the block + /// whose root is `block_root`, proposed at `slot`. + /// + /// Only needed when `produce_block` came back without its payload + /// (`Eth-Execution-Payload-Included: false`). An answer whose + /// `beacon_block_root` is not `block_root` is an `Err`, for the reason + /// [`validate_produced_block`] is enforced in the implementations. + async fn execution_payload_envelope( + &self, + slot: Slot, + block_root: Root, + ) -> Result; + + /// Publish a signed envelope, given its SSZ body. + /// + /// `blob_data_included` is the `Eth-Blob-Data-Included` header: `true` + /// when the body is `SignedExecutionPayloadEnvelopeContents`, `false` when + /// it is the bare signed envelope and the node attaches the blobs it + /// cached at production time. + async fn publish_execution_payload_envelope( + &self, + body: &[u8], + blob_data_included: bool, + ) -> Result<()>; + + /// The payload timeliness committee duties for `indices` in `epoch`. + /// + /// An epoch before gloas answers with no duties rather than an error. + async fn ptc_duties(&self, epoch: Epoch, indices: &[ValidatorIndex]) -> Result; + + /// What a payload timeliness committee member should vote on at `slot`, or + /// `None` when the node knows no block for it (204), which is an ordinary + /// outcome and not an error. + /// + /// # Contract: the answer is for `slot`, or this is an `Err` + /// + /// For the reason [`Self::attestation_data`] states. + async fn payload_attestation_data( + &self, + slot: Slot, + ) -> Result>; + + /// Submit signed payload attestation messages to the node's pool. Returns + /// how many it accepted, with the same partial-success reading as + /// [`Self::submit_attestations`]. + async fn submit_payload_attestations( + &self, + messages: &[gloas::PayloadAttestationMessage], + ) -> Result; + /// Submits signed attestations to the node's pool, so they reach gossip. /// `fork_name` names the fork the attestations were produced under, since /// the endpoint requires it as a header rather than inferring it. @@ -327,8 +440,9 @@ pub trait BeaconNodeApi: Send + Sync { /// `attestation_data_root` at `slot`. /// /// `committee_index` is separate from the root and not derivable from it. - /// From electra on, `AttestationData.index` is required to be zero, so the - /// root no longer distinguishes one committee's votes from another's in the + /// From electra on, `AttestationData.index` no longer names a committee (it + /// is zero, and from gloas it is the payload signal, 0 or 1), so the root + /// no longer distinguishes one committee's votes from another's in the /// same slot; the committee moved into the attestation's `committee_bits`. /// That is exactly why the v1 form of this endpoint was removed rather than /// deprecated: it had no way to ask the question. @@ -353,11 +467,8 @@ pub trait BeaconNodeApi: Send + Sync { /// hand-written mapping this crate avoids, which is why the two differ. /// /// The endpoint takes a list even for one aggregate, so this does too. - async fn publish_aggregates( - &self, - fork: ForkName, - aggregates: &[SignedAggregateAndProof], - ) -> Result<()>; + async fn publish_aggregates(&self, fork: ForkName, aggregates: &SignedAggregates) + -> Result<()>; /// Tells the node where to pay each validator's execution-layer block /// rewards, so it has somewhere to send them when it builds a payload. diff --git a/crates/validator/src/duties.rs b/crates/validator/src/duties.rs index 5f05c7ab..6b9744df 100644 --- a/crates/validator/src/duties.rs +++ b/crates/validator/src/duties.rs @@ -19,6 +19,13 @@ //! dropped as they come in rather than at every lookup. //! - **A change here is not a change for subscriptions.** Subnet subscriptions //! follow committees, and a proposer has none. +//! +//! Payload timeliness committee (PTC) duties, which exist from gloas, are held +//! in a third map on the attester schedule's terms: the current epoch and the +//! next, replaced when the `dependent_root` changes. They are narrowed to this +//! client's indices on arrival, and are not part of subnet subscriptions +//! either. Whether an epoch is a gloas one is the caller's to decide, since +//! this service holds no fork schedule. use std::collections::HashMap; use std::sync::Arc; @@ -28,7 +35,7 @@ use ethlambda_types::beacon::primitives::{Epoch, Root, Slot, ValidatorIndex}; use tracing::{info, warn}; use crate::beacon_node::BeaconNodeApi; -use crate::beacon_node::dto::{AttesterDutyDto, ProposerDutyDto}; +use crate::beacon_node::dto::{AttesterDutyDto, ProposerDutyDto, PtcDutyDto}; use crate::error::Result; /// One epoch's schedule of some kind, and the block root it is derived from. @@ -46,6 +53,8 @@ pub struct DutiesService { /// Proposer duties, already narrowed to `indices`. See the module doc for /// why this is a second map rather than another field on the first. proposers: HashMap>, + /// Payload timeliness committee duties, already narrowed to `indices`. + ptc: HashMap>, /// Whether the "no validator indices resolved" warning has already fired. /// Without this, an idle client would repeat it every epoch forever; with /// it, the operator still gets exactly one signal that duties are not @@ -60,6 +69,7 @@ impl DutiesService { indices, by_epoch: HashMap::new(), proposers: HashMap::new(), + ptc: HashMap::new(), warned_no_indices: false, } } @@ -197,6 +207,74 @@ impl DutiesService { Ok(()) } + /// Fetch `epoch`'s payload timeliness committee duties for this client's + /// indices, replacing anything held for it if the `dependent_root` has + /// changed. + /// + /// Only to be called for a gloas epoch: an earlier one answers with no + /// duties, which is harmless but a request wasted every epoch. + pub async fn refresh_ptc(&mut self, epoch: Epoch) -> Result<()> { + if self.indices.is_empty() { + return Ok(()); + } + + let fetched = self.beacon_node.ptc_duties(epoch, &self.indices).await?; + + let held = self.ptc.get(&epoch); + if held.is_some_and(|held| held.dependent_root == fetched.dependent_root) { + return Ok(()); + } + if held.is_some() { + warn!( + %epoch, + "Payload timeliness committee duties invalidated by a reorg; replacing the schedule" + ); + } + + // The node was asked about these indices only, but a node that answers + // for more must not make this client vote for validators it does not + // hold. + let mine: Vec = fetched + .duties + .into_iter() + .filter(|duty| self.indices.contains(&duty.validator_index)) + .collect(); + if mine.is_empty() { + tracing::debug!(%epoch, "No payload timeliness committee duties this epoch"); + } else { + info!(%epoch, count = mine.len(), "Payload timeliness committee duties updated"); + } + self.ptc.insert( + epoch, + EpochDuties { + dependent_root: fetched.dependent_root, + duties: mine, + }, + ); + Ok(()) + } + + /// Forget payload timeliness committee duties for epochs before `epoch`. + pub fn prune_ptc_before(&mut self, epoch: Epoch) { + self.ptc.retain(|held, _| *held >= epoch); + } + + /// This client's committee duties at `slot`. More than one is ordinary: a + /// client holding several validators can have several seats in one slot's + /// committee. + pub fn ptc_at_slot(&self, slot: Slot, epoch: Epoch) -> Vec { + self.ptc + .get(&epoch) + .map(|held| { + held.duties + .iter() + .filter(|duty| duty.slot == slot) + .cloned() + .collect() + }) + .unwrap_or_default() + } + /// This client's proposer duty for `slot`, if it has one. /// /// At most one, and not because of any filtering here: a slot has exactly @@ -605,4 +683,85 @@ mod tests { service.refresh_proposers(3).await.expect("no-op"); assert_eq!(node.proposer_duties_call_count(), 0); } + + fn ptc_duty(validator_index: ValidatorIndex, slot: Slot) -> PtcDutyDto { + PtcDutyDto { + pubkey: "0x00".to_string(), + validator_index, + slot, + } + } + + #[tokio::test] + async fn payload_committee_duties_are_fetched_filtered_and_indexed_by_slot() { + let node = Arc::new(MockBeaconNode::new().with_ptc_duties( + 12, + Root::repeat_byte(1), + // Validator 99 is not ours and must be dropped on arrival. + vec![ptc_duty(7, 390), ptc_duty(99, 390), ptc_duty(7, 395)], + )); + let mut service = DutiesService::new(node.clone(), vec![7]); + + service.refresh_ptc(12).await.expect("refreshes"); + + let at_390 = service.ptc_at_slot(390, 12); + assert_eq!(at_390.len(), 1); + assert_eq!(at_390[0].validator_index, 7); + assert_eq!(service.ptc_at_slot(395, 12).len(), 1); + assert!(service.ptc_at_slot(391, 12).is_empty()); + assert!( + service.ptc_at_slot(390, 13).is_empty(), + "scoped to the epoch" + ); + } + + /// A reorg past the dependent root replaces the schedule; an unchanged + /// root keeps it, which is what stops a refresh every epoch from churning. + #[tokio::test] + async fn a_changed_ptc_dependent_root_replaces_the_schedule() { + let node = Arc::new(MockBeaconNode::new().with_ptc_duties( + 12, + Root::repeat_byte(1), + vec![ptc_duty(7, 390)], + )); + let mut service = DutiesService::new(node.clone(), vec![7]); + service.refresh_ptc(12).await.expect("first"); + + node.set_ptc_duties(12, Root::repeat_byte(1), vec![ptc_duty(7, 391)]); + service.refresh_ptc(12).await.expect("same root"); + assert_eq!( + service.ptc_at_slot(390, 12).len(), + 1, + "unchanged root keeps it" + ); + + node.set_ptc_duties(12, Root::repeat_byte(2), vec![ptc_duty(7, 391)]); + service.refresh_ptc(12).await.expect("new root"); + assert!(service.ptc_at_slot(390, 12).is_empty()); + assert_eq!(service.ptc_at_slot(391, 12).len(), 1); + } + + #[tokio::test] + async fn pruning_forgets_earlier_payload_committee_epochs() { + let node = Arc::new( + MockBeaconNode::new() + .with_ptc_duties(12, Root::repeat_byte(1), vec![ptc_duty(7, 390)]) + .with_ptc_duties(13, Root::repeat_byte(1), vec![ptc_duty(7, 420)]), + ); + let mut service = DutiesService::new(node, vec![7]); + service.refresh_ptc(12).await.expect("12"); + service.refresh_ptc(13).await.expect("13"); + + service.prune_ptc_before(13); + assert!(service.ptc_at_slot(390, 12).is_empty()); + assert_eq!(service.ptc_at_slot(420, 13).len(), 1); + } + + #[tokio::test] + async fn no_indices_means_no_payload_committee_request() { + let node = Arc::new(MockBeaconNode::new()); + let mut service = DutiesService::new(node.clone(), Vec::new()); + service.refresh_ptc(12).await.expect("idles"); + assert_eq!(node.ptc_call_count(), 0); + } } diff --git a/crates/validator/src/lib.rs b/crates/validator/src/lib.rs index 9eca256e..95c2c9cb 100644 --- a/crates/validator/src/lib.rs +++ b/crates/validator/src/lib.rs @@ -39,6 +39,7 @@ pub mod error; pub mod http_api; pub mod keys; pub mod metrics; +pub mod payload_attestation; pub mod proposal; pub mod proposal_guard; pub(crate) mod secure_fs; @@ -64,6 +65,7 @@ use crate::beacon_node::fallback::FallbackBeaconNode; use crate::beacon_node::http::HttpBeaconNode; use crate::duties::DutiesService; use crate::keys::ValidatorStore; +use crate::payload_attestation::PayloadAttestationService; use crate::proposal::ProposalService; use crate::signing::SigningContext; use crate::slot_clock::SlotClock; @@ -167,12 +169,7 @@ pub async fn run(config: ValidatorConfig) -> Result<()> { "Validator client starting" ); - let clock = SlotClock::new( - genesis.genesis_time, - spec.slot_duration_ms, - spec.attestation_due_bps, - spec.aggregate_due_bps, - ); + let clock = SlotClock::from_config(genesis.genesis_time, &spec); let context = Arc::new(SigningContext { config: spec, genesis_validators_root: genesis.genesis_validators_root, @@ -181,6 +178,7 @@ pub async fn run(config: ValidatorConfig) -> Result<()> { let mut duties = DutiesService::new(beacon_node.clone(), Vec::new()); let attestation = AttestationService::new(beacon_node.clone(), context.clone()); let aggregation = AggregationService::new(beacon_node.clone(), context.clone()); + let payload_attestation = PayloadAttestationService::new(beacon_node.clone(), context.clone()); let proposal = ProposalService::new( beacon_node.clone(), context.clone(), @@ -297,72 +295,167 @@ pub async fn run(config: ValidatorConfig) -> Result<()> { tokio::time::sleep(clock.until_attestation(slot, SystemTime::now())).await; let slot_duties = duties.at_slot(slot, epoch); - if slot_duties.is_empty() { - continue; + let committee = duties.ptc_at_slot(slot, epoch); + serve_slot_duties( + &attestation, + &aggregation, + &payload_attestation, + &clock, + slot, + &slot_duties, + &committee, + &store, + ) + .await; + } +} + +/// Everything a slot owes after its proposal: attest and aggregate, then the +/// payload timeliness committee vote. +/// +/// The two halves are independent on purpose. Committee membership is drawn +/// separately from attester duties, so a client can sit on a slot's committee +/// with no attester duty in it at all, and an early return for "no attester +/// duties" would skip exactly those votes. The committee half is also gated on +/// the slot being a gloas one: before the fork there is no committee and no +/// payload to vote on. +/// +/// Three quarters into the slot the committee sleeps until its own deadline, +/// then runs bounded by the end of the slot like the other duties. +#[allow(clippy::too_many_arguments)] +async fn serve_slot_duties( + attestation: &AttestationService, + aggregation: &AggregationService, + payload_attestation: &PayloadAttestationService, + clock: &SlotClock, + slot: u64, + slot_duties: &[crate::beacon_node::dto::AttesterDutyDto], + committee: &[crate::beacon_node::dto::PtcDutyDto], + store: &RwLock, +) { + if !slot_duties.is_empty() { + attest_and_aggregate(attestation, aggregation, clock, slot, slot_duties, store).await; + } + + if clock.is_gloas(slot) && !committee.is_empty() { + tokio::time::sleep(clock.until_payload_attestation(slot, SystemTime::now())).await; + payload_attest(payload_attestation, clock, slot, committee, store).await; + } +} + +/// Run one slot's attestation duty and, when it produced data, the aggregation +/// duty that follows it. +/// +/// Split out of the loop so the loop can go on to the payload committee duty +/// whether or not this client has an attester duty in the slot. +async fn attest_and_aggregate( + attestation: &AttestationService, + aggregation: &AggregationService, + clock: &SlotClock, + slot: u64, + slot_duties: &[crate::beacon_node::dto::AttesterDutyDto], + store: &RwLock, +) { + // Bound the slot's work by what is left of the slot. + // + // Nothing else does. The per-request timeout in `HttpBeaconNode` is 8 + // seconds, failover tries each node in turn, and `attest` makes two + // calls, so one hung node can carry a 12-second slot's duty well past + // the slot itself and into the next one, whose duty is then late in + // turn. An attestation that misses its slot is worth little; one that + // also delays the next slot's is worth less than nothing. + // + // Dropping the future mid-flight is safe here specifically because of + // `AttestationGuard`: it records at signing time, so a duty abandoned + // between signing and submission cannot be re-signed next slot under + // the same target. That is the intended outcome and matches `attest`'s + // own "do not retry within a slot" rule; the attestation is simply + // lost. + let budget = clock.remaining_in(slot, SystemTime::now()); + let attempt = tokio::time::timeout(budget, attestation.attest(slot, slot_duties, store)); + match attempt.await.unwrap_or_else(|_| { + warn!( + %slot, + budget_ms = budget.as_millis() as u64, + "Attestation duty ran past the end of its slot and was abandoned" + ); + metrics::inc_attestation_deadline_missed(); + Err(Error::AttestationDeadline { slot }) + }) { + // `attest` scopes its own read guard away from every await (see + // its doc comment), so passing the shared `store` straight + // through here holds nothing across this call. + Ok(attested) => { + if attested.published > 0 { + let elapsed = SystemTime::now() + .duration_since(clock.start_of(slot)) + .unwrap_or_default(); + metrics::observe_publication_delay(elapsed.as_secs_f64()); + } + + // Aggregation, for whichever of this slot's duties this client + // was selected for. + // + // Reached only when the attestation duty produced data, since + // that data is what names the aggregate to ask for. It is not + // gated on anything having been *published*, though: an + // aggregator collects the whole committee's votes, so the duty + // is still owed when this client's own signatures were refused. + if let Some(data) = attested.data { + let delay = clock.until_aggregation(slot, SystemTime::now()); + tokio::time::sleep(delay).await; + aggregate(aggregation, clock, slot, &data, slot_duties, store).await; + } + } + Err(err) => { + error!(%slot, %err, "Failed to publish attestations for this slot"); + metrics::inc_attestation_failures(); + // Every error `attest` can return originates from the beacon + // node (a failed fetch, a bad or mismatched response, a + // failed submission), so a failure here is exactly the signal + // this gauge exists for: `refresh_epoch` only runs once an + // epoch and would otherwise leave it reporting stale + // availability for up to that long. + metrics::set_beacon_node_available(false); } + } +} - // Bound the slot's work by what is left of the slot. - // - // Nothing else does. The per-request timeout in `HttpBeaconNode` is 8 - // seconds, failover tries each node in turn, and `attest` makes two - // calls, so one hung node can carry a 12-second slot's duty well past - // the slot itself and into the next one, whose duty is then late in - // turn. An attestation that misses its slot is worth little; one that - // also delays the next slot's is worth less than nothing. - // - // Dropping the future mid-flight is safe here specifically because of - // `AttestationGuard`: it records at signing time, so a duty abandoned - // between signing and submission cannot be re-signed next slot under - // the same target. That is the intended outcome and matches `attest`'s - // own "do not retry within a slot" rule; the attestation is simply - // lost. - let budget = clock.remaining_in(slot, SystemTime::now()); - let attempt = tokio::time::timeout(budget, attestation.attest(slot, &slot_duties, &store)); - match attempt.await.unwrap_or_else(|_| { +/// Run one slot's payload timeliness committee duty, bounded by what is left of +/// the slot. +/// +/// The budget is the end of the slot, like the aggregation's and for the same +/// reason: this is the last duty in its slot, so the only thing past its +/// deadline is the next slot's work. A vote that arrives after its slot is +/// worthless, since the committee's attestations are only counted for the slot +/// they are about. +/// +/// Abandoning it is safe: the service records per validator at signing time, so +/// a duty dropped mid-flight is lost rather than re-signed. Failures are logged +/// and counted rather than propagated, like every other duty's. +async fn payload_attest( + service: &PayloadAttestationService, + clock: &SlotClock, + slot: u64, + committee: &[crate::beacon_node::dto::PtcDutyDto], + store: &RwLock, +) { + let budget = clock.remaining_in(slot, SystemTime::now()); + let attempt = tokio::time::timeout(budget, service.attest(slot, committee, store)); + match attempt.await { + Ok(Ok(_)) => {} + Ok(Err(err)) => { + error!(%slot, %err, "Failed to publish payload attestations for this slot"); + metrics::inc_payload_attestation_failures(); + metrics::set_beacon_node_available(false); + } + Err(_) => { warn!( %slot, budget_ms = budget.as_millis() as u64, - "Attestation duty ran past the end of its slot and was abandoned" + "Payload attestation duty ran past the end of its slot and was abandoned" ); - metrics::inc_attestation_deadline_missed(); - Err(Error::AttestationDeadline { slot }) - }) { - // `attest` scopes its own read guard away from every await (see - // its doc comment), so passing the shared `store` straight - // through here holds nothing across this call. - Ok(attested) => { - if attested.published > 0 { - let elapsed = SystemTime::now() - .duration_since(clock.start_of(slot)) - .unwrap_or_default(); - metrics::observe_publication_delay(elapsed.as_secs_f64()); - } - - // Aggregation, two thirds in, for whichever of this slot's - // duties this client was selected for. - // - // Reached only when the attestation duty produced data, since - // that data is what names the aggregate to ask for. It is not - // gated on anything having been *published*, though: an - // aggregator collects the whole committee's votes, so the duty - // is still owed when this client's own signatures were refused. - if let Some(data) = attested.data { - let delay = clock.until_aggregation(slot, SystemTime::now()); - tokio::time::sleep(delay).await; - aggregate(&aggregation, &clock, slot, &data, &slot_duties, &store).await; - } - } - Err(err) => { - error!(%slot, %err, "Failed to publish attestations for this slot"); - metrics::inc_attestation_failures(); - // Every error `attest` can return originates from the beacon - // node (a failed fetch, a bad or mismatched response, a - // failed submission), so a failure here is exactly the signal - // this gauge exists for: `refresh_epoch` only runs once an - // epoch and would otherwise leave it reporting stale - // availability for up to that long. - metrics::set_beacon_node_available(false); - } + metrics::inc_payload_attestation_failures(); } } } @@ -544,6 +637,7 @@ async fn refresh_epoch( } let changed = duties.refresh_around(epoch).await?; + refresh_ptc(duties, epoch, context).await; metrics::set_duties_held(duties.all().len() as u64); if changed { subscriptions::subscribe(beacon_node, &duties.all(), store, context).await?; @@ -552,6 +646,34 @@ async fn refresh_epoch( Ok(()) } +/// Fetch the payload timeliness committee schedule for this epoch and the next, +/// each only if it is a gloas epoch. +/// +/// Gated per epoch rather than once, because the fork can activate between the +/// two: the lookahead is the epoch the first gloas slots belong to when the +/// current one is the last before the fork. A failed fetch is logged and +/// shrugged off, like the proposer and lookahead ones: it must not cost this +/// epoch's attester schedule, which is already in hand. +async fn refresh_ptc( + duties: &mut DutiesService, + epoch: u64, + context: &SigningContext, +) { + for target in [epoch, epoch + 1] { + if context.config.fork_at_epoch(target) < ethlambda_types::beacon::fork::ForkName::Gloas { + continue; + } + if let Err(err) = duties.refresh_ptc(target).await { + warn!( + epoch = target, + %err, + "Payload committee duties fetch failed; this client may miss its votes" + ); + } + } + duties.prune_ptc_before(epoch); +} + #[cfg(test)] mod tests { use super::*; @@ -732,4 +854,186 @@ mod tests { refresh(node.clone(), Some(H160([0xab; 20]))).await; assert!(node.preparations().is_empty()); } + + // ---- gloas ---------------------------------------------------------- + + fn gloas_context(epoch: u64) -> SigningContext { + SigningContext { + config: ethlambda_types::beacon::config::Config::mainnet() + .with_fork_epoch(ethlambda_types::beacon::fork::ForkName::Gloas, epoch), + genesis_validators_root: Root::ZERO, + } + } + + fn ptc_duty(index: u64, slot: u64) -> crate::beacon_node::dto::PtcDutyDto { + crate::beacon_node::dto::PtcDutyDto { + pubkey: crate::beacon_node::dto::encode_hex(&[0u8; 48]), + validator_index: index, + slot, + } + } + + /// The committee schedule is fetched for gloas epochs only, so a chain that + /// has not forked is never asked, and the lookahead is gated on its own + /// epoch: the last pre-gloas epoch looks ahead into the first gloas one. + #[tokio::test] + async fn the_committee_schedule_is_fetched_for_gloas_epochs_only() { + let node = Arc::new( + node() + .with_ptc_duties(4, Root::repeat_byte(1), vec![ptc_duty(11, 130)]) + .with_ptc_duties(5, Root::repeat_byte(1), Vec::new()), + ); + let mut duties = DutiesService::new(node.clone(), Vec::new()); + let keys = [pubkey(1), pubkey(2)]; + + // Epoch 3 is the last before gloas at epoch 4: only the lookahead asks. + node.set_duties(3, Root::repeat_byte(1), Vec::new()); + refresh_epoch( + &node, + &mut duties, + &keys, + 3, + None, + &empty_store(), + &gloas_context(4), + ) + .await + .expect("refreshes"); + assert_eq!( + node.ptc_call_count(), + 1, + "only epoch 4, the first gloas one" + ); + assert_eq!(duties.ptc_at_slot(130, 4).len(), 1); + + // A chain with no gloas scheduled is never asked at all. + let node = Arc::new(node_without_ptc()); + let mut duties = DutiesService::new(node.clone(), Vec::new()); + refresh_epoch( + &node, + &mut duties, + &keys, + 3, + None, + &empty_store(), + &context(), + ) + .await + .expect("refreshes"); + assert_eq!(node.ptc_call_count(), 0); + } + + fn node_without_ptc() -> MockBeaconNode { + node() + } + + /// A failed committee fetch must not cost the epoch's attester schedule. + #[tokio::test] + async fn a_failed_committee_fetch_does_not_fail_the_refresh() { + let node = Arc::new(node().failing_call("ptc_duties", "node is unhappy")); + let mut duties = DutiesService::new(node.clone(), Vec::new()); + let keys = [pubkey(1), pubkey(2)]; + refresh_epoch( + &node, + &mut duties, + &keys, + 3, + None, + &empty_store(), + &gloas_context(0), + ) + .await + .expect("the refresh survives it"); + assert_eq!(duties.indices(), &[11, 22]); + } + + /// The restructured loop: a client on a slot's committee with no attester + /// duty at all must still vote. This is the case the old early `continue` + /// on "no attester duties" would have skipped. + #[tokio::test] + async fn a_committee_member_with_no_attester_duty_still_votes() { + use crate::beacon_node::mock::MockBeaconNode; + use ethlambda_types::beacon::containers::gloas::PayloadAttestationData; + + let mut config = ethlambda_types::beacon::config::Config::mainnet() + .with_fork_epoch(ethlambda_types::beacon::fork::ForkName::Gloas, 0); + config.slot_duration_ms = 1_000; + let context = Arc::new(SigningContext { + config: config.clone(), + genesis_validators_root: Root::ZERO, + }); + let now_secs = SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("after the epoch") + .as_secs(); + let clock = SlotClock::from_config(now_secs - 100, &config); + let slot = clock.now().expect("after genesis") + 1; + + let mut store = ValidatorStore::new(); + let secret: [u8; 32] = + hex::decode("000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f") + .expect("hex") + .try_into() + .expect("32 bytes"); + let key = store.insert_secret("test", &secret).expect("inserts"); + let store = RwLock::new(store); + + let node = Arc::new(MockBeaconNode::new().with_payload_attestation_data( + PayloadAttestationData { + beacon_block_root: Root::repeat_byte(5), + slot, + payload_present: true, + blob_data_available: true, + }, + )); + let committee = [crate::beacon_node::dto::PtcDutyDto { + pubkey: crate::beacon_node::dto::encode_hex(&key.0), + validator_index: 3, + slot, + }]; + + serve_slot_duties( + &AttestationService::new(node.clone(), context.clone()), + &AggregationService::new(node.clone(), context.clone()), + &PayloadAttestationService::new(node.clone(), context.clone()), + &clock, + slot, + &[], + &committee, + &store, + ) + .await; + + assert_eq!(node.attestation_data_call_count(), 0, "no attester duty"); + assert_eq!(node.submitted_payload_attestations().len(), 1); + } + + /// Before gloas there is no committee: even a stale schedule entry is not + /// acted on. + #[tokio::test] + async fn no_committee_vote_is_cast_before_gloas() { + use crate::beacon_node::mock::MockBeaconNode; + + let config = ethlambda_types::beacon::config::Config::mainnet(); + let context = Arc::new(SigningContext { + config: config.clone(), + genesis_validators_root: Root::ZERO, + }); + let clock = SlotClock::from_config(0, &config); + let node = Arc::new(MockBeaconNode::new()); + let store = empty_store(); + + serve_slot_duties( + &AttestationService::new(node.clone(), context.clone()), + &AggregationService::new(node.clone(), context.clone()), + &PayloadAttestationService::new(node.clone(), context.clone()), + &clock, + 5, + &[], + &[ptc_duty(3, 5)], + &store, + ) + .await; + assert_eq!(node.payload_attestation_data_call_count(), 0); + } } diff --git a/crates/validator/src/metrics.rs b/crates/validator/src/metrics.rs index d8455d95..a47da859 100644 --- a/crates/validator/src/metrics.rs +++ b/crates/validator/src/metrics.rs @@ -139,6 +139,42 @@ static BLOCK_PROPOSAL_FAILURES_TOTAL: LazyLock = LazyLock::new(|| { .unwrap() }); +/// Envelopes of self-built gloas payloads the node accepted. +static ENVELOPES_PUBLISHED_TOTAL: LazyLock = LazyLock::new(|| { + register_int_counter!( + "ethlambda_validator_envelopes_published_total", + "Execution payload envelopes published for blocks this client proposed" + ) + .unwrap() +}); + +/// Gloas proposals whose block went out but whose envelope did not: the slot's +/// payload is then withheld, which costs the proposer its execution rewards and +/// the chain a payload. +static ENVELOPE_FAILURES_TOTAL: LazyLock = LazyLock::new(|| { + register_int_counter!( + "ethlambda_validator_envelope_failures_total", + "Proposed blocks whose self-built envelope was not published" + ) + .unwrap() +}); + +static PAYLOAD_ATTESTATIONS_PUBLISHED_TOTAL: LazyLock = LazyLock::new(|| { + register_int_counter!( + "ethlambda_validator_payload_attestations_published_total", + "Payload timeliness committee votes the beacon node accepted" + ) + .unwrap() +}); + +static PAYLOAD_ATTESTATION_FAILURES_TOTAL: LazyLock = LazyLock::new(|| { + register_int_counter!( + "ethlambda_validator_payload_attestation_failures_total", + "Slots whose payload timeliness committee duty did not result in published votes" + ) + .unwrap() +}); + /// End to end, from the slot's start to the node accepting the block. /// /// The buckets are tighter at the low end than the attestation histogram's, @@ -242,6 +278,10 @@ pub fn init() { LazyLock::force(&BLOCKS_BROADCAST_NOT_IMPORTED_TOTAL); LazyLock::force(&BLOCKS_REFUSED_TOTAL); LazyLock::force(&BLOCK_PROPOSAL_FAILURES_TOTAL); + LazyLock::force(&ENVELOPES_PUBLISHED_TOTAL); + LazyLock::force(&ENVELOPE_FAILURES_TOTAL); + LazyLock::force(&PAYLOAD_ATTESTATIONS_PUBLISHED_TOTAL); + LazyLock::force(&PAYLOAD_ATTESTATION_FAILURES_TOTAL); LazyLock::force(&BLOCK_PUBLICATION_DELAY_SECONDS); LazyLock::force(&AGGREGATES_PUBLISHED_TOTAL); LazyLock::force(&AGGREGATION_FAILURES_TOTAL); @@ -299,6 +339,22 @@ pub fn inc_block_proposal_failures() { BLOCK_PROPOSAL_FAILURES_TOTAL.inc(); } +pub fn inc_envelopes_published() { + ENVELOPES_PUBLISHED_TOTAL.inc(); +} + +pub fn inc_envelope_failures() { + ENVELOPE_FAILURES_TOTAL.inc(); +} + +pub fn inc_payload_attestations_published(count: u64) { + PAYLOAD_ATTESTATIONS_PUBLISHED_TOTAL.inc_by(count); +} + +pub fn inc_payload_attestation_failures() { + PAYLOAD_ATTESTATION_FAILURES_TOTAL.inc(); +} + /// Record how long after a slot's start its block was accepted. pub fn observe_block_publication_delay(seconds: f64) { BLOCK_PUBLICATION_DELAY_SECONDS.observe(seconds); diff --git a/crates/validator/src/payload_attestation.rs b/crates/validator/src/payload_attestation.rs new file mode 100644 index 00000000..0870c038 --- /dev/null +++ b/crates/validator/src/payload_attestation.rs @@ -0,0 +1,379 @@ +//! Voting on whether a slot's payload arrived: the payload timeliness +//! committee (PTC) duty that gloas adds. +//! +//! A small committee is drawn each slot, and its members attest to two facts +//! about the block they saw: whether the builder revealed the payload on time +//! (`payload_present`) and whether the blob data was available +//! (`blob_data_available`). The beacon node assembles those facts from its own +//! view, so this client does not judge them: it asks for the data, checks it is +//! about the right slot, signs it and submits it, which is the same division of +//! labour the attestation duty has. +//! +//! # Timing +//! +//! Due at `PAYLOAD_ATTESTATION_DUE_BPS` of the slot, three quarters of the way +//! in, which is when the node's view of the payload is as complete as it will +//! be. See [`crate::slot_clock::SlotClock::until_payload_attestation`]. +//! +//! # No block, no vote +//! +//! A node that knows no block for the slot answers 204. That is not an error +//! and not a vote against the payload: there is nothing to attest to, so +//! nothing is signed. Voting `payload_present = false` for a block this client +//! never saw would be a claim about a block it cannot name. +//! +//! # Duplicates +//! +//! The specification attaches no slashing condition to a PTC vote, but a second, +//! different vote from one member for one slot is equivocation the network +//! penalises in gossip. So the slot is recorded per validator at signing time, +//! the way the attestation guard records, and a duty abandoned between signing +//! and submission is lost rather than re-signed. In memory only: this is not +//! slashing protection, and does not pretend to be. + +use std::collections::HashSet; +use std::sync::Arc; + +use ethlambda_types::beacon::containers::gloas::PayloadAttestationMessage; +use ethlambda_types::beacon::primitives::{Slot, ValidatorIndex}; +use tokio::sync::RwLock; +use tracing::{error, info, warn}; + +use crate::beacon_node::BeaconNodeApi; +use crate::beacon_node::dto::{PtcDutyDto, parse_pubkey}; +use crate::error::Result; +use crate::keys::ValidatorStore; +use crate::signing::SigningContext; + +/// How many slots of dedup history are kept. Far more than the one slot a duty +/// is ever owed for; the margin only absorbs a clock that steps back a little. +const HISTORY_SLOTS: u64 = 64; + +pub struct PayloadAttestationService { + beacon_node: Arc, + context: Arc, + /// Which (validator, slot) pairs already produced a signature. + /// + /// A `std::sync::Mutex` for the reason the attestation service's guard is: + /// the critical section is a lookup and an insert with no await inside it. + done: std::sync::Mutex>, +} + +impl PayloadAttestationService { + pub fn new(beacon_node: Arc, context: Arc) -> Self { + Self { + beacon_node, + context, + done: std::sync::Mutex::new(HashSet::new()), + } + } + + /// Sign and submit this client's committee votes for `slot`. + /// + /// Returns how many votes the node accepted: zero when the node has no + /// block for the slot, or when every member was already served. + /// + /// # Do not retry this call within a slot + /// + /// For the reason the attestation duty's is not retried: a second call + /// fetches the data again, and a head that moved in between would give a + /// different vote from the same member. The dedup refuses it, so a retry is + /// merely useless, but callers should not rely on it. + pub async fn attest( + &self, + slot: Slot, + duties: &[PtcDutyDto], + store: &RwLock, + ) -> Result { + if duties.is_empty() { + return Ok(0); + } + + // Cheap pre-check, so a slot already served costs the node no request. + let pending: Vec<&PtcDutyDto> = { + let done = self.lock(); + duties + .iter() + .filter(|duty| !done.contains(&(duty.validator_index, slot))) + .collect() + }; + if pending.is_empty() { + return Ok(0); + } + + let Some(data) = self.beacon_node.payload_attestation_data(slot).await? else { + info!(%slot, "No block known for this slot; not voting on its payload"); + return Ok(0); + }; + + // Checked again here, having already been checked by the + // implementation (see `BeaconNodeApi::payload_attestation_data`), for + // the reason the attestation duty does: the trait is public, and this + // is the last point before the data becomes a signature. + if data.slot != slot { + return Err(crate::error::Error::InconsistentResponse(format!( + "requested payload attestation data for slot {slot}, node answered for slot {}", + data.slot + ))); + } + + let mut messages = Vec::with_capacity(pending.len()); + { + let _timing = crate::metrics::time_signing(); + let store = store.read().await; + for duty in pending { + let pubkey = match parse_pubkey(&duty.pubkey) { + Ok(pubkey) => pubkey, + Err(err) => { + error!(%slot, validator = duty.validator_index, %err, "Duty carried an unreadable pubkey"); + continue; + } + }; + + // Recorded before signing, so a signature that exists is one + // the record already knows about. + if !self.record(duty.validator_index, slot) { + continue; + } + + match self + .context + .sign_payload_attestation(&store, &pubkey, &data) + { + Ok(signature) => messages.push(PayloadAttestationMessage { + validator_index: duty.validator_index, + data, + signature, + }), + Err(err) => { + error!(%slot, validator = duty.validator_index, %err, "Failed to sign payload attestation"); + crate::metrics::inc_signing_failures(); + } + } + } + } + + if messages.is_empty() { + return Ok(0); + } + + let submitted = messages.len(); + let published = self + .beacon_node + .submit_payload_attestations(&messages) + .await?; + if published < submitted { + warn!( + %slot, + published, + submitted, + "Some payload attestations in this slot's batch were rejected by the beacon node" + ); + } + info!( + %slot, + count = published, + payload_present = data.payload_present, + blob_data_available = data.blob_data_available, + "Published payload attestations" + ); + crate::metrics::inc_payload_attestations_published(published as u64); + Ok(published) + } + + /// Record that `validator` is being served for `slot`. `false` when it + /// already was, or when the record is unusable, which is treated as a + /// refusal for the reason the other guards' poisoned locks are. + fn record(&self, validator: ValidatorIndex, slot: Slot) -> bool { + let Ok(mut done) = self.done.lock() else { + error!(%slot, "Payload attestation record is poisoned; refusing to sign"); + return false; + }; + done.retain(|(_, held)| held + HISTORY_SLOTS > slot); + done.insert((validator, slot)) + } + + fn lock(&self) -> std::sync::MutexGuard<'_, HashSet<(ValidatorIndex, Slot)>> { + // A poisoned set is still a set: the worst a panic mid-insert could + // leave is a missing entry, and the signing path re-checks through + // `record`, which refuses on poison. + self.done + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::beacon_node::dto::encode_hex; + use crate::beacon_node::mock::MockBeaconNode; + use crate::error::Error; + use ethlambda_types::beacon::config::Config; + use ethlambda_types::beacon::containers::gloas::PayloadAttestationData; + use ethlambda_types::beacon::fork::ForkName; + use ethlambda_types::beacon::preset::SLOTS_PER_EPOCH; + use ethlambda_types::beacon::primitives::{BlsPubkey, Root}; + + const GLOAS_EPOCH: u64 = 10; + + fn secret() -> [u8; 32] { + hex::decode("000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f") + .expect("valid hex") + .try_into() + .expect("32 bytes") + } + + fn context() -> Arc { + Arc::new(SigningContext { + config: Config::mainnet().with_fork_epoch(ForkName::Gloas, GLOAS_EPOCH), + genesis_validators_root: Root::ZERO, + }) + } + + fn store() -> (RwLock, BlsPubkey) { + let mut store = ValidatorStore::new(); + let pubkey = store.insert_secret("test", &secret()).expect("inserts"); + (RwLock::new(store), pubkey) + } + + fn slot() -> Slot { + GLOAS_EPOCH * SLOTS_PER_EPOCH + 4 + } + + fn data(slot: Slot) -> PayloadAttestationData { + PayloadAttestationData { + beacon_block_root: Root::repeat_byte(5), + slot, + payload_present: true, + blob_data_available: false, + } + } + + fn duty(pubkey: &BlsPubkey, validator_index: u64, slot: Slot) -> PtcDutyDto { + PtcDutyDto { + pubkey: encode_hex(&pubkey.0), + validator_index, + slot, + } + } + + #[tokio::test] + async fn a_committee_member_signs_and_submits_the_nodes_data() { + use blst::min_pk::{PublicKey, Signature}; + + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_payload_attestation_data(data(slot()))); + let service = PayloadAttestationService::new(node.clone(), context()); + + let published = service + .attest(slot(), &[duty(&pubkey, 9, slot())], &store) + .await + .expect("attests"); + assert_eq!(published, 1); + + let messages = node.submitted_payload_attestations(); + assert_eq!(messages.len(), 1); + assert_eq!(messages[0].validator_index, 9); + assert_eq!(messages[0].data, data(slot())); + + // Over the data's root alone, under the PTC domain at the data's epoch. + let root = context().payload_attestation_signing_root(&messages[0].data); + let pk = PublicKey::from_bytes(&pubkey.0).expect("valid pubkey"); + let sig = Signature::from_bytes(&messages[0].signature.0).expect("valid signature"); + assert_eq!( + sig.verify( + true, + root.as_slice(), + b"BLS_SIG_BLS12381G2_XMD:SHA-256_SSWU_RO_POP_", + &[], + &pk, + true + ), + blst::BLST_ERROR::BLST_SUCCESS + ); + } + + /// A 204 is an ordinary answer: nothing is signed and nothing fails. + #[tokio::test] + async fn no_block_for_the_slot_means_no_vote() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new()); + let service = PayloadAttestationService::new(node.clone(), context()); + + let published = service + .attest(slot(), &[duty(&pubkey, 9, slot())], &store) + .await + .expect("a 204 is not an error"); + assert_eq!(published, 0); + assert_eq!(node.payload_attestation_data_call_count(), 1); + assert!(node.submitted_payload_attestations().is_empty()); + } + + /// One vote per (validator, slot): the second call neither asks the node + /// nor signs again. + #[tokio::test] + async fn a_member_is_served_once_per_slot() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_payload_attestation_data(data(slot()))); + let service = PayloadAttestationService::new(node.clone(), context()); + let duties = [duty(&pubkey, 9, slot())]; + + service + .attest(slot(), &duties, &store) + .await + .expect("first"); + let again = service + .attest(slot(), &duties, &store) + .await + .expect("second"); + + assert_eq!(again, 0); + assert_eq!(node.submitted_payload_attestations().len(), 1); + assert_eq!(node.payload_attestation_data_call_count(), 1); + } + + /// A new slot is a new vote, and a second member of the same slot is + /// served even after the first. + #[tokio::test] + async fn dedup_is_per_validator_and_per_slot() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_payload_attestation_data(data(slot()))); + let service = PayloadAttestationService::new(node.clone(), context()); + + service + .attest(slot(), &[duty(&pubkey, 9, slot())], &store) + .await + .expect("first member"); + service + .attest(slot(), &[duty(&pubkey, 10, slot())], &store) + .await + .expect("second member"); + assert_eq!(node.submitted_payload_attestations().len(), 2); + } + + /// Data about another slot is never signed. + #[tokio::test] + async fn data_for_another_slot_is_rejected() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_payload_attestation_data(data(slot() - 1))); + let service = PayloadAttestationService::new(node.clone(), context()); + + let err = service + .attest(slot(), &[duty(&pubkey, 9, slot())], &store) + .await + .expect_err("must reject"); + assert!(matches!(err, Error::InconsistentResponse(_)), "got {err:?}"); + assert!(node.submitted_payload_attestations().is_empty()); + } + + #[tokio::test] + async fn no_duties_asks_the_node_for_nothing() { + let (store, _) = store(); + let node = Arc::new(MockBeaconNode::new().with_payload_attestation_data(data(slot()))); + let service = PayloadAttestationService::new(node.clone(), context()); + + assert_eq!(service.attest(slot(), &[], &store).await.expect("no-op"), 0); + assert_eq!(node.payload_attestation_data_call_count(), 0); + } +} diff --git a/crates/validator/src/proposal.rs b/crates/validator/src/proposal.rs index 91993f7c..af61f58e 100644 --- a/crates/validator/src/proposal.rs +++ b/crates/validator/src/proposal.rs @@ -29,6 +29,16 @@ //! deadline safe: a proposal abandoned between signing and publishing cannot be //! re-signed, because the guard already counts that slot as proposed. //! +//! # Gloas: the block, then its envelope +//! +//! From gloas the block commits to a bid and the payload is revealed in an +//! envelope the builder signs. This client only self-builds: when the bid names +//! the self-build sentinel, the proposer's own key signs the envelope under the +//! builder domain, and it is published right after the block, in the same +//! budget. A bid naming anyone else is a builder's to reveal, and is left alone. +//! +//! The envelope is not slashable, so the guard is untouched by it. +//! //! # Do not retry this call within a slot //! //! For the same reason [`crate::attestation::AttestationService::attest`] must @@ -40,13 +50,15 @@ use std::sync::Arc; +use ethlambda_types::beacon::constants::BUILDER_INDEX_SELF_BUILD; use ethlambda_types::beacon::fork::ForkName; -use ethlambda_types::beacon::primitives::{Bytes32, ExecutionAddress, HashTreeRoot as _, Slot}; +use ethlambda_types::beacon::primitives::{BlsPubkey, Bytes32, ExecutionAddress, Root, Slot}; use ethlambda_types::beacon::signing::compute_epoch_at_slot; use tokio::sync::RwLock; use tracing::{info, warn}; -use crate::beacon_node::dto::{ProposerDutyDto, parse_pubkey}; +use crate::beacon_node::block_contents::{GloasPayload, signed_envelope_ssz}; +use crate::beacon_node::dto::{ProposerDutyDto, encode_hex, parse_pubkey}; use crate::beacon_node::{BeaconNodeApi, BlockRequest, Published, validate_produced_block}; use crate::error::{Error, Result}; use crate::keys::ValidatorStore; @@ -124,7 +136,9 @@ impl ProposalService { let Some(expected) = self.fee_recipient else { return; }; - let actual = produced.block().body.execution_payload.fee_recipient; + // The payload's before gloas, the bid's from it on: the same address + // the execution layer is told to pay, read where the block keeps it. + let actual = produced.fee_recipient(); if actual != expected { tracing::error!( %slot, @@ -195,11 +209,12 @@ impl ProposalService { let request = BlockRequest { slot, + fork: self.context.config.fork_at_epoch(epoch), proposer_index: duty.validator_index, randao_reveal, graffiti: self.graffiti, }; - let produced = self.beacon_node.produce_block(&request).await?; + let mut produced = self.beacon_node.produce_block(&request).await?; // Checked again here, having already been checked by the // implementation this call went through. Not redundant, and the @@ -249,7 +264,7 @@ impl ProposalService { // The root is taken from the decoded container, never from anything // reassembled here, and it is taken once: the same value is what the // guard's slot is recorded against and what the signature covers. - let block_root = produced.block().hash_tree_root(); + let block_root = produced.block_root(); let signature = { let store = store.read().await; @@ -278,14 +293,111 @@ impl ProposalService { self.context.sign_block(&store, &pubkey, block_root, slot)? }; + // Taken out before the block is signed into its body: the envelope is + // published on its own, after the block, and the bare gloas block body + // has no place for it. + let payload = produced.take_gloas_payload(); + let builder_index = produced.builder_index(); + let body = produced.into_signed_ssz(signature); let published = self .publish(fork, &body, slot, duty.validator_index) .await?; crate::metrics::inc_blocks_proposed(); + + // Only a self-built payload is this client's to reveal. + if builder_index == Some(BUILDER_INDEX_SELF_BUILD) { + // Logged and counted rather than propagated. The block is out and + // the proposal did happen; what failed is the reveal, which has its + // own series because it is the failure that costs the payload. + let _ = self + .publish_envelope( + slot, + duty.validator_index, + &pubkey, + block_root, + payload, + store, + ) + .await + .inspect(|()| crate::metrics::inc_envelopes_published()) + .inspect_err(|err| { + tracing::error!( + %slot, + validator = duty.validator_index, + block_root = %ethlambda_types::ShortRoot(&block_root.0), + %err, + "Block published but its envelope was not; the slot's payload is withheld" + ); + crate::metrics::inc_envelope_failures(); + }); + } else if let Some(builder_index) = builder_index { + info!( + %slot, + builder_index, + "Block commits to a builder's bid; leaving the payload reveal to that builder" + ); + } Ok(published) } + /// Reveal the self-built payload of the block just published. + /// + /// `payload` is what `produceBlockV4` returned with the block, or `None` + /// when the node did not include it, in which case it is fetched by + /// `(slot, block_root)` from the node's cache. Either way it is checked to + /// be for this block and this self-build before it is signed: the signature + /// covers the envelope as it stands, so one for another block would be a + /// valid signature on the wrong payload. + async fn publish_envelope( + &self, + slot: Slot, + validator: u64, + pubkey: &BlsPubkey, + block_root: Root, + payload: Option, + store: &RwLock, + ) -> Result<()> { + let included = payload.is_some(); + let envelope = match &payload { + Some(payload) => payload.envelope.clone(), + None => { + self.beacon_node + .execution_payload_envelope(slot, block_root) + .await? + } + }; + if envelope.beacon_block_root != block_root { + return Err(Error::InconsistentResponse(format!( + "envelope for slot {slot} reveals the payload of block {}, not of the block \ + this client proposed ({})", + encode_hex(&envelope.beacon_block_root.0), + encode_hex(&block_root.0) + ))); + } + if envelope.builder_index != BUILDER_INDEX_SELF_BUILD { + return Err(Error::InconsistentResponse(format!( + "envelope for slot {slot} names builder {}, but the bid was a self-build", + envelope.builder_index + ))); + } + + let signature = { + let store = store.read().await; + self.context + .sign_execution_payload_envelope(&store, pubkey, &envelope, slot)? + }; + let body = match payload { + Some(payload) => payload.into_signed_contents_ssz(signature), + None => signed_envelope_ssz(envelope, signature), + }; + self.beacon_node + .publish_execution_payload_envelope(&body, included) + .await?; + info!(%slot, validator, bytes = body.len(), blob_data_included = included, "Envelope published"); + Ok(()) + } + /// Send the signed body and report what the node made of it. /// /// Split out so the 202 case has one place to be explained rather than @@ -328,6 +440,7 @@ mod tests { use crate::beacon_node::block_contents::SignedBlockContents; use crate::beacon_node::mock::MockBeaconNode; use ethlambda_types::beacon::config::Config; + use ethlambda_types::beacon::primitives::HashTreeRoot as _; use ethlambda_types::beacon::primitives::{BlsPubkey, H160, Root}; use libssz::SszDecode as _; @@ -641,3 +754,249 @@ mod tests { assert_eq!(node.published_blocks().len(), 1); } } + +#[cfg(test)] +mod gloas_tests { + use super::*; + use crate::beacon_node::block_contents::SignedExecutionPayloadEnvelopeContents; + use crate::beacon_node::mock::MockBeaconNode; + use ethlambda_types::beacon::config::Config; + use ethlambda_types::beacon::containers::gloas; + use ethlambda_types::beacon::preset::SLOTS_PER_EPOCH; + use ethlambda_types::beacon::primitives::{BlsPubkey, HashTreeRoot as _}; + use libssz::SszDecode as _; + + const GLOAS_EPOCH: u64 = 10; + const DST: &[u8] = b"BLS_SIG_BLS12381G2_XMD:SHA-256_SSWU_RO_POP_"; + + fn secret() -> [u8; 32] { + hex::decode("000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f") + .expect("valid hex") + .try_into() + .expect("32 bytes") + } + + fn context() -> Arc { + Arc::new(SigningContext { + config: Config::mainnet().with_fork_epoch(ForkName::Gloas, GLOAS_EPOCH), + genesis_validators_root: Root::ZERO, + }) + } + + fn store() -> (RwLock, BlsPubkey) { + let mut store = ValidatorStore::new(); + let pubkey = store.insert_secret("test", &secret()).expect("inserts"); + (RwLock::new(store), pubkey) + } + + fn slot() -> Slot { + GLOAS_EPOCH * SLOTS_PER_EPOCH + 5 + } + + fn duty(pubkey: &BlsPubkey) -> ProposerDutyDto { + ProposerDutyDto { + pubkey: encode_hex(&pubkey.0), + validator_index: 7, + slot: slot(), + } + } + + fn service(node: Arc) -> ProposalService { + ProposalService::new(node, context(), Bytes32::repeat_byte(0xab), None) + } + + fn verifies(pubkey: &BlsPubkey, signature: &[u8; 96], root: Root) -> bool { + use blst::min_pk::{PublicKey, Signature}; + let pk = PublicKey::from_bytes(&pubkey.0).expect("valid pubkey"); + let sig = Signature::from_bytes(signature).expect("valid signature"); + sig.verify(true, root.as_slice(), DST, &[], &pk, true) == blst::BLST_ERROR::BLST_SUCCESS + } + + /// The block goes out bare, under the gloas header, signed over the gloas + /// block's root, and then the envelope follows with the blobs, signed under + /// the builder domain by the proposer's own key. + #[tokio::test] + async fn a_self_built_block_is_published_and_then_its_envelope() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_gloas_block( + slot(), + 7, + BUILDER_INDEX_SELF_BUILD, + true, + )); + + let published = service(node.clone()) + .propose(slot(), &duty(&pubkey), &store) + .await + .expect("proposes"); + assert_eq!(published, Published::Imported); + + let request = node.block_requests().remove(0); + assert_eq!(request.fork, ForkName::Gloas); + + let blocks = node.published_blocks(); + assert_eq!(blocks.len(), 1); + assert_eq!(blocks[0].0, ForkName::Gloas); + let signed = gloas::SignedBeaconBlock::from_ssz_bytes(&blocks[0].1) + .expect("the body is a bare gloas signed block"); + let block_root = signed.message.hash_tree_root(); + let block_signing_root = context().block_signing_root(block_root, slot()); + assert!(verifies(&pubkey, &signed.signature.0, block_signing_root)); + + let envelopes = node.published_envelopes(); + assert_eq!(envelopes.len(), 1); + let (body, blob_data_included) = &envelopes[0]; + assert!(*blob_data_included, "the contents form carries the blobs"); + let contents = SignedExecutionPayloadEnvelopeContents::from_ssz_bytes(body) + .expect("the body is the signed envelope contents"); + let envelope = &contents.signed_execution_payload_envelope; + assert_eq!(envelope.message.beacon_block_root, block_root); + assert!( + verifies( + &pubkey, + &envelope.signature.0, + context().envelope_signing_root(&envelope.message, slot()) + ), + "the envelope must be signed under the builder domain" + ); + assert!( + node.envelope_requests().is_empty(), + "an included payload needs no fetch" + ); + } + + /// The builder domain is not the proposer domain: an envelope signature + /// must not verify as a block signature over the same root. + #[tokio::test] + async fn the_envelope_signature_is_under_the_builder_domain_not_the_proposers() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_gloas_block( + slot(), + 7, + BUILDER_INDEX_SELF_BUILD, + true, + )); + service(node.clone()) + .propose(slot(), &duty(&pubkey), &store) + .await + .expect("proposes"); + + let (body, _) = node.published_envelopes().remove(0); + let contents = SignedExecutionPayloadEnvelopeContents::from_ssz_bytes(&body).expect("ok"); + let envelope = contents.signed_execution_payload_envelope; + let as_proposer = context().block_signing_root(envelope.message.hash_tree_root(), slot()); + assert!(!verifies(&pubkey, &envelope.signature.0, as_proposer)); + } + + /// Without `Eth-Execution-Payload-Included` the envelope is fetched by + /// (slot, block root) and published bare, with the blob header false. + #[tokio::test] + async fn an_envelope_the_node_did_not_include_is_fetched_and_published_bare() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_gloas_block( + slot(), + 7, + BUILDER_INDEX_SELF_BUILD, + false, + )); + + service(node.clone()) + .propose(slot(), &duty(&pubkey), &store) + .await + .expect("proposes"); + + let signed = gloas::SignedBeaconBlock::from_ssz_bytes(&node.published_blocks()[0].1) + .expect("bare block"); + let block_root = signed.message.hash_tree_root(); + assert_eq!(node.envelope_requests(), vec![(slot(), block_root)]); + + let (body, blob_data_included) = node.published_envelopes().remove(0); + assert!(!blob_data_included); + let envelope = + gloas::SignedExecutionPayloadEnvelope::from_ssz_bytes(&body).expect("bare envelope"); + assert_eq!(envelope.message.beacon_block_root, block_root); + assert!(verifies( + &pubkey, + &envelope.signature.0, + context().envelope_signing_root(&envelope.message, slot()) + )); + } + + /// A bid naming a builder is that builder's to reveal; this client signs + /// nothing for it. + #[tokio::test] + async fn a_builders_bid_gets_no_envelope_from_this_client() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_gloas_block(slot(), 7, 3, true)); + + service(node.clone()) + .propose(slot(), &duty(&pubkey), &store) + .await + .expect("proposes"); + assert_eq!(node.published_blocks().len(), 1); + assert!(node.published_envelopes().is_empty()); + assert!(node.envelope_requests().is_empty()); + } + + /// An envelope for another block must never be signed. The block is out + /// already, so the proposal still reports it, but nothing is revealed. + #[tokio::test] + async fn an_envelope_for_another_block_is_not_signed() { + let (store, pubkey) = store(); + let node = Arc::new( + MockBeaconNode::new() + .with_gloas_block(slot(), 7, BUILDER_INDEX_SELF_BUILD, false) + .with_envelope_for(Root::repeat_byte(9)), + ); + + let result = service(node.clone()) + .propose(slot(), &duty(&pubkey), &store) + .await; + assert!(result.is_ok(), "the block was published"); + assert_eq!(node.published_blocks().len(), 1); + assert!(node.published_envelopes().is_empty()); + } + + /// The guard's semantics are unchanged: a second proposal for the slot is + /// refused before asking for anything, and reveals no second envelope. + #[tokio::test] + async fn a_repeated_gloas_slot_is_refused_and_reveals_nothing_more() { + let (store, pubkey) = store(); + let node = Arc::new(MockBeaconNode::new().with_gloas_block( + slot(), + 7, + BUILDER_INDEX_SELF_BUILD, + true, + )); + let service = service(node.clone()); + service + .propose(slot(), &duty(&pubkey), &store) + .await + .expect("first"); + let err = service + .propose(slot(), &duty(&pubkey), &store) + .await + .expect_err("a second block for one slot must be refused"); + assert!(matches!(err, Error::ProposalRefused { .. }), "got {err:?}"); + assert_eq!(node.block_requests().len(), 1); + assert_eq!(node.published_envelopes().len(), 1); + } + + /// The block is out by the time the envelope fails, so the proposal is + /// still reported as published; the failure is logged and counted. + #[tokio::test] + async fn a_failed_envelope_publication_does_not_fail_the_proposal() { + let (store, pubkey) = store(); + let node = Arc::new( + MockBeaconNode::new() + .with_gloas_block(slot(), 7, BUILDER_INDEX_SELF_BUILD, true) + .failing_call("publish_execution_payload_envelope", "node is unhappy"), + ); + let published = service(node.clone()) + .propose(slot(), &duty(&pubkey), &store) + .await + .expect("the block was published"); + assert_eq!(published, Published::Imported); + assert!(node.published_envelopes().is_empty()); + } +} diff --git a/crates/validator/src/signing.rs b/crates/validator/src/signing.rs index 533b5547..30563481 100644 --- a/crates/validator/src/signing.rs +++ b/crates/validator/src/signing.rs @@ -7,8 +7,11 @@ use ethlambda_types::beacon::config::Config; use ethlambda_types::beacon::constants::{ - DOMAIN_AGGREGATE_AND_PROOF, DOMAIN_BEACON_ATTESTER, DOMAIN_BEACON_PROPOSER, DOMAIN_RANDAO, - DOMAIN_SELECTION_PROOF, + DOMAIN_AGGREGATE_AND_PROOF, DOMAIN_BEACON_ATTESTER, DOMAIN_BEACON_BUILDER, + DOMAIN_BEACON_PROPOSER, DOMAIN_PTC_ATTESTER, DOMAIN_RANDAO, DOMAIN_SELECTION_PROOF, +}; +use ethlambda_types::beacon::containers::gloas::{ + ExecutionPayloadEnvelope, PayloadAttestationData, }; use ethlambda_types::beacon::containers::shared::AttestationData; use ethlambda_types::beacon::primitives::{ @@ -124,6 +127,28 @@ impl SigningContext { compute_signing_root(root, domain) } + /// The root an execution payload envelope's signature is computed over. + /// + /// `slot` is the slot of the block the envelope reveals the payload for, + /// not anything read out of the envelope: the domain is the builder's, at + /// the epoch containing that slot (`get_domain(state, DOMAIN_BEACON_BUILDER, + /// compute_epoch_at_slot(state.slot))` in the specification's + /// `process_execution_payload`, where the state is the block's own). + pub fn envelope_signing_root(&self, envelope: &ExecutionPayloadEnvelope, slot: Slot) -> Root { + let domain = self.domain(DOMAIN_BEACON_BUILDER, compute_epoch_at_slot(slot)); + compute_signing_root(envelope.hash_tree_root(), domain) + } + + /// The root a payload timeliness committee member's vote is computed over. + /// + /// The domain is taken at the epoch of `data.slot`, and the message is the + /// data alone: the validator index travels beside the signature in the + /// message wrapper and is deliberately not signed over. + pub fn payload_attestation_signing_root(&self, data: &PayloadAttestationData) -> Root { + let domain = self.domain(DOMAIN_PTC_ATTESTER, compute_epoch_at_slot(data.slot)); + compute_signing_root(data.hash_tree_root(), domain) + } + /// Sign an already-computed signing root on behalf of `pubkey`. /// /// Every public signing method funnels through here, so there is one place @@ -211,6 +236,38 @@ impl SigningContext { ) } + /// Sign the envelope revealing the payload of the block proposed for `slot` + /// on behalf of `pubkey`, which for a self-built payload is the proposer's + /// own key. + /// + /// Not slashable and not guarded. The specification attaches no slashing + /// condition to an envelope: signing two for one slot is at worst a + /// payload the network ignores one of. The block it belongs to is the + /// guarded signature. + pub fn sign_execution_payload_envelope( + &self, + store: &ValidatorStore, + pubkey: &BlsPubkey, + envelope: &ExecutionPayloadEnvelope, + slot: Slot, + ) -> Result { + self.sign_root(store, pubkey, self.envelope_signing_root(envelope, slot)) + } + + /// Sign a payload timeliness committee vote on behalf of `pubkey`. + /// + /// Not slashable by the specification, and deduplicated per validator and + /// slot by [`crate::payload_attestation`] only to avoid publishing the same + /// vote twice. + pub fn sign_payload_attestation( + &self, + store: &ValidatorStore, + pubkey: &BlsPubkey, + data: &PayloadAttestationData, + ) -> Result { + self.sign_root(store, pubkey, self.payload_attestation_signing_root(data)) + } + /// Sign the block whose root is `block_root`, proposed for `slot`, on /// behalf of `pubkey`. /// diff --git a/crates/validator/src/slot_clock.rs b/crates/validator/src/slot_clock.rs index dc5bf6ad..e402b80b 100644 --- a/crates/validator/src/slot_clock.rs +++ b/crates/validator/src/slot_clock.rs @@ -7,6 +7,9 @@ use std::time::{Duration, SystemTime, UNIX_EPOCH}; +use ethlambda_types::beacon::config::Config; +use ethlambda_types::beacon::constants::FAR_FUTURE_EPOCH; +use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::preset::SLOTS_PER_EPOCH; use ethlambda_types::beacon::primitives::{Epoch, Slot}; @@ -29,12 +32,19 @@ const BASIS_POINTS: u64 = 10_000; /// both offsets to 2500 and 5000 basis points, which no fixed fraction /// expresses. Holding the basis points means this clock follows the network it /// was told about rather than the one it was compiled for. -#[derive(Debug, Clone, Copy)] +/// +/// # The offsets depend on the fork of the slot +/// +/// Gloas moves the attester and aggregator offsets earlier (2500 and 5000 +/// basis points) and adds a payload timeliness committee deadline. So the clock +/// holds the whole [`Config`] and answers by the fork of the slot's own epoch +/// (`Config::fork_at_epoch`), which keeps a client running across the boundary +/// on the right offsets on both sides of it. +#[derive(Debug, Clone)] pub struct SlotClock { genesis_time: u64, slot_duration_ms: u64, - attestation_due_bps: u64, - aggregate_due_bps: u64, + config: Config, } impl SlotClock { @@ -51,15 +61,33 @@ impl SlotClock { attestation_due_bps: u64, aggregate_due_bps: u64, ) -> Self { - assert!(slot_duration_ms > 0, "slot_duration_ms must be nonzero"); + let mut config = Config::mainnet().with_fork_epoch(ForkName::Gloas, FAR_FUTURE_EPOCH); + config.slot_duration_ms = slot_duration_ms; + config.attestation_due_bps = attestation_due_bps; + config.aggregate_due_bps = aggregate_due_bps; + Self::from_config(genesis_time, &config) + } + + /// A clock for the network `config` describes, which is how the offsets + /// of every fork reach it. See the type's docs for why it keeps the whole + /// configuration rather than one pair of offsets. + pub fn from_config(genesis_time: u64, config: &Config) -> Self { + assert!( + config.slot_duration_ms > 0, + "slot_duration_ms must be nonzero" + ); Self { genesis_time, - slot_duration_ms, - attestation_due_bps, - aggregate_due_bps, + slot_duration_ms: config.slot_duration_ms, + config: config.clone(), } } + /// Whether `slot` is a gloas slot. + pub fn is_gloas(&self, slot: Slot) -> bool { + self.config.fork_at_epoch(self.epoch_of(slot)) >= ForkName::Gloas + } + /// The slot containing `now`, or `None` before genesis. /// /// Divides in milliseconds, not seconds, so a chain whose slot is not a @@ -94,7 +122,20 @@ impl SlotClock { /// When the attester duty for `slot` should run. pub fn attestation_time(&self, slot: Slot) -> SystemTime { - self.offset_into(slot, self.attestation_due_bps) + let bps = if self.is_gloas(slot) { + self.config.attestation_due_bps_gloas + } else { + self.config.attestation_due_bps + }; + self.offset_into(slot, bps) + } + + /// When the payload timeliness committee's duty for `slot` should run. + /// + /// Only meaningful for a gloas slot; the offset is the configured one + /// whatever the slot, and callers gate on [`Self::is_gloas`]. + pub fn payload_attestation_time(&self, slot: Slot) -> SystemTime { + self.offset_into(slot, self.config.payload_attestation_due_bps) } /// When the aggregation duty for `slot` should run. @@ -103,7 +144,12 @@ impl SlotClock { /// necessarily: an aggregator folds together votes its beacon node has /// collected, and before the attesters have voted there is nothing to fold. pub fn aggregation_time(&self, slot: Slot) -> SystemTime { - self.offset_into(slot, self.aggregate_due_bps) + let bps = if self.is_gloas(slot) { + self.config.aggregate_due_bps_gloas + } else { + self.config.aggregate_due_bps + }; + self.offset_into(slot, bps) } /// `bps` basis points of the way into `slot`. @@ -170,6 +216,17 @@ impl SlotClock { .unwrap_or(Duration::ZERO) } + /// How long from `now` until the payload timeliness committee's duty for + /// `slot`, or zero once that instant has passed. + /// + /// The PTC counterpart to [`Self::until_aggregation`]: the loop sleeps on + /// it between aggregation and the committee vote. + pub fn until_payload_attestation(&self, slot: Slot, now: SystemTime) -> Duration { + self.payload_attestation_time(slot) + .duration_since(now) + .unwrap_or(Duration::ZERO) + } + /// The next slot to serve, given the last one served, and how long until /// it begins. /// @@ -500,6 +557,52 @@ mod tests { assert_eq!(now + delay, clock.start_of(0)); } + /// Slots before the gloas epoch keep the pre-gloas offsets and slots from + /// it on take the gloas ones, on both sides of the boundary. + #[test] + fn the_offsets_change_at_the_gloas_boundary() { + let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, 2); + let clock = SlotClock::from_config(GENESIS, &config); + let last_fulu = 2 * SLOTS_PER_EPOCH - 1; + let first_gloas = 2 * SLOTS_PER_EPOCH; + let ms = + |t: SystemTime, slot: u64| t.duration_since(clock.start_of(slot)).expect("after start"); + + assert!(!clock.is_gloas(last_fulu)); + assert!(clock.is_gloas(first_gloas)); + assert_eq!( + ms(clock.attestation_time(last_fulu), last_fulu), + Duration::from_millis(3_999) + ); + assert_eq!( + ms(clock.aggregation_time(last_fulu), last_fulu), + Duration::from_millis(8_000) + ); + assert_eq!( + ms(clock.attestation_time(first_gloas), first_gloas), + Duration::from_millis(3_000) + ); + assert_eq!( + ms(clock.aggregation_time(first_gloas), first_gloas), + Duration::from_millis(6_000) + ); + } + + #[test] + fn the_payload_attestation_deadline_is_three_quarters_in() { + let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, 0); + let clock = SlotClock::from_config(GENESIS, &config); + let slot = 5; + assert_eq!( + clock.until_payload_attestation(slot, clock.start_of(slot)), + Duration::from_millis(9_000) + ); + assert_eq!( + clock.until_payload_attestation(slot, clock.start_of(slot) + Duration::from_secs(10)), + Duration::ZERO + ); + } + #[test] fn the_wait_until_the_attester_duty_is_the_rest_of_the_offset() { let clock = clock(); diff --git a/docs/beacon_engine.md b/docs/beacon_engine.md index f26847fa..97cd52ee 100644 --- a/docs/beacon_engine.md +++ b/docs/beacon_engine.md @@ -33,10 +33,16 @@ used from gloas on; the startup handshake warns when the execution client does n Three method families a full client would have are deliberately absent: -- **`engine_getPayload*` and `PayloadAttributesV3`.** This node is a follower and - never proposes, so it never asks an execution client to start building a block. - That also keeps `forkchoiceUpdated` outside the fork-scheduling rules attached - to `payloadAttributes.timestamp`: the second parameter is always `null`. +- **`engine_getPayload*` outside the gloas path.** The follower itself never + asks an execution client to start building a block, so its own + `forkchoiceUpdated` calls carry `null` payload attributes. Block production for + a validator client (`produceBlockV3` for fulu, `produceBlockV4` for gloas) does: + `forkchoiceUpdatedV3` with `PayloadAttributesV3` then `engine_getPayloadV5` + before gloas, and `forkchoiceUpdatedV4` with `PayloadAttributesV4` (V3's fields + plus `slotNumber` and `targetGasLimit`, the parent bid's gas limit) then + `engine_getPayloadV6` (`ExecutionPayloadV4`, with `blockAccessList` and + `slotNumber`) from gloas on. The handshake warns when a gloas network's + execution client does not advertise `engine_getPayloadV6`. - **`engine_getBlobs*`.** There is no blob-pool fetch path here; data columns come from peers over the network. - **`engine_getPayloadBodies*`.** Nothing consumes them. diff --git a/docs/cli.md b/docs/cli.md index a013c07a..358d8a3b 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -284,7 +284,8 @@ implementation's, so none of the [common flags](#common-flags) apply to it. ### What it does today -Attestations, block proposals and attestation aggregation. +Attestations, block proposals and attestation aggregation, and from gloas on +the self-built payload envelope and the payload timeliness committee vote. Each epoch it resolves its validators' indices, fetches their attester duties for this epoch and the next, fetches this epoch's proposer duties, subscribes @@ -327,17 +328,38 @@ computing its `hash_tree_root`, which only the typed container can give; a hand-written JSON mapping of an execution payload would be a large surface on which a single wrong field silently produces a signature over the wrong block. +From gloas on a proposal is two messages. The client asks for the block with +`include_payload=true` (`produceBlockV4`), publishes the block bare, then signs +the envelope with the proposer's own key under the builder domain and publishes +it with its blobs and cell proofs (`Eth-Blob-Data-Included: true`). Both go out as +SSZ. The envelope shares the proposal's budget (the attester offset, below), so +a proposal that overruns it is abandoned wherever it is. A failure to publish +the envelope is logged at error and counted, and does not fail the proposal. A +block that commits to a builder's bid is left alone. + +Each epoch it also fetches the payload timeliness committee duties for the +validators it holds, for this epoch and the next when they are gloas ones. A member sleeps until `PAYLOAD_ATTESTATION_DUE_BPS` of the +slot, asks the node for the +payload attestation data, signs it under the PTC domain and submits it. A `204` +(the node knows no block for the slot) means no vote. So a gloas slot runs in +this order: propose at the boundary, attest at `ATTESTATION_DUE_BPS_GLOAS`, +aggregate at `AGGREGATE_DUE_BPS_GLOAS`, then the committee vote. The attestation +and the committee vote are each bounded by the end of the slot. + Not implemented: the builder flow and blinded blocks (the client asks for an unblinded block and refuses a blinded one), sync-committee duties, voluntary exits, doppelganger protection and remote signing. ### Duty offsets come from the network -The slot length and the two duty offsets are read from the beacon node's +The slot length and the duty offsets are read from the beacon node's `/eth/v1/config/spec`, not divided out of a compiled-in constant. The specification states them as `SLOT_DURATION_MS` plus basis points of it, `ATTESTATION_DUE_BPS` and `AGGREGATE_DUE_BPS`, which on mainnet's 12-second slot -work out at 3999 ms and 8000 ms. +work out at 3999 ms and 8000 ms. Gloas moves both earlier +(`ATTESTATION_DUE_BPS_GLOAS`, `AGGREGATE_DUE_BPS_GLOAS`) and adds +`PAYLOAD_ATTESTATION_DUE_BPS` (and `PAYLOAD_DUE_BPS`, which the node uses), so +the clock picks the pair by the fork of each slot's own epoch. `SECONDS_PER_SLOT` no longer exists in the specification and is accepted only as a fallback, since deployed nodes still send it. A node sending both is required diff --git a/docs/metrics.md b/docs/metrics.md index 64644d7c..cbdc72d6 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -607,6 +607,10 @@ and absent is not the same as zero. | `ethlambda_validator_blocks_broadcast_not_imported_total` | Counter | Blocks a node broadcast but could not import into its own database, which is what a 202 means. A subset of `blocks_proposed_total`, not a failure: the block reached the network. Points at that node's execution layer | | `ethlambda_validator_blocks_refused_total` | Counter | Blocks deliberately not signed, because this process had already proposed that slot for that validator. Should normally read zero | | `ethlambda_validator_block_proposal_failures_total` | Counter | Proposal duties that did not end in a published block, including ones abandoned for overrunning | +| `ethlambda_validator_envelopes_published_total` | Counter | Execution payload envelopes a beacon node accepted for blocks this client proposed (gloas) | +| `ethlambda_validator_envelope_failures_total` | Counter | Proposed gloas blocks whose self-built envelope was not published. The block is out and the proposal counts as made, but the slot's payload is withheld, so this is the failure that costs the payload. Should read zero | +| `ethlambda_validator_payload_attestations_published_total` | Counter | Payload timeliness committee votes a beacon node accepted (gloas) | +| `ethlambda_validator_payload_attestation_failures_total` | Counter | Slots whose payload timeliness committee duty did not result in published votes | | `ethlambda_validator_aggregates_published_total` | Counter | Aggregates accepted by a beacon node. Bursty rather than steady: a validator is selected a few times a day, so hours at zero are normal for a small deployment | | `ethlambda_validator_aggregation_failures_total` | Counter | Aggregation duties that ended in no published aggregate, including ones abandoned for overrunning the slot | | `ethlambda_validator_fee_recipient_mismatches_total` | Counter | Blocks paying execution rewards to an address this client did not request. Should read zero forever; a non-zero value means every proposal is paying somewhere else | @@ -632,11 +636,20 @@ a failure. `blocks_proposed_total` counts blocks a node accepted; but that the node answering could not import, which is a beacon-node fault rather than a validator one and is why it is not folded into either. +Gloas adds two pairs. A gloas proposal ends in a block and then an envelope, +and the envelope's failure does not fail the proposal: `blocks_proposed_total` +counts the block, and `envelope_failures_total` the reveal that did not follow. +The committee vote has the same split as the attestation's: `payload_attestations_published_total` +counts votes accepted and `payload_attestation_failures_total` the slots whose +duty produced none. A slot where the node knows no block for the committee to +vote on is neither, since there was nothing to sign. + Two series should read zero for the life of a healthy deployment, and are the ones worth alerting on at any non-zero value rather than on a rate: `attestations_refused_total` and `blocks_refused_total` mean this client's own guards caught a duty they judged unsafe, and `fee_recipient_mismatches_total` -means blocks are being proposed that pay someone else. +means blocks are being proposed that pay someone else. `envelope_failures_total` +belongs with them on a gloas network. ## Troubleshooting diff --git a/docs/rpc.md b/docs/rpc.md index 212bd87e..3c335723 100644 --- a/docs/rpc.md +++ b/docs/rpc.md @@ -219,7 +219,7 @@ curl -X POST http://127.0.0.1:5052/lean/v0/admin/aggregator \ ## Beacon API Server (`:5052`, on `ethlambda beacon`) The subset of the [Ethereum Beacon API](https://ethereum.github.io/beacon-APIs/) -that this follower can answer from its own store. It **replaces** the `/lean/v0` +that this node can answer from its own store. It **replaces** the `/lean/v0` surface rather than sitting beside it; a `/lean/v0` path on a beacon node is a `404`. @@ -240,13 +240,20 @@ surface rather than sitting beside it; a `/lean/v0` path on a beacon node is a | `GET`, `POST` | `/eth/v1/beacon/states/{state_id}/validators` | JSON | Registry entries by index or pubkey, with status | | `GET` | `/eth/v1/validator/duties/proposer/{epoch}` | JSON | Proposers for the head's epoch or the next | | `POST` | `/eth/v1/validator/duties/attester/{epoch}` | JSON | Committee assignments for the given indices | +| `POST` | `/eth/v1/validator/duties/ptc/{epoch}` | JSON | Payload timeliness committee seats for the given indices (gloas) | | `GET` | `/eth/v1/validator/attestation_data` | JSON | What to attest to at `slot` | +| `GET` | `/eth/v1/validator/payload_attestation_data` | JSON or SSZ | What a committee member signs for `slot` (gloas) | | `POST` | `/eth/v2/beacon/pool/attestations` | *(status only)* | Validate and gossip `SingleAttestation`s | +| `POST` | `/eth/v1/beacon/pool/payload_attestations` | *(status only)* | Validate, pool and gossip `PayloadAttestationMessage`s (gloas) | +| `GET` | `/eth/v1/beacon/pool/payload_attestations` | JSON | The pool's votes as aggregated `PayloadAttestation`s (gloas) | | `POST` | `/eth/v1/validator/beacon_committee_subscriptions` | *(status only)* | Aggregators' entries join their committee's subnet | | `GET` | `/eth/v2/validator/aggregate_attestation` | JSON | The pooled votes for a data root and committee, aggregated | | `POST` | `/eth/v2/validator/aggregate_and_proofs` | *(status only)* | Validate and gossip `SignedAggregateAndProof`s | -| `GET` | `/eth/v3/validator/blocks/{slot}` | SSZ or JSON | An unsigned block built on the head (`produceBlockV3`) | -| `POST` | `/eth/v2/beacon/blocks` | *(status only)* | Gossip and import a signed block (`publishBlockV2`, SSZ) | +| `GET` | `/eth/v3/validator/blocks/{slot}` | SSZ or JSON | An unsigned fulu block built on the head (`produceBlockV3`) | +| `POST` | `/eth/v4/validator/blocks/{slot}` | SSZ or JSON | An unsigned self-built gloas block, with its envelope and blobs when asked (`produceBlockV4`) | +| `GET` | `/eth/v1/validator/execution_payload_envelopes/{slot}/{beacon_block_root}` | SSZ or JSON | The unsigned envelope `produceBlockV4` built (gloas) | +| `POST` | `/eth/v2/beacon/blocks` | *(status only)* | Gossip and import a signed fulu or gloas block (`publishBlockV2`, SSZ) | +| `POST` | `/eth/v1/beacon/execution_payload_envelopes` | *(status only)* | Gossip a signed envelope and its data columns (gloas, SSZ) | | `POST` | `/eth/v1/validator/prepare_beacon_proposer` | *(status only)* | Acknowledged, not acted on (see below) | ### Validator endpoints @@ -260,17 +267,23 @@ the chain actor writes, so no request waits on the actor. next; attester duties cover the head's previous, current and next epoch, which is as far as its shuffling is already fixed. Anything else is a `400`. `dependent_root` follows each endpoint's v1 definition. Attester duties walk - every committee of the epoch, a full shuffle per request on mainnet. An - epoch the node's schedule places at gloas is a `400` on both duty endpoints: - the head is a fulu state, and the node serves no validator duties from gloas - (it follows the fork but has no builder, payload-timeliness or gloas proposer - support). + every committee of the epoch, a full shuffle per request on mainnet. Gloas + epochs are served like fulu ones: gloas keeps fulu's `proposer_lookahead` and + `upgrade_to_gloas` carries it over, so a fulu head already answers proposer + duties for the first gloas epoch. - **`attestation_data`** follows phase0's `validator.md`: the head block, the epoch's boundary block as target, and as source the current justified checkpoint of the head state advanced to the slot's epoch (through fork choice's cached `checkpoint_state`, and only when the head is in an earlier - epoch). A slot before the head, or past the wall clock, is a `400`, as is a - slot in a gloas epoch: the answer would be a fulu-shaped vote under gloas. + epoch). A slot before the head, or past the wall clock, is a `400`. + `committee_index` is optional and ignored: from electra on every committee of + a slot attests to the same data. `data.index` is `0` before gloas. At a gloas + slot it is the payload-present flag: `0` when the head block is from the + requested slot itself (its payload cannot be revealed yet), otherwise `1` + exactly when fork choice holds the head block's payload as FULL + (`Store::head_payload_status`, or one fresh `get_head_node` walk when none is + recorded for this head). A pre-gloas head counts as FULL (see + [Spec Deviations](./spec_deviations.md#a-pre-gloas-head-counts-as-full-in-attestation_dataindex)). - **`pool/attestations`** checks each attestation against the electra `beacon_attestation_{subnet_id}` gossip conditions it can evaluate (clock window, `data.index == 0`, target epoch, the voted block known and the target @@ -279,9 +292,15 @@ the chain actor writes, so no request waits on the actor. back in an `IndexedErrorMessage` with its position; the valid ones in the same batch are still published. There is no seen-attestation cache. Each accepted attestation also goes into the node's **attestation pool**, since - gossip never delivers a node its own messages. `Eth-Consensus-Version: - gloas` is refused with a `400` here and on `aggregate_and_proofs`, as is a - gloas slot on the aggregate query below: the pool holds electra-shaped votes. + gossip never delivers a node its own messages. `Eth-Consensus-Version` must + name electra, fulu or gloas here and on `aggregate_and_proofs`, and must match + the fork of each item's slot: a `gloas` header on a pre-gloas slot, or an older + one on a gloas slot, fails that item (not the batch) with + `Eth-Consensus-Version does not match the attestation slot's fork`. At a + gloas slot `data.index` may be `0` or `1`, checked with gloas's payload-status + gossip rule (a vote in the voted block's own slot must be `0`; `1` needs the + node to have verified the payload, otherwise the item fails). A gloas + `SingleAttestation` is electra's own container. - **`beacon_committee_subscriptions`**: each aggregator's entry makes the node join its committee's attestation subnet until the end of that slot, so the committee's votes from other validators reach the pool too: every @@ -290,18 +309,21 @@ the chain actor writes, so no request waits on the actor. subnets are left once their slot has passed, and never appear in `attnets`. - **`aggregate_attestation`** answers from the pool: every vote held for the data root and committee, as electra's `Attestation` with the BLS aggregate of - their signatures. `404` when nothing is held. + their signatures; at a gloas slot the answer is `{version: "gloas", data}` + with gloas's `Attestation` (the same JSON shape). JSON only. `404` when + nothing is held. - **`aggregate_and_proofs`** checks each aggregate with the same `beacon_aggregate_and_proof` gossip conditions this node applies to its peers' aggregates (`gossip::aggregate`), signatures included, against a fresh seen-cache (a node never receives its own messages, so P2P's says nothing about them). What passes is gossiped on the topic and goes into the - pool. + pool. The `Eth-Consensus-Version` header picks the decoder: `gloas` takes + gloas's `SignedAggregateAndProof`, the others electra's. - **The attestation pool** holds, the best-covered per data root and committee: votes from `pool/attestations` and the aggregator subnets, aggregates from `aggregate_and_proofs`, and every electra gossip aggregate - P2P accepts, once all three of its signatures have verified. Gossip - aggregates are pooled on arrival, so a slot's aggregates are there when the + P2P accepts (gloas aggregates included, in gloas's container), once all three + of its signatures have verified. Gossip aggregates are pooled on arrival, so a slot's aggregates are there when the next slot's block is asked for. Entries more than an epoch old are dropped once a slot. - **`prepare_beacon_proposer`** records each validator's fee recipient, in @@ -320,18 +342,138 @@ the chain actor writes, so no request waits on the actor. `Eth-Execution-Payload-Blinded: false`; there is no builder flow. It is a **`503`** without a configured execution client, or when the payload carries blobs. A slot the schedule does not place at fulu (gloas included) is a - `400`, checked before the execution client. -- **`POST beacon/blocks`** takes SSZ `SignedBlockContents`, checks the block - is after the head and its proposer signature, then gossips it on - `beacon_block` and hands it to the chain actor to import. A slot the - schedule does not place at fulu (gloas included) is a `400`, checked before - the head state is advanced. - -**Blobs are not supported yet.** Publishing a blob-carrying block means -computing and gossiping its data column sidecars, which this node does not do, -and peers will not import a block they cannot sample. Such payloads are refused -at production (`503`, which a validator client fails over on) and such blocks -at publication (`400`). + `400`, checked before the execution client; `produceBlockV3` is the fulu + endpoint, and a gloas slot goes to `produceBlockV4`. +- **`POST beacon/blocks`** takes SSZ and the `Eth-Consensus-Version` header + (`fulu` or `gloas`, else `400`; a non-SSZ content type is a `415`). Fulu: a + `SignedBlockContents`; the node checks the block is after the head and its + proposer signature, then gossips it on `beacon_block` and hands it to the + chain actor to import. A slot the schedule does not place at fulu is a `400`, + checked before the head state is advanced. Gloas: a bare + `gloas::SignedBeaconBlock` (a gloas block carries no payload or blobs; they + follow in the envelope). The slot must be scheduled at gloas, the parent must + be held, the slot must be after the parent's, and the proposer and signature + are checked against the parent's state advanced to the slot (all `400` + otherwise), before the block is gossiped and handed to the chain actor. + +**Fulu blobs are not supported.** Publishing a blob-carrying fulu block means +computing and gossiping its data column sidecars, which this node does not do +for fulu, and peers will not import a block they cannot sample. Such payloads +are refused at production (`503`, which a validator client fails over on) and +such blocks at publication (`400`). Gloas blobs are supported: the envelope +publication below builds and gossips the columns. + +### Gloas block production + +A gloas proposer signs two things on their own, the block and the envelope that +reveals its payload, and this node serves both halves. It only builds for +itself: no builder bid is taken from gossip or a builder API, and no +`SignedProposerPreferences` is read (see +[Spec Deviations](./spec_deviations.md#self-build-only)). + +**`POST /eth/v4/validator/blocks/{slot}`** (`produceBlockV4`): + +| | | +|---|---| +| Query | `randao_reveal` (required), `include_payload` (required, `true` or `false`), `graffiti` (optional, 32-byte hex), `skip_randao_verification` (accepted, ignored) | +| Request headers | `Eth-Consensus-Version` is optional but must be `gloas` when present. `Accept: application/octet-stream` for SSZ, JSON otherwise | +| Body | A `BuilderConfig` (`min_bid`, `builder_boost_factor`, `builders`), JSON, or SSZ with `Content-Type: application/octet-stream`. It is decoded and otherwise ignored; a missing or undecodable one is a `400` | +| `200` SSZ | With `include_payload=true`, `BlockContents` (`block`, `execution_payload_envelope`, `kzg_proofs`, `blobs`); with `false`, the bare `gloas::BeaconBlock` | +| `200` JSON | `{version: "gloas", consensus_block_value, execution_payload_value, execution_payload_included, data}`, where `data` is the same container as the SSZ body | +| Response headers | `Eth-Consensus-Version: gloas`, `Eth-Execution-Payload-Included` (`true` or `false`), `Eth-Execution-Payload-Value` (wei, decimal), `Eth-Consensus-Block-Value` (always `0`: no builder comparison happens on this node) | +| `400` | A slot the schedule does not place at gloas, a missing or malformed query, a wrong `Eth-Consensus-Version`, a bad body, a slot not after the head block, or a `randao_reveal` that does not verify against the slot's proposer. The slot check precedes the execution-client check, so it is a `400` on any node | +| `503` | No execution client configured, the execution client did not start or return a build, or the node is building on a FULL parent whose envelope it does not hold | + +The node advances the head state to the slot, and decides which parent payload +to build on with `should_build_on_full` over the payload status fork choice +recorded for the head. It then asks its execution client to build +(`forkchoiceUpdatedV4` with `PayloadAttributesV4`, then `getPayloadV6`) with the +proposer's `prepare_beacon_proposer` fee recipient. The body packs the +attestation pool's best aggregates and the payload attestation pool's votes for +the parent block (an aggregate that does not verify against the advanced state +is dropped, since one bad operation fails the block), and the state root comes +from running the block through `process_block`. The bid is a zero-value +self-build bid read off the built payload. What was built is cached by `(slot, +block root)`, for the current and previous slot only. + +**`GET /eth/v1/validator/execution_payload_envelopes/{slot}/{beacon_block_root}`** +serves the cached unsigned envelope, for a client that asked for the block with +`include_payload=false`: `{version: "gloas", data}` as JSON, or the SSZ +`ExecutionPayloadEnvelope` on `Accept: application/octet-stream`, with +`Eth-Consensus-Version: gloas`. `404` when nothing is cached for the pair +(unknown, or older than the previous slot); `400` for an unparseable slot or +root. + +**`POST /eth/v1/beacon/execution_payload_envelopes`** takes the signed envelope, +SSZ only (`415` otherwise), with `Eth-Consensus-Version: gloas` and +`Eth-Blob-Data-Included` (both required, else `400`): + +- `true`: the body is `SignedExecutionPayloadEnvelopeContents` (the signed + envelope, `kzg_proofs`, `blobs`). +- `false`: the body is the bare signed envelope, and the blobs and proofs come + from the production cache. `400` when none is cached and the block's bid + commits to blobs. + +The envelope's block is waited for, polling every 50 ms for up to 4 seconds, +since a validator client publishes the envelope right after the block returns, +which is before the chain actor has imported it; a block still unknown after +that is a `400` (`unknown block`). The envelope must fulfill the block's bid +(builder index and block hash), the blobs must match the bid's commitments, the +signature must verify under `DOMAIN_BEACON_BUILDER` against the block's +post-state, and every data column built from the blobs must verify against the +commitments and cell proofs; any failure is a `400`. A success is `200` once the +envelope and all `NUMBER_OF_COLUMNS` gloas data column sidecars are handed to +P2P, which gossips them together. A node that does not subscribe to a column's +subnet publishes through gossipsub fanout. + +### Payload timeliness committee + +- **`POST /eth/v1/validator/duties/ptc/{epoch}`** takes a JSON array of quoted + validator indices (an empty array or a bad index is a `400`) and answers + `{dependent_root, execution_optimistic, data: [{pubkey, validator_index, + slot}]}`: one duty per validator, the first slot of the epoch whose committee + seats it. An index with no seat, or unknown, gets none. The epoch may be at + most one past the later of the head state's and the wall clock's epoch (else + `400`). An epoch before the gloas fork answers `200` with an empty `data`, so + a client can ask every epoch. A gloas head state answers from its + `ptc_window`; an epoch the head has not caught up to, or the first gloas epoch + under a fulu head, is answered from a copy of the head state advanced to the + epoch's first slot, crossing the gloas upgrade where it falls, on a blocking + thread. A gloas epoch more than one before the head state's is a `400`. + `dependent_root` is defined as for attester duties. There is no `503`. +- **`GET /eth/v1/validator/payload_attestation_data?slot=`** answers + `{version: "gloas", data: {beacon_block_root, slot, payload_present, + blob_data_available}}` as JSON, or the SSZ `PayloadAttestationData` on + `Accept: application/octet-stream`, with `Eth-Consensus-Version: gloas`. The + data is for the block of that very slot on the head's chain; a slot with no + such block (or whose block is not a gloas block) answers `204` with no body, + which a client reads as "cast no vote". A slot before the gloas fork is a + `400`, and a node that is syncing answers `503`. `payload_present` is whether + the block's envelope reached this node before the slot's start plus + `PAYLOAD_DUE_BPS` of it, judged by arrival time (see + [Spec Deviations](./spec_deviations.md#payload_present-is-judged-by-the-envelopes-arrival-time)). + `blob_data_available` is true when the bid commits to no blobs, the payload is + verified, or every column this node custodies for the block is stored. +- **`POST /eth/v1/beacon/pool/payload_attestations`** takes a JSON array of + `PayloadAttestationMessage`s; `Eth-Consensus-Version` is optional and must be + `gloas` when present. Each message passes the checks gossip applies to a + peer's (the current slot, a known block at the slot, committee membership, + signature) against a fresh seen-cache. A valid one is pooled and gossiped; one + already pooled is neither pooled nor gossiped again and still counts as + accepted. A rejected one comes back in an `IndexedErrorMessage` (`400`) as + `{outcome}: {reason}` with its position, and the rest still go out; a + head-state cache miss is a per-item `ignore: state_unavailable`. +- **`GET /eth/v1/beacon/pool/payload_attestations?slot=`** answers `{version: + "gloas", data}` with the pool's votes as the specification's + `PayloadAttestation`s, those of `slot` only when given: one aggregate per slot + and `PayloadAttestationData`, a bit per seat of that slot's committee + (`get_ptc` on the head state) and the seats' signatures aggregated, the way + block production packs them. The pool itself keeps the individual messages. + JSON only; a vote whose slot falls outside the head state's committee window + is left out. + +The payload attestation pool is shared with P2P, which fills it from accepted +gossip, so a block this node builds carries the votes its peers sent too. `tooling/kurtosis-validator/network_params_ethlambda_beacon.yaml` points the validator client at this node alone, so every block on that devnet is one this diff --git a/docs/spec_deviations.md b/docs/spec_deviations.md index fd194d45..f3e00b67 100644 --- a/docs/spec_deviations.md +++ b/docs/spec_deviations.md @@ -64,6 +64,17 @@ same keys, already signed. close the shapes reachable through ordinary operation: a backward wall-clock step, or a duty schedule replaced mid-epoch. Both are held in memory only, so a restart empties them and neither knows anything about any other process. +- **Gloas messages:** a gloas proposer also signs an execution payload + envelope, and a payload timeliness committee member signs a payload + attestation. Neither is slashable, so neither has a guard. An envelope is + published once per proposal, right after its block, and a failure to publish it + is logged at error and counted without failing the proposal. A PTC vote has an + in-memory dedup of `(validator, slot)` pairs recorded at signing time + (`PayloadAttestationService`), so a duty abandoned between signing and + submission is lost rather than re-signed. It is emptied by a restart and says + nothing about another process, so it is not slashing protection either: a + second, different vote for one slot is equivocation that peers penalize in + gossip, not a slashing. - **Why a proposer slashing is the worse of the two:** a double vote needs a second validator's attestation to be caught. Two signed block headers for one slot are the whole of the evidence on their own. @@ -82,6 +93,91 @@ same keys, already signed. runs, and treat a restart as an event that needs the same care a manual key move would. The client warns about this at startup on every run. +## Self-build only + +The gloas validator duties are served for a proposer that builds its own +payload, and for no one else. + +- **The specification:** a gloas proposer may take a builder's signed bid + (`SignedExecutionPayloadBid`, from gossip or a builder API) instead of building + its own payload, and publishes a `SignedProposerPreferences` so builders know + its fee recipient and gas limit target. `produceBlockV4` takes a + `BuilderConfig` (`min_bid`, `builder_boost_factor`, `builders`) to steer that + choice. +- **ethlambda:** the beacon node never takes a bid, from gossip or otherwise, and + never builds or reads a `SignedProposerPreferences`. `produceBlockV4` decodes + the `BuilderConfig` body (a missing or undecodable one is a `400`, as the + specification requires) and ignores it, logging at debug when it names + builders. Every block commits to a zero-value self-build bid + (`BUILDER_INDEX_SELF_BUILD`, the G2 point at infinity as signature), and + `Eth-Consensus-Block-Value` is always `0` since there is nothing to compare. + The validator client signs the envelope with the proposer's own key under + `DOMAIN_BEACON_BUILDER`, and leaves a block that commits to anyone else's bid + alone. +- **Why:** a scope decision for the first gloas duties: taking bids needs a bid + pool, builder payment handling and a builder-facing API, none of which this + node has. +- **Consequence:** a validator run through this node never earns a builder's + payment, and an execution client's own block value is what + `Eth-Execution-Payload-Value` reports. + +## `target_gas_limit` is the parent bid's gas limit + +- **The specification:** the payload is built toward the `target_gas_limit` of + the proposer's `SignedProposerPreferences`, and `bid.gas_limit` must be + compatible with it (`is_gas_limit_target_compatible`). +- **ethlambda:** with no preferences to read, `PayloadAttributesV4.targetGasLimit` + is the `gas_limit` of `latest_execution_payload_bid` of the state being built + on (`gloas_payload_inputs`), so the execution client holds the gas limit where + it is. +- **Equivalence:** the bid is built from the payload the execution client + returns, so `bid.gas_limit` is whatever that payload carries and is always + consistent with the block. A validator that wants the limit to move cannot say + so through this node: it follows the previous block's. + +## `skip_randao_verification` is ignored + +- **The specification:** `produceBlockV4`'s `skip_randao_verification` lets a + caller skip the node's check of `randao_reveal`; when it is set, the reveal + must be the point at infinity. +- **ethlambda:** the parameter is accepted and not honored. The block's state + root comes from running it through the state transition, which verifies the + reveal, so the endpoint checks the reveal against the slot's proposer up front + and answers `400` (`invalid randao_reveal`) for one that does not verify, + infinity included. +- **Consequence:** a caller cannot get a block without a real reveal. Every + caller this client knows of (including its own validator client) sends one. + +## A pre-gloas head counts as FULL in `attestation_data.index` + +- **The specification:** at a gloas slot `data.index` is `0` when the attested + block is from the attestation's own slot, and otherwise `0` for an `EMPTY` and + `1` for a `FULL` payload status in the validator's fork choice. It does not say + which a pre-gloas block is. +- **ethlambda:** at a gloas slot `data.index` is `0` when the head block is from + the requested slot, `1` when fork choice holds the head's payload as FULL + (`Store::head_payload_status`, or one `get_head_node` walk when none is + recorded), and `1` for a pre-gloas head, the same boundary rule + [fork choice applies](#the-fulu-to-gloas-fork-choice-boundary). So the first + gloas slots vote for the last fulu block's payload. +- **Why:** the vote should name the node fork choice itself walks to, and + that rule makes a pre-gloas block's one node FULL. + +## `payload_present` is judged by the envelope's arrival time + +- **The specification:** a PTC member sets `payload_present` when it has + received the block's envelope by `get_payload_due_ms()` into the slot. +- **ethlambda:** `GET payload_attestation_data` compares the time the envelope + reached the node (`Store::beacon_envelope_seen_ms`, recorded by the chain + actor from the arrival time of the envelope, not the time it finished + verifying; the earliest arrival wins) with the slot's start plus + `get_payload_due_ms()`. An envelope held for its block or columns is + therefore judged by when it came, as the rule intends. The record lives in + memory only, pruned with the execution block hash cache. +- **Consequence:** a restart forgets it. Only the slot in flight can be wrong + (a member asked after a restart answers `payload_present = false` for an + envelope that arrived before it), and the next slot's record starts fresh. + ## `/eth/v1/node/identity` reports no ENR The endpoint's `enr`, `p2p_addresses` and `discovery_addresses` are empty