refactor(eth2api): hand-write the beacon node client by emlautarom1-agent[bot] · Pull Request #695 · NethermindEth/pluto

emlautarom1-agent · 2026-09-09T01:00:13Z

Closes #611

Summary

pluto-eth2api ships a hand-written beacon node client for the endpoints Pluto calls. The oas3-gen build step, its CI and nix pins and the generated types are removed. Responses decode straight into the spec::* and v1 domain types, every failure is a variant of EthBeaconNodeApiClientError, and an HTTP failure is an HttpError carrying status, method, endpoint path and the decoded error body. core and app match on those variants; string matching on error text and anyhow are out of eth2api, core, app and testutil.

The client is Arc-shared and owns its chain config cache (spec, genesis, fork schedule), which replaces the process-global cache keyed by URL. Request metrics are recorded inside the client methods, so a consumer cannot forget to instrument a call.

A tests/integration target runs the client against an isolated Lighthouse container on mainnet genesis: 55 tests over every endpoint method, about seven seconds after the container starts, on every CI run through --all-features.

Differences from Charon

Every non-2xx response is an Http error. Only get_block_v2 maps 404 to None, which is the one lookup Charon checks for a missing block; the fetcher maps 404 from the aggregate attestation and sync contribution endpoints to its own not-found variants.
The validator API answers with the upstream's own status for 400 and 503 on the duty endpoints and 400 and 404 on validators, 502 for any other upstream failure and 504 for an upstream timeout. Charon answers 500 for every upstream failure.
PriorAttestationKnown is recognised on the decoded error body message and on per-item failures. Charon substring-matches the whole error string.
Metrics keep the twenty-two labels already in use. Methods without a label (block root, block header, selections, subscriptions, proposal preparations) are unrecorded until the labels are assigned in a follow-up.

Out of scope

Multi-endpoint eth2wrap (first-success over clients and fallbacks, app_eth2_using_fallback), tracked in Implement app/eth2wrap #60 and Close out the deferred node wiring #610. TODO(#402 part B) marks the single-endpoint reduction.
SSZ negotiation for block production, block retrieval and the publish endpoints.
A devnet with an execution client, needed for the success paths that require signing keys, a builder or an Altair state.
Golden tests over recorded beacon node responses.

… build step `client.rs` implements the 29 Beacon API endpoints Pluto calls directly on reqwest, including the status-to-variant response mapping, and is fully linted and documented. `types.rs` is checked in and holds only the request and response types reachable from the workspace, plus the builders callers use, each item verbatim from the OpenAPI generator output. Removes the build script, the OpenAPI spec file and the oas3-gen installs from CI, the nix shell and CONTRIBUTING.md. `bon`, `validator`, `regex` and `oas3-gen-support` remain for the kept type definitions. Part of #611.

`types.rs` holds hand-written client-level types: request options, response envelopes that carry metadata next to `data`, the `HttpError`/`ErrorBody` pair for non-2xx statuses, and string enums. Entities live in `v1` (`AttesterDuty`, `ProposerDuty`, `SyncCommitteeDuty`, `Validator`, `ValidatorStatus`, `SyncState`, `PeerCount`, `Genesis`, `ProposalPreparation`, `SyncCommitteeSubscription`, `BeaconBlockHeader`), `spec::phase0` gains `Fork` and `Validator`, and `versioned` gains `ProposalBlock`, `VersionedProposal` and `SignedBeaconBlock`. All of them serialize to the JSON the beacon node speaks and the validator API router forwards. Client methods take typed parameters and return the decoded payload or envelope; a non-2xx status is an `Err` wrapping a downcastable `HttpError`. Consumers in core, app and testutil read typed fields directly: the response enums, the `impl_beacon_response!` macro, the `TryFrom` bridges in `spec/` and the JSON round-tripping in `validator_duty.rs`, `core::signeddata`, `validatorapi/component.rs` and the validator mock are removed. The block endpoints request JSON explicitly. Removes `oas3-gen-support`, `validator`, `bon` and `http` from `eth2api` and the `oas3-gen` install from the Dockerfile. Part of #611.

DutyDefinition carries v1::AttesterDuty, v1::ProposerDuty and v1::SyncCommitteeDuty directly, matching Charon's zero-field embedding; the pubkey lives in the duty and in the DutyDefinitionSet key. bcast reads status and activation epoch off v1::Validator without a wrapper type.

Removes resolve_fork_version, fetch_genesis_validators_root and fetch_genesis_fork_version, which had no callers, and inlines the single-use parse_body and filter_active helpers in the test mocks.

Spec is a struct with the slot, fork, aggregator and domain fields Pluto reads, decoded straight from the wire encoding, so the string-keyed lookups and their missing-key errors go away. DOMAIN_APPLICATION_BUILDER defaults to the builder-specs constant when a node omits it, as in go-eth2-client. DomainName::domain_type selects the field for a signing domain; the unused BlobSidecar domain is dropped. Test specs overlay the mock's default spec.

The client error enum gets its own module; BeaconNodeEvent and the domain computations live with the other types; the cached config fetches, the event stream and the proposer-duty filter are client methods.

ProposalBlock serializes to its Beacon API form and both proposal block enums decode from it given the version and blinded flag, so the client, the cluster-wire decoder in core and the validator API router use one fork dispatch table.

The scheduler, broadcaster and recaster take the EthBeaconNodeApiClient and the ValidatorCache directly. The cache is built from the cluster pubkeys before its consumers, so the set-after-construction slot and its lock go away.

A 404 on GET /eth/v2/beacon/blocks/{block_id} means no block was proposed, so the client reports it as Option::None and the inclusion tracker no longer inspects HTTP statuses.

The request-shape table asserts the exact JSON only for the two bodies the client assembles itself; the serde encodings are covered by the type tests.

Delete the reduced attester duty copies in signeddata, validator_duty and tracker::inclusion so the beacon API type is the only one in the workspace. The cluster-wire SSZ and JSON codecs for AttestationData now carry the 48-byte pubkey as charon's attesterDutySSZ does.

`EthBeaconNodeApiClient` returns `EthBeaconNodeApiClientError`, a `thiserror` enum that separates transport failures, non-2xx responses (`HttpError` with status, method, endpoint and body), JSON decode failures naming the offending path, base URL validation and payload validation (`PayloadError`). Callers in `pluto-core`, `pluto-app` and `pluto-testutil` match on these variants directly, and `anyhow` leaves their dependency lists. Callers submit through the client methods directly. `fetch_attester_duties_for_indices`, `fetch_beacon_attester_domain` and `submit_validator_registrations` live next to the other `fetch_*` helpers in `client.rs`, and `DataVersion::is_before_electra` tells the attestation wire shape.

Each `EthBeaconNodeApiClient` method with an endpoint label records its own `app_eth2_*` request, latency and error metrics under that label, covering the request, the response read and the decode. The labels are the twenty-two Charon `eth2wrap` names the workspace reports (`attestation_data`, `attester_duties`, `proposer_duties`, `sync_committee_duties`, `aggregate_attestation`, `sync_committee_contribution`, `validators`, `proposal`, `submit_*`, `spec`, `genesis`, `fork_schedule`, `node_version`, `node_syncing`, `node_peer_count`); methods without a label are unrecorded. Callers in `pluto-core`, `pluto-app` and the `fetch_*` helpers call the client methods directly, and `metrics::instrument` is private to the crate.

The fetcher propagates `EthBeaconNodeApiClientError` from `attestation_data` unchanged and maps only an `Http` 404 from `aggregate_attestation` and `sync_committee_contribution` to `AggregateAttestationNotFound` and `SyncContributionNotFound`; every other client error propagates as `FetcherError::BeaconNode`. `FetcherError::NilAttestationData` and `FetcherError::UnexpectedResponse` have no producer and are removed. The broadcaster tolerates a `PriorAttestationKnown` rejection of an attestation submit by inspecting the `Http` body: the top-level message or any per-item failure message containing that text counts as success. `bcast::Error::Client` carries the `EthBeaconNodeApiClientError` itself as its source; the validator cache sites destructure `ValidatorCacheError` to reach it. Tests cover the two fetcher 404 mappings, a propagated 500 on attestation data, and both body shapes of the tolerated rejection.

`EthBeaconNodeApiClient` is a handle over an `Arc<Inner>` holding the HTTP client, the base URL and a `ChainConfigCache` of `OnceCell`s for the spec, genesis and fork schedule. Clones share the connection pool and the cache; `fetch_spec` and the private genesis and fork-schedule helpers return values cloned out of the cells. A failed fetch leaves its cell empty so the next call retries; a successful response is kept until the process exits. The `client` and `base_url` fields are private, with `base_url()` serving the SSE listener. Consumers hold `EthBeaconNodeApiClient` directly rather than `Arc<EthBeaconNodeApiClient>` (`validatorapi::Component`, `sigagg::new_verifier`, `SlotAttester`). `BeaconMock` and the client tests rely on each test constructing its own client for cache isolation.

The deadline test modules are unit-returning tests that assert with assert!/assert_eq! and unwrap setup and infallible-in-practice operations with expect(..) carrying the original context messages; slot_start_overflows_on_huge_slot matches the DeadlineError::ArithmeticOverflow variant directly, the unreachable duty-type arm in duty_deadline_durations panics, and the module doctest returns pluto_core::deadline::Result so the crate builds and tests without anyhow.

The display shows the response status followed by the body message; per-item batch failures are available structurally through `body.failures`.

Each endpoint method reads its response body with text() and decodes it in place: decode routes the JSON through serde_path_to_error so a decoding failure names the offending field path, data-envelope endpoints take the data field of a decoded Data<T>, and endpoints without a payload return Ok(()) once send succeeds.

`Error::Client` carries the beacon node client error; the endpoint and method recorded inside it identify the failed call.

The validator API identifies each beacon node call it makes on behalf of a VC request with the `Upstream` enum, which the `ApiError` helpers take and render into the client-visible message. Upstream failures carry the client error or the `HttpError` itself as the `ApiError` source, so the debug log retains the typed response. VC-facing status codes and messages are unchanged.

Free functions are called through their module path so the defining module is visible at each call site.

The Lighthouse-backed tests form the `integration` test target of `pluto-eth2api`, gated by `required-features = ["integration"]` so a plain `cargo test` skips it. `cargo test -p pluto-eth2api --all-features` builds the target and runs its four tests against a `sigp/lighthouse` container started through testcontainers.

The container runs with discovery, UPnP and peering disabled, so the node's head stays at the mainnet genesis block for the whole run. Every expectation is a mainnet genesis literal: roots, signing domains, the first validators, duties and the empty phase0 block. The target covers genesis, spec, fork schedule, domains, node status, block root/header/body, validators, attester and proposer duties, attestation data, block production and the event stream.

…ghthouse The target accepts proposer preparations, sync committee subscriptions and empty attestation, sync message and contribution batches. Rejected requests surface as `HttpError` with status, method and path: 404 for unknown roots and absent aggregates, 400 for infinity-signed payloads, the DVT selection endpoints and epochs beyond the node's horizon, 500 for builder registrations without a builder and a blinded block with a zero execution block hash. Batch endpoints report the failing item by index, and an empty aggregate batch is refused client-side as `PayloadError::Empty` before any request is sent.

The validator cache changes from #667 are covered by the branch passing EthBeaconNodeApiClient and ValidatorCache to their consumers directly; the seen-pubkeys observer removal from #687 is kept.

emlautarom1-agent

Inline notes for reviewers: deliberate deviations from Charon, follow-ups, and the reasons behind the live test fixture.

emlautarom1-agent · 2026-09-09T01:00:59Z

+}
+
+/// Upstream statuses the duty endpoints propagate to the VC as-is.
+const DUTIES_PROPAGATED_STATUSES: &[StatusCode] =


Differs from Charon. Charon's validatorapi answers 500 for every upstream failure. Pluto forwards 400 and 503 from the duty endpoints to the VC, so a syncing beacon node shows up as 503 rather than 500. Kept deliberately.

emlautarom1-agent · 2026-09-09T01:00:59Z

+                if http.body.message.contains("PriorAttestationKnown")
+                    || http
+                        .body
+                        .failures
+                        .iter()
+                        .any(|failure| failure.message.contains("PriorAttestationKnown")) =>


Parity quirk. Charon does strings.Contains(err.Error(), "PriorAttestationKnown") on the whole error string. Here the match is on the decoded body message and on per-item failures, the two places a beacon node puts the text.

emlautarom1-agent · 2026-09-09T01:00:59Z

+            Err(EthBeaconNodeApiClientError::Http(http))
+                if http.status == StatusCode::NOT_FOUND =>


Only 404 mapping in the client. Charon checks 404 only for block lookups (tracker inclusion, synthetic proposer). Every other endpoint surfaces 404 as Http; the fetcher maps aggregate attestation and sync contribution 404s to its own not-found variants.

emlautarom1-agent · 2026-09-09T01:00:59Z

+            // Failed to fetch by slot, fall back to head state.
+            Err(_) => (self.fetch("head").await?, false),


Parity quirk. Charon's GetBySlot falls back to head on any error, not only on 404. Kept as is.

emlautarom1-agent · 2026-09-09T01:00:59Z


    // Broadcasting uses a separate client with the (distinct) submit timeout.
-    let submission_api = build_api_client(&beacon_node_addr, config.beacon_node_submit_timeout)?;
+    let submission_client = build_api_client(&beacon_node_addr, config.beacon_node_submit_timeout)?;


Two caches. Each client owns its chain config cache, so this submission client fetches spec, genesis and fork schedule once on its own. One client with per-request timeouts would remove that; left for a follow-up.

emlautarom1-agent · 2026-09-09T01:01:00Z

+}
+
+impl BeaconNodeContainer {
+    pub(crate) fn client(&self) -> EthBeaconNodeApiClient {


Why a client per test. Each #[tokio::test] runs on its own runtime, and a pooled reqwest connection is bound to the runtime that opened it. A client shared across tests hands later tests dead connections (DispatchGone).

emlautarom1-agent · 2026-09-09T01:01:00Z

+                "--disable-discovery",
+                "--disable-upnp",
+                "--target-peers",
+                "0",


Fixture. Without these flags the node finds peers and starts syncing during the run. With them the head stays at the genesis block, so every asserted value is a fixed mainnet genesis literal and the suite needs no network after the image pull.

emlautarom1-agent · 2026-09-09T01:01:00Z

+}
+
+#[tokio::test]
+async fn get_sync_committee_duties_beyond_the_horizon_is_a_bad_request() {


Why this epoch. Any request that needs a post-genesis state (sync duties at epoch 0, attester duties at the Altair epoch, any sync committee message) makes Lighthouse spend about a minute advancing the genesis state before failing. An epoch beyond the node's horizon fails instantly with 400.

emlautarom1-agent · 2026-09-09T01:01:00Z

+}
+
+#[tokio::test]
+async fn publish_blinded_block_v2_cannot_reconstruct_a_payload_with_a_zero_block_hash() {


Why Bellatrix. Blinded blocks exist from Bellatrix, so the client has no phase0 blinded variant. Lighthouse rejects a zero execution block hash with 500 before checking signatures, which gives this method a deterministic live request.

emlautarom1-agent · 2026-09-09T01:01:00Z

+}
+
+#[tokio::test]
+async fn submit_pool_attestations_v2_fails_a_single_attestation_by_index() {


Lighthouse quirk. POST /eth/v2/beacon/pool/attestations is decoded by wall-clock fork regardless of Eth-Consensus-Version; phase0-shaped attestations get 400 missing field committee_index. Only the Electra SingleAttestation branch of pool_attestations is reachable live, which is also the only branch any current network exercises.

The free-function qualification from #668 is applied to the branch's typed code in exit.rs, signeddata.rs and unsigneddata.rs; the string re-parsing helpers in serde_utils.rs stay deleted.

emlautarom1

Steered the agent to produce this PR. Changes are for the most part mechanical, reducing error plumbing and merging type definitions. Semantics ought to be the same.

Follow ups include the multi-beacon-endpoint support and complete endpoint metrics instrumentation. This also supersedes #667.

iamquang95 · 2026-09-11T11:24:18Z

+        &self,
+        selections: &[v1::BeaconCommitteeSelection],
+    ) -> Result<Vec<v1::BeaconCommitteeSelection>> {
+        let body = self


Do we need metrics::instrument?

iamquang95 · 2026-09-11T11:24:38Z

+        &self,
+        preparations: &[v1::ProposalPreparation],
+    ) -> Result<()> {
+        self.send(


Do we need metrics::instrument?

iamquang95 · 2026-09-11T11:24:45Z

+        &self,
+        selections: &[v1::SyncCommitteeSelection],
+    ) -> Result<Vec<v1::SyncCommitteeSelection>> {
+        let body = self


Do we need metrics::instrument?

iamquang95 · 2026-09-11T11:24:54Z

+        &self,
+        subscriptions: &[v1::SyncCommitteeSubscription],
+    ) -> Result<()> {
+        self.send(


Do we need metrics::instrument?

emlautarom1-agent Bot added 25 commits September 1, 2026 19:39

refactor: inline pass-through helpers and drop unused eth2api extensions

e191e88

Removes resolve_fork_version, fetch_genesis_validators_root and fetch_genesis_fork_version, which had no callers, and inlines the single-use parse_body and filter_active helpers in the test mocks.

refactor(eth2api): dissolve the extensions module

eea3c9b

The client error enum gets its own module; BeaconNodeEvent and the domain computations live with the other types; the cached config fetches, the event stream and the proposer-duty filter are client methods.

refactor(eth2api): drop BeaconNodeClient

8feff13

The scheduler, broadcaster and recaster take the EthBeaconNodeApiClient and the ValidatorCache directly. The cache is built from the cluster pubkeys before its consumers, so the set-after-construction slot and its lock go away.

refactor(eth2api): return None from get_block_v2 when no block exists

b5472ca

A 404 on GET /eth/v2/beacon/blocks/{block_id} means no block was proposed, so the client reports it as Option::None and the inclusion tracker no longer inspects HTTP statuses.

test(eth2api): check derived request bodies for presence only

388a59a

The request-shape table asserts the exact JSON only for the two bodies the client assembles itself; the serde encodings are covered by the type tests.

refactor(eth2api): derive the HttpError display

47ec4ae

The display shows the response status followed by the body message; per-item batch failures are available structurally through `body.failures`.

refactor(core): wrap beacon client errors directly in bcast

4dab0c1

`Error::Client` carries the beacon node client error; the endpoint and method recorded inside it identify the failed call.

style: qualify free-function calls in eth2api and beaconmock

1b43f62

Free functions are called through their module path so the defining module is visible at each call site.

Merge branch 'main' into eth2api-client

55b3d39

The validator cache changes from #667 are covered by the branch passing EthBeaconNodeApiClient and ValidatorCache to their consumers directly; the seen-pubkeys observer removal from #687 is kept.

emlautarom1-agent Bot commented Sep 9, 2026

View reviewed changes

Merge branch 'main' into eth2api-client

70d5853

The free-function qualification from #668 is applied to the branch's typed code in exit.rs, signeddata.rs and unsigneddata.rs; the string re-parsing helpers in serde_utils.rs stay deleted.

emlautarom1 marked this pull request as ready for review September 10, 2026 23:57

emlautarom1 approved these changes Sep 11, 2026

View reviewed changes

iamquang95 approved these changes Sep 11, 2026

View reviewed changes

		Err(EthBeaconNodeApiClientError::Http(http))
		if http.status == StatusCode::NOT_FOUND =>

		// Failed to fetch by slot, fall back to head state.
		Err(_) => (self.fetch("head").await?, false),

Uh oh!

Conversation

emlautarom1-agent Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Differences from Charon

Out of scope

Uh oh!

emlautarom1-agent Bot left a comment

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1-agent Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

emlautarom1 left a comment

Choose a reason for hiding this comment

Uh oh!

iamquang95 Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

iamquang95 Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

iamquang95 Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

iamquang95 Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

emlautarom1-agent Bot commented Sep 9, 2026 •

edited

Loading