rpc: public testnet endpoints disagree on standard methods, and identical eth_getProof requests return two different error codes

[osr21]

Summary

The four public testnet RPC endpoints referenced in docs/running-an-arc-node.md do not serve the same JSON-RPC method set, and two of them return two different, semantically contradictory error codes for byte-identical requests. A client written against the canonical rpc.testnet.arc.io cannot use EIP-1186 state proofs at all, while the same call succeeds on rpc.drpc.testnet.arc.io.

Capability matrix

Single (non-batch) requests, 3 consecutive samples each:

Endpoint	eth_getProof	eth_createAccessList
rpc.testnet.arc.io	-32014 / -32601 (varies)	-32601 ×3
rpc.drpc.testnet.arc.io	OK ×3	OK ×3
rpc.quicknode.testnet.arc.io	-32601 / -32014 (varies)	-32601 ×3
rpc.blockdaemon.testnet.arc.io	OK ×3	-32003 "Request method filtered" ×3

Also unavailable on the canonical endpoint: eth_simulateV1, eth_blobBaseFee, txpool_status (all -32601). eth_getBlockReceipts and eth_maxPriorityFeePerGas work.

The part that actually breaks clients: unstable error codes

Ten consecutive identical eth_getProof calls, same hostname, ~0.8s apart:

rpc.testnet.arc.io            -32014 -32601 -32601 -32014 -32014 -32014 -32014 -32014 -32014 -32014
rpc.quicknode.testnet.arc.io  -32601 -32601 -32601 -32601 -32601 -32014 -32014 -32601 -32014 -32014

So rpc.testnet.arc.io answered -32014 8/10 and -32601 2/10; quicknode split 6/4 the other way. These two codes mean opposite things to a client:

• -32601 "method not supported" is permanent — a well-behaved client disables the code path or falls back.
• -32014 "requested data not available" reads as transient/state-related — a client retries, or retries against a different block.

Whichever branch a library takes, it takes at random, per call. Retry logic and capability detection both become nondeterministic. Any capability probe run once at startup will cache whichever answer it happened to get.

Batch semantics diverge too

Batch of three, where the middle request is eth_createAccessList:

rpc.testnet.arc.io             array of 3 -> id=1:OK, id=2:-32601, id=3:OK
rpc.drpc.testnet.arc.io        array of 3 -> id=1:OK, id=2:OK,     id=3:OK
rpc.quicknode.testnet.arc.io   array of 3 -> id=1:OK, id=2:-32601, id=3:OK
rpc.blockdaemon.testnet.arc.io TOP-LEVEL ERROR (whole batch rejected): {"code":-32003,"message":"Request method filtered"}

Blockdaemon discards the entire batch and replies with a single error object ("id": null) instead of an array. JSON-RPC 2.0 §6 specifies a batch response as an array of individual responses, so a single filtered method silently kills the two unrelated calls beside it. Confirmed it is the method and not batching in general — a batch of eth_chainId + eth_blockNumber succeeds on that endpoint, and eth_getProof on its own succeeds there too.

Where this appears to come from

grep -rIn 'getProof\|get_proof\|create_access_list' --include='*.rs' crates/ returns nothing, so these are inherited from upstream reth rather than overridden in this repo, and rpc.drpc.testnet.arc.io returns real proofs for the same address and block tag. The state is therefore available on the network; the divergence looks like per-provider gateway policy rather than a node capability limit. That also explains the intra-endpoint variation, which is consistent with load-balancing across a fleet whose members do not answer identically.

This matters for the docs specifically because docs/running-an-arc-node.md lists these endpoints together as interchangeable --follow.endpoint values (lines ~211-213 and ~883-885), which reads as "equivalent".

Suggested resolution

Any one of these would remove the ambiguity:

1. Serve a consistent method set across the documented endpoints, or
2. Document per-endpoint capabilities so eth_getProof consumers know to select drpc, and
3. Return a stable code for an unsupported method on a given endpoint — pick -32601 and keep it, so capability detection and retry logic are deterministic.

The batch behaviour on blockdaemon is worth raising with that provider separately; a filtered method should produce a per-request error inside the array, not a top-level rejection.

Reproduction

curl -s -X POST -H 'Content-Type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"eth_getProof","params":["0x3600000000000000000000000000000000000000",["0x0"],"latest"]}' \
  https://rpc.testnet.arc.io

Repeat ~10 times against each host and compare. Chain id 0x4cef52 (5042002) on all four.

---

Disclosure: I am external to Circle — an unaffiliated community contributor with no write access to this repository. This report is advisory only; measurements are from my own probing of the public endpoints and should be reproduced before acting on them.

[kutluhaneth46]

Taking the docs half of the suggested resolution: I'll add a short note in docs/running-an-arc-node.md that the listed public testnet endpoints are not interchangeable for methods like eth_getProof / eth_createAccessList, with a compact capability matrix. Not changing provider gateways from this repo.

[osr21]

Thanks for picking this up — and agreed that gateway policy isn't changeable from this repo, so a docs note is the right scope.

One caveat before you write it, because I think it changes where the note should go and what it should say. The two diverging methods are irrelevant to the --follow.endpoint use case that this file is about.

I traced the follow client's actual public-RPC surface in crates/eth-engine/src/:

Method	refs
eth_getBlockByNumber	14
eth_call	4
eth_chainId	2
net_listening	1

The engine_* family (newPayloadV1-V4, forkchoiceUpdatedV1-V3, getPayloadV1-V5, exchangeCapabilities, …) goes over the auth socket, not the public endpoint. And reth_subscribePersistedBlock, used by the persistence meter when --execution-persistence-backpressure is set, is passed engine.subscription_endpoint() in crates/malachite-app/src/handlers/consensus_ready.rs — the local execution client via --eth-socket, not anything in the --follow.endpoint list.

All four follow-required methods work on all four endpoints:

rpc.testnet.arc.io               getBlockByNumber:OK blockNumber:OK chainId:OK eth_call:OK net_listening:OK
rpc.drpc.testnet.arc.io          getBlockByNumber:OK blockNumber:OK chainId:OK eth_call:OK net_listening:OK
rpc.quicknode.testnet.arc.io     getBlockByNumber:OK blockNumber:OK chainId:OK eth_call:OK net_listening:OK
rpc.blockdaemon.testnet.arc.io   getBlockByNumber:OK blockNumber:OK chainId:OK eth_call:OK net_listening:OK

So a eth_getProof/eth_createAccessList matrix sitting next to the --follow.endpoint block (lines ~211-213, ~883-885) is likely to be mis-read as "your follow setup may be degraded depending on provider," which isn't true. Either give it its own section away from that command, or state plainly that follow is unaffected and name the four methods it needs. The audience actually harmed by the getProof gap is dapp/bridge/light-client developers, and they don't read the node-operator runbook — so this note may also be worth mirroring wherever the dapp-facing RPC docs live.

The interchangeability trap that is real in this file

Lines 211-213 already encode a per-provider asymmetry with no explanation:

--follow.endpoint https://rpc.testnet.arc.io,wss=rpc.testnet.arc.io
--follow.endpoint https://rpc.drpc.testnet.arc.io,wss=rpc.drpc.testnet.arc.io
--follow.endpoint https://rpc.blockdaemon.testnet.arc.io,wss=rpc.blockdaemon.testnet.arc.io/websocket

Blockdaemon needs the /websocket path suffix; the other two take the bare host. Someone adding or swapping an endpoint by pattern-matching these lines will get it wrong. That is a genuine "not interchangeable" caveat for node operators, and it belongs in your note more than getProof does. Also worth deciding whether to list quicknode here — it's a live endpoint and appears in the capability matrix, but isn't in this example.

Please document the verdict, not the error code

I re-ran the probe ~24h after filing. The capability verdicts are stable — drpc serves both methods, arc.io and quicknode serve neither, blockdaemon serves eth_getProof and filters eth_createAccessList. Safe to put in a table.

The error codes are not. rpc.testnet.arc.io was 8×-32014 / 2×-32601 over ten samples at filing time; today it was 3×-32601 / 2×-32014 over five. Same verdict, different mix. If the matrix hardcodes a specific code, it will be wrong a meaningful fraction of the time — better to say "rejected (code varies between -32601 and -32014)", which is also the honest signal to client authors that branching on the code is unsound.

Two suggestions to keep the note from rotting: stamp it "observed as of " and include the one-line curl repro, so a reader can re-derive it rather than trusting a frozen table. And consider a footnote that blockdaemon rejects an entire JSON-RPC batch with a single top-level -32003 when one member method is filtered, instead of a per-item error inside the array — unrelated calls in that batch are silently lost, which is easy to hit and hard to diagnose.

---

Disclosure: I am external to Circle — an unaffiliated community contributor with no write access to this repository. Advisory only; all measurements are from my own probing and code reading, and should be confirmed independently.

[huklaa]

Hi, I would like to work on this issue. Could you assign it to me?