Skip to content

docs: note public testnet RPC method differences across follow endpoints - #373

Open
kutluhaneth46 wants to merge 2 commits into
circlefin:mainfrom
kutluhaneth46:cursor/docs-rpc-endpoint-capabilities-88c1
Open

docs: note public testnet RPC method differences across follow endpoints#373
kutluhaneth46 wants to merge 2 commits into
circlefin:mainfrom
kutluhaneth46:cursor/docs-rpc-endpoint-capabilities-88c1

Conversation

@kutluhaneth46

Copy link
Copy Markdown
Contributor

Summary

  • Clarifies that the public testnet --follow.endpoint URLs in docs/running-an-arc-node.md are interchangeable for follow sync, but not for application JSON-RPC method support.
  • Adds a compact capability matrix for eth_getProof / eth_createAccessList and a note about unstable -32601 vs -32014 answers on some hosts.

Docs-only; does not change provider gateways. Distinct from #222 / #299 (rate limits / gas caps / fallback config).

Addresses #371.

Test plan

  • Diff review of the callout next to the follow-mode endpoint list
  • Maintainer confirm matrix still matches live fleet before merge

The follow endpoints in running-an-arc-node.md are equivalent for sync,
but not for application JSON-RPC (eth_getProof / eth_createAccessList).
Add a compact capability matrix and guidance for unstable error codes.
Addresses circlefin#371.
@osr21

osr21 commented Sep 9, 2026

Copy link
Copy Markdown

Thanks for turning this around so quickly, and for splitting follow-sync from application RPC — that was my main concern on #371 and this resolves it cleanly.

Discharging the open test-plan box

Your checklist has "maintainer confirm matrix still matches live fleet before merge" unchecked. Re-ran it just now, 2026-09-09 17:46 UTC, 5 samples per method per host:

rpc.testnet.arc.io        getProof: -32601 -32601 -32014 -32014 -32014 | createAccessList: -32601 ×3
rpc.drpc.testnet.arc.io   getProof: OK OK OK OK OK                     | createAccessList: OK ×3
rpc.quicknode...arc.io    getProof: -32601 -32014 -32601 -32601 -32014 | createAccessList: -32601 ×3
rpc.blockdaemon...arc.io  getProof: OK OK OK OK OK                     | createAccessList: -32003 ×3

Every row in your table matches. No drift since #371 was filed.

I also verified the "interchangeable for follow sync" claim rather than assuming it: the follow client's public-RPC surface is eth_getBlockByNumber, eth_call, eth_chainId, net_listening, and all four are served by all four hosts both individually and in a single batch. The engine_* family goes over the auth socket. Worth noting the follow client doesn't batch anyway — the serde_json::json!([...]) sites in crates/eth-engine/src/rpc/ethereum_rpc.rs are params arrays, not batch envelopes — so blockdaemon's batch filtering cannot reach it. The claim is safe as written.

One thing I'd change before merge: the probe advice is unsound

treat capability probes as sticky only after a stable -32601, not after a single sample

I think this bakes in a recipe that can't be followed, and it's my fault for not being clearer in #371. There is no "stable -32601" state to wait for. Same host, same request, three sittings:

when rpc.testnet.arc.io rpc.quicknode.testnet.arc.io
2026-09-08 (n=10) -32014, 2×-32601 -32601, 4×-32014
2026-09-09 14:55Z (n=5) -32601, 2×-32014 -32601, 3×-32014
2026-09-09 17:46Z (n=5) -32601, 3×-32014 -32601, 2×-32014

The mix reshuffles every time and neither code ever settles. A client told to wait for a stable -32601 before caching "unsupported" will keep probing indefinitely against a host that is permanently unable to serve the method — the exact opposite of the intended outcome.

The guidance the data actually supports is: don't branch on the code at all. Either code means "this host will not serve this method"; treat any error from a capability probe as unsupported and fail over to a host that does. Suggested replacement:

Some hosts return -32601 (method unsupported) for one call and -32014 (data unavailable) for the next, for the same request. Do not branch on the code — treat any error from a capability probe as "unsupported on this host" and fail over.

Smaller points

quicknode is missing from the table. rpc.quicknode.testnet.arc.io is a live public endpoint with the same divergence (data above: getProof unsupported, createAccessList -32601 3/3). Since the note is framed for application clients rather than follow operators, and says "measured against the public fleet", a reader may well be pointed at it. Either add the row or scope the sentence to "the follow endpoints listed above".

"rejects whole batches" reads stronger than the behaviour. Blockdaemon handles batches fine — the four-method follow batch above succeeded there. What it does is reject a batch containing a filtered method, discarding the unrelated calls beside it with a single top-level error ("id": null) instead of a per-item error inside the array. Maybe: "filtered (-32003); a batch containing it is rejected in full, losing unrelated calls".

The WS suffix deserves a clause here. The note now says the endpoints are interchangeable for follow sync, and sits directly above a list where blockdaemon uniquely needs wss=rpc.blockdaemon.testnet.arc.io/websocket while the other two take the bare host. A reader acting on "interchangeable" by swapping WS URLs pattern-wise will get it wrong. Half a sentence — "note the differing WebSocket paths above" — closes it.

Date-stamp the matrix. This is provider-side state that can change without any commit to this repo, and the unchecked box in your test plan is effectively an unbounded recurring ask on maintainers. An "observed YYYY-MM-DD" line plus the one-line repro makes it falsifiable by any reader instead:

curl -s -X POST -H 'Content-Type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"eth_getProof","params":["0x3600000000000000000000000000000000000000",["0x0"],"latest"]}' \
  https://rpc.testnet.arc.io

Nit: the hunk adds two consecutive blank lines before ### Verify operation.


Disclosure: I'm an external community contributor, not affiliated with Circle, with no write access to this repository. Advisory review only, not an approval — all measurements above are from my own probing of the public endpoints and should be confirmed independently.

Address review on circlefin#373/circlefin#371: clarify follow sync is unaffected, fix
probe advice (do not wait for stable -32601), add quicknode + date stamp,
soften blockdaemon batch wording, note differing WS paths, and drop the
extra blank line.
@kutluhaneth46

Copy link
Copy Markdown
Contributor Author

Thanks @osr21 — excellent catch on the probe advice and the follow-vs-app split.

Updated the note to:

  • state plainly that follow sync is unaffected (and name the four methods)
  • replace the "stable `-32601`" guidance with "treat any probe error as unsupported and fail over"
  • add the quicknode row + observed 2026-09-09 date stamp + one-line repro
  • soften the blockdaemon batch wording
  • mention the differing WebSocket paths
  • drop the extra blank line

Matrix re-check appreciated — ready for another pass whenever you have a minute.

@osr21

osr21 commented Sep 9, 2026

Copy link
Copy Markdown

Revision looks good — the probe guidance, quicknode row, date stamp, batch wording, and WebSocket clause all read correctly now. Two things before you consider this final, and the first is me correcting my own review.

I got the batching claim wrong

In my last comment I wrote that "the follow client doesn't batch — the serde_json::json!([...]) sites in crates/eth-engine/src/rpc/ethereum_rpc.rs are params arrays, not batch envelopes."

That was the wrong file. EthereumRPC is the general execution-client wrapper. The actual follow client is crates/malachite-app/src/rpc_sync/client.rs, and fetch_blocks_batch builds two genuine JSON-RPC batches per height range and fires them in parallel via tokio::join!. So follow does batch. My conclusion held, but the reasoning behind it didn't, and the blockdaemon batch caveat is more load-bearing for follow than I implied, not less.

The four-method list is missing two methods

The note now says the follow client "only needs eth_getBlockByNumber, eth_call, eth_chainId, and net_listening". That's the list I gave you in #371 and it's incomplete. The follow path also uses:

  1. arc_getCertificate — batched one per height in fetch_blocks_batch, right alongside eth_getBlockByNumber. Non-standard, Arc-specific, and absolutely required for follow.
  2. eth_subscribe("newHeads") — over the wss= URL in rpc_sync/ws_subscription.rs (alloy's subscribe_blocks()), for peer height tracking, with unbounded retry/backoff.

I verified all three follow endpoints serve eth_subscribe("newHeads") over WSS. Your WebSocket clause is also empirically correct and worth keeping — the bare host genuinely fails:

wss://rpc.blockdaemon.testnet.arc.io/websocket   OK  subId=0x8407e45be509
wss://rpc.blockdaemon.testnet.arc.io  (bare)     connection failure

The bigger one: blockdaemon cannot serve follow catch-up below ~41.86M

Adding arc_getCertificate to the list matters because the hosts do not agree on it, which makes "those work on every public host below" true only for recent heights.

arc_getCertificate            h=1     h=1e3   h=1e6   h=1e7   h=3e7   h=5e7
rpc.testnet.arc.io            OK      OK      OK      OK      OK      OK
rpc.drpc.testnet.arc.io       OK      OK      OK      OK      OK      OK
rpc.blockdaemon...arc.io      MISS    MISS    MISS    MISS    MISS    OK

blockdaemon returns -32004 "Certificate not found" for old heights. This is not block pruning — eth_getBlockByNumber at height 1,000,000 returns a full block on that same host. It has the blocks and not the certificates.

Binary-searching the cutoff gives a single sharp boundary, not sporadic gaps:

MISS at 41,863,834   |   OK at 41,863,835      (head 61,306,927, measured 2026-09-09 23:3x UTC)

So roughly the last 19.44M blocks (~112 days at 500 ms), though one measurement can't distinguish a rolling window from a fixed restore point.

Why this is operationally sharp rather than cosmetic: in send_batch_request, any per-item JSON-RPC error maps to Err for the whole batch, and fetch_range propagates it. One missing certificate in a range fails the entire range fetch, not just that height. A node catching up from genesis — or from any height below the cutoff — against --follow.endpoint https://rpc.blockdaemon.testnet.arc.io will fail there, and that is exactly the new-operator path this document is for.

Suggested addition after the table:

Follow sync also uses arc_getCertificate (batched per height) and eth_subscribe("newHeads") over the wss= URL. rpc.blockdaemon.testnet.arc.io serves certificates only for recent history (observed: nothing below block ~41,863,835 on 2026-09-09) while still serving the blocks themselves, so use rpc.testnet.arc.io or rpc.drpc.testnet.arc.io when catching up from genesis or from deep history.

Repro:

curl -s -X POST -H 'Content-Type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"arc_getCertificate","params":[1000000]}' \
  https://rpc.blockdaemon.testnet.arc.io
# {"jsonrpc":"2.0","id":1,"error":{"code":-32004,"message":"Certificate not found"}}

Worth a maintainer's eye on whether that retention boundary is intended for a documented follow endpoint, since it may be a provider configuration issue rather than a docs issue. Either way the doc shouldn't imply the three are interchangeable for catch-up.

Minor, in support of your current wording

txpool_status / txpool_inspect also exist on EthereumRPC, but they have no production consumer — only eth-engine/tests/integration.rs and crates/test/checks/src/mempool.rs — and no public endpoint serves them (all three return -32601). Correctly excluded from the follow list; noting it so the omission doesn't look accidental to a later reader.


Disclosure: I'm an external community contributor, not affiliated with Circle, with no write access to this repository. Advisory review only, not an approval — all measurements are from my own probing of the public endpoints on 2026-09-09 and should be confirmed independently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants