fix: serve /peers from the same address cache as /providers#153
Open
lidel wants to merge 6 commits into
Open
Conversation
someguy is a caching routing proxy, so /routing/v1/peers/{peerid} now
answers from the cached address book and host peerstore before falling
back to a DHT lookup, matching /providers. A relay-dependent peer that
is absent from peer routing but recently seen as a provider is no longer
answered with an empty result.
- cachedRouter.FindPeers: check the peerbook first, fall back to peer
routing only on a miss, and enrich address-less hits from cache
- cachedAddrBook.GetCachedAddrs: fall back to the host peerstore, which
the DHT fills with provider addresses during FindProviders
- cachedAddrBook.CacheAddrs: persist addresses observed in provider
records so later peer lookups can serve them
Add docs/peer-address-caching.md explaining the two address stores, how the cache is filled and refreshed by the probe loop, and the cache-first read path on /providers and /peers, with mermaid diagrams. Note that caching requires a DHT-backed instance and is bypassed with --dht=disabled. - README: add a Documentation section linking all docs, and replace cid.contact references with the Delegated Routing V1 spec - AGENTS.md: add repo orientation, build/test commands, code map, and doc index
Cached peer addresses previously only accumulated: provider records, DHT gossip, and successive identifies each unioned into the address book, so a peer could collect outdated certhashes, dead relay circuits, and rotated NAT ports that never expired until each entry's TTL lapsed. On a completed identify, replace the peer's stored set with its current advertised addresses (signed peer record, else identify listen addresses) unioned with any live-connection address, so a reachable peer collapses back to its current set each refresh. An identify carrying no usable addresses returns early and leaves the existing set untouched.
Cache-first FindPeers recorded the metric twice on a cache miss: once when the cache-first lookup missed, then again in the post-DHT enrich step when the record already carried addresses. That made sum(origin=peers) exceed the request count and the hit-rate uncomputable. Read the cache directly in the enrich step so it no longer records a lookup; FindPeers now emits exactly one peer_addr_lookups increment per request (hit when served cache-first, miss when it falls back to the DHT). Also point the CHANGELOG doc link at an absolute URL so it resolves when copied into GitHub release notes.
On identify, the live-connection addresses unioned into the cache came from every connection's RemoteMultiaddr, including inbound ones. An inbound connection's remote address is the peer's ephemeral source port, which is not dialable, so caching it reintroduced the junk the identify-time prune is meant to remove. Keep only outbound (and direction-unknown) connection remotes. Rename the host-peerstore fallback field from peerstore to hostPeerstore so it no longer shadows the imported peerstore package and reads as the counterpart to addrBook (someguy's own book vs the libp2p host's).
Addresses came back in nondeterministic order because the peerstore holds them in a map, so repeated requests for the same peer or provider returned the same addresses shuffled differently. Sort by bytes in filterPrivateMultiaddr, which sanitizeRouter already runs on every record from every router, so providers, peers, and closest-peers responses are stable regardless of whether the addresses came from the cache, the DHT, or a delegated HTTP backend.
lidel
force-pushed
the
fix/peers-use-cached-peerbook
branch
from
06fb299 to
fa15bdf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
/routing/v1/peers/{peerid}queried peer routing on every request and returned an empty result whenever the DHT could not locate the peer, even when someguy already held that peer's addresses. A relay-dependent, NAT'd peer that announces content appears under/routing/v1/providers/{cid}with addresses but is not findable by DHT peer routing, so/peersanswered{"Peers":[]}for a peer whose addresses someguy already knew. The two endpoints drew from different sources for the same peer.Fixing this surfaced a second problem: cached addresses only accumulated. Provider records, DHT gossip, and successive identifies each unioned into the address book, so a peer could collect 150+ (or more) stale entries (dead relay circuits, outdated certhashes, rotated NAT ports) that lingered until each entry's TTL lapsed.
Fix
/peersis now cache-first, like/providers: it answers from the cached address book and the libp2p host peerstore (which the DHT fills with provider addresses duringFindProviders) before falling back to a DHT lookup. someguy is a caching routing proxy, not a libp2p node, so it favors a low-latency answer from recently seen, actively probed peers.peer_addr_lookupsis recorded once per/peersrequest, not twice on a cache miss, so its hit-rate is meaningful again.docs/peer-address-caching.mddocuments both stores, the refresh loop, and the per-endpoint read paths. The improvement is cache-driven: it helps when the peer was recently seen as a provider, but a cold instance that has never seen the peer still returns not-found.