feat(addressbook): prune stale entries by last-seen time#5513
feat(addressbook): prune stale entries by last-seen time#5513martinconic wants to merge 1 commit into
Conversation
155323e to
1716cdb
Compare
|
Here, prune here runs once, at startup only, and it can lead that if we have some stable connections over bigger period of time, where we haven't updated LastSeen, it will remove those on next restart. But those are our most valuable connections. Maybe we could use One gap is left, what do to with the peer we only hear over hive gossip, but we didn't connect to? |
| now := time.Now().Unix() | ||
|
|
||
| for _, e := range batch { | ||
| var fields map[string]json.RawMessage |
There was a problem hiding this comment.
this is not needed. you can shadow the type which is currently on master as a local type scoped to the function, and use it directly to unmarshal and conversion onto the new type. when i think of it, you could even have the latest version (just annotate the last seen field with the json omitempty instruction), and you could unmarshal the existing old serialization format into it, just set the timestamp and write it back into the db.
| k.connectedPeers.Add(peer.addr) | ||
|
|
||
| if err := k.addressBook.UpdateLastSeen(peer.addr); err != nil { | ||
| k.logger.Debug("could not update last seen for peer", "peer_address", peer.addr, "error", err) |
| k.connectedPeers.Add(addr) | ||
| k.waitNext.Remove(addr) | ||
| if err := k.addressBook.UpdateLastSeen(addr); err != nil { | ||
| k.logger.Debug("could not update last seen for peer", "peer_address", addr, "error", err) |
This is what we've converged to with @janos as a naive starting point to ship.
For now let's leave them in. The first line of defense is to get rid of garbage which is in the statestore. A peer which is seen over hive and is not connected to is still a potential peer you'd like to connect to. I agree that over time yes, probably you don't wanna keep those. I suggest to ship this first and see it works, then evolve into more well defined scenarios. |
Checklist
Description
Implements address book pruning (#5491).
The address book recently gained a wrapping
verifiedAddress{Address, Verified}struct (v2.8.0, #5477). This PR extends it with a last-seen timestamp and uses
it to drop peers we have not seen for over a month, preventing the address book
from accumulating stale, unreachable peers indefinitely.
Changes
pkg/addressbookLastSeen(unix seconds) toverifiedAddress.PutstampsLastSeenon every write — covers the hive gossip path, whichPuts on each sighting.UpdateLastSeen(overlay): read-modify-write that bumps only the timestamp; no-op if the overlay is absent.Prune(before time.Time): removes entries last seen before the cutoff. Entries withLastSeen == 0are kept.pkg/topology/kademlia: callUpdateLastSeenon successful connection, both outbound (connectionAttemptsHandler) and inbound (onConnected). Failures are debug-logged and non-fatal.pkg/node: prune entries not seen for 30 days at startup, right after the address book is constructed. Warning-logged and non-fatal so it never blocks boot.pkg/statestore/storeadapter: migration step 10 stampslast_seen = now()onto existing entries that lack it (merging into the JSON object so all fields are preserved). Without this, the first prune after upgrade would wipe the whole address book.AI Disclosure