Skip to content

doh: keep pooled conns shorter-lived than server idle-timeouts; h2 PING health-checks#242

Open
ShiroiKuma0 wants to merge 1 commit into
celzero:n2from
ShiroiKuma0:kojiki-doh-idle
Open

doh: keep pooled conns shorter-lived than server idle-timeouts; h2 PING health-checks#242
ShiroiKuma0 wants to merge 1 commit into
celzero:n2from
ShiroiKuma0:kojiki-doh-idle

Conversation

@ShiroiKuma0

Copy link
Copy Markdown

Fixes #241.

h2() pools DoH connections for 3 minutes (IdleConnTimeout) with no HTTP/2 keep-alive health-check. Several resolvers close idle DoH connections far sooner — measured from a fixed network, same day, reusing a TLS connection after an idle period:

resolver idle survived
dns.quad9.net EOF at ≤ 30 s
dns.mullvad.net EOF at ≤ 90 s
cloudflare-dns.com alive at > 240 s

With Quad9 (or any resolver whose idle timeout is under 3 minutes), every DNS lull longer than the server's idle window leaves the pool full of dead connections. The next queries are written into them, hang ~8–10 s, and surface as Post "…": unexpected EOF or http-status: 502; maxEOFTries often just picks the next dead pooled connection, so bursts of parallel queries fail together. Downstream on Android this expires alg mappings (realips([])), killing established flows of allowed apps, and fails the OS connectivity probes routed through the tunnel.

Change

Two adjustments in h2():

  1. IdleConnTimeout 3m → 10s — never keep a pooled connection longer than the shortest observed server-side idle window.
  2. http2.ConfigureTransports + ReadIdleTimeout = 10s / PingTimeout = 5s — actively evict half-dead h2 connections via PING instead of losing a query to them.

golang.org/x/net is already a module dependency, so the only new import is golang.org/x/net/http2.

Testing

Built from source and run on-device (Android, v055x/v055y engines). Same device, same resolver (Quad9), driven with worst-case traffic — bursts of 10 parallel uncached queries after guaranteed > 30 s idle gaps, ~40 cycles over 33 min:

  • Stock: query-failure bursts on nearly every cycle.
  • Patched: zero idle-pool failures; the log instead shows the health-check discovering and evicting each server-closed connection (retrier: read … err: EOF with no query error following), roughly one per idle cycle.

One caveat: with the PING enabled I saw a single rare stream error: stream ID N; PROTOCOL_ERROR reset taking out one burst on one connection (once in ~40 cycles; recovered on redial). If that turns out to be an interaction with the custom dialer, the IdleConnTimeout reduction alone already removes the corpse-pool state and may be the safer minimal change.

…NG health-checks

Some resolvers close idle DoH connections well under 30s (Quad9 measured
at <=30s from the same LAN; Cloudflare survives >240s). With the 3m
IdleConnTimeout, any DNS lull left only dead connections in the pool and
subsequent queries were written into them, hanging ~10s and surfacing as
'Post ...: unexpected EOF' / http-status 502 bursts (maxEOFTries retries
often just picked the next corpse). Downstream this expired alg mappings
(realips([])), killed established flows of allowed apps, and failed the
OS connectivity probes routed through the tunnel.

- IdleConnTimeout 3m -> 10s: never reuse a connection older than the
  shortest observed server-side idle window.
- http2.ConfigureTransports + ReadIdleTimeout/PingTimeout: actively
  evict half-dead h2 connections via PING instead of losing a query.
Comment thread intra/doh/doh.go
// subsequent queries were written into them — hanging ~10s and dying with
// "unexpected EOF" / http-status 502 bursts (maxEOFTries retries often just pick the next
// corpse). Keep pooled connections shorter-lived than any observed server-side idle window.
IdleConnTimeout: 10 * time.Second,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10s seems too low. Perhaps this could be et to 30s instead?

Comment thread intra/doh/doh.go
// Health-check h2 connections with PING frames so half-dead ones are evicted instead of
// eating a query first (net/http does not do this by default).
if t2, err := http2.ConfigureTransports(tr); err == nil && t2 != nil {
t2.ReadIdleTimeout = 10 * time.Second

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what these are required for. Just the 30s idle timeout seems sufficient...?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants