feat(gateway): load balancing - #370
Conversation
|
💬 Discussion in Slack: #pr-review-cli-370-feat-gateway-report-active-connection-count-for-pool-load-bala Posted by Review Police — reviews, comments, new commits, and CI failures will stream into this channel. |
|
| Filename | Overview |
|---|---|
| packages/api/api.go | Adds the v2 gateway load-report endpoint wrapper; its synchronous request inherits the shared client's lack of a timeout. |
| packages/api/model.go | Adds the activeChannels request DTO with the expected JSON field. |
| packages/gateway-v2/gateway.go | Adds atomic channel accounting and periodic reporting, with a non-blocking concern around indefinitely stalled HTTP requests. |
Reviews (1): Last reviewed commit: "feat(gateway): report active channel cou..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
bernie-g
changed the title
Counts channels in handleIncomingChannel, the single point every inbound channel passes through whatever opened it, and reports the total every 10s so the platform can route new work to the least busy member of a pool. Kept off the heartbeat deliberately: a heartbeat makes the platform dial back through the relay and write to its database, which is far too costly at the cadence selection needs. A missed report only costs accuracy, so failures are logged at debug and the platform falls back to its own view.
The report ran with no deadline, so an endpoint that accepts the connection and then stalls would block the reporting loop indefinitely: no further reports, and no reaction to cancellation either, so shutdown would hang behind it. Each report now runs on a context with a 5s deadline derived from the gateway's own context, which keeps a stall from reaching the next tick and lets cancellation abort a request already in flight.
c8f7b62 to
1d66d56
Compare
Clears GO-2026-6163, a panic on a malformed XOR-MAPPED-ADDRESS attribute. Reached through pion/turn in the gateway relay path. v3.1.5 is the first fixed release; later patches pull in dtls and transport bumps this does not need.
The e2e module replaces the root module, so its go.sum needs the same versions or every e2e job fails with "updates to go.mod needed".
The doc on the reporting loop said the platform falls back to its own view when a gateway stops reporting. It no longer does: a non-reporting member takes its whole pool off load-aware selection.
A handler that never returns left the count high for the life of the process, so the gateway reported itself permanently busy and stopped attracting work. The count resets when a relay connection is established, since channels do not outlive the connection they arrived on, and the decrement is floored so a handler from the previous connection cannot push it negative. Reports against a platform with no metrics endpoint retried every 10s forever. After five consecutive failures the interval drops to five minutes and warns once, naming the consequence: pools containing the gateway select at random. Backs off rather than stopping, so an upgraded platform recovers without a gateway restart. reportMetrics becomes startMetricsReport, since it spawns the loop.
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
| if current <= 0 { | ||
| return | ||
| } | ||
| if g.activeChannels.CompareAndSwap(current, current-1) { |
There was a problem hiding this comment.
Low: Stale handlers decrement the new connection's count
After activeChannels is reset during reconnect, this CAS cannot tell whether the releasing handler belongs to the old or current connection. An authenticated client can keep old handlers stalled across a reconnect and close them after new channels are counted, making those stale handlers decrement the new connection's load and attract additional traffic. Track counts per connection generation, or pass a generation-specific counter into each handler, so releases only affect the connection that acquired them.
PR overviewThis pull request adds load balancing to the gateway, including tracking active channels across connection and reconnection lifecycles. One low-impact load-accounting issue remains open. An authenticated client can hold handlers across a reconnect and later cause the new connection's active-channel count to be understated, potentially skewing load balancing and attracting excess traffic to that connection. Open issues (1)
Fixed/addressed: 0 · PR risk: 5/10 |
3 participants
Description 📣
The gateway now keeps a count of how many connections it is currently handling and reports that number to Infisical every 10 seconds. This lets gateway pools send new work to whichever gateway is least busy, including work the platform never sees directly such as PAM sessions opened from the CLI.
Pairs with the platform change in Infisical/infisical#7703. Older gateways simply don't report, and the platform falls back to its own view for those.
Type ✨
Tests 🛠️
Ran two gateways in a pool and watched the reported count while holding connections open. It rises with each connection, holds while they are open, and returns to zero when they close. Verified this includes a PAM session opened with
infisical pam access, which the platform hands certificates to and never proxies itself.