worldtree: multiplex per-zone source reads over a bounded connection pool#41
Merged
Merged
Conversation
…pool The socket-served source path (Mode B, `tec --sha`) opened one worldtree control-socket connection per clean zone in `getZoneStorePath`. A broad evaluation — the Tartarus planning bundle over a large changeset, run with parallel-eval + `eval-cores 0` — fans out to hundreds of zones at once, so it opened >512 connections in a burst and tripped worldtreed's per-listener connection cap (`ServeConfig::max_connections`, 512). Past the cap the daemon accepts-then-drops, which the C++ client sees as `worldtree: read(): Connection reset by peer`, aborting the eval. A hard cap on concurrent connections cannot fix this: the planner needs more zones mounted simultaneously than any safe per-listener cap allows. The daemon already supports many ephemeral RO sessions per connection (its owned-ephemeral set is per-connection and is drained on disconnect), so instead multiplex the per-zone scoped sessions over a bounded pool of connections. - `WorldtreeConn` gains a ws-parametric API (`openScopedSession`, `closeSessionOn`, `readTreeOn`, `readBlobOn`) so one physical connection can host many independent per-zone sessions; the existing single-session methods (used by the control connection) are unchanged. - New `WorldtreeZoneSession` handle owns one zone's scoped session on a shared carrier connection and releases it (`close_ro`) when the zone's `WorldtreeSourceAccessor` — its sole owner — drops. - `EvalState::acquireWorldtreeZoneSession` draws a carrier round-robin from a lazily-grown pool bounded by the new `tectonix-worldtree-max-connections` setting (default 64, well under the daemon's 512 cap), and opens the zone's scoped session on it. `getZoneStorePath` uses it instead of one connection per zone. Per-zone confinement is unchanged: each zone still gets its own `scoped` session (the daemon enforces the cone per session), just hosted on a shared connection. Reads that land on the same carrier serialize on it; cross-zone parallelism now comes from the pool holding several carriers rather than one connection per zone. Co-authored-by: Sonke Hahn <sonke.hahn@shopify.com>
sonkehahn-shopify
force-pushed
the
river-worldtree-connection-pool
branch
from
7daf343 to
cdcf61c
Compare
Cover the client pool in EvalState::acquireWorldtreeZoneSession, the logic most likely to silently regress to one connection per zone: bounded lazy growth, round-robin reuse past the cap, and close_ro on accessor drop. A minimal in-process fake worldtreed over a real AF_UNIX socket answers scoped.open_ro/close_ro on the actual control-socket wire, so the real Client codec, pool, and session RAII all run unstubbed. A narrow `friend class WorldtreePoolTest` exposes the private pool seam to the test.
burke
approved these changes
Jul 10, 2026
The accessor pool test relied on transitive includes for fetchers::Settings and openStore, which the darwin/source-tree include graph provides but the installed x86_64-linux -dev headers do not (eval.hh only forward-declares fetchers::Settings). Include the headers directly, matching the other libexpr-tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The socket-served source path (Mode B,
tec --sha) opened one worldtree control-socket connection per clean zone ingetZoneStorePath. A broad evaluation — the Tartarus planning bundle over a large changeset, run with parallel-eval +eval-cores 0— fans out to hundreds of zones at once, so it opened >512 connections in a burst and tripped worldtreed's per-listener connection cap (ServeConfig::max_connections, 512). Past the cap the daemon accepts-then-drops, which the C++ client sees asworldtree: read(): Connection reset by peer, aborting the eval.A hard cap on concurrent connections cannot fix this: the planner needs more zones mounted simultaneously than any safe per-listener cap allows. The daemon already supports many ephemeral RO sessions per connection (its owned-ephemeral set is per-connection and is drained on disconnect), so instead multiplex the per-zone scoped sessions over a bounded pool of connections.
WorldtreeConngains a ws-parametric API (openScopedSession,closeSessionOn,readTreeOn,readBlobOn) so one physical connection can host many independent per-zone sessions; the existing single-session methods (used by the control connection) are unchanged.WorldtreeZoneSessionhandle owns one zone's scoped session on a shared carrier connection and releases it (close_ro) when the zone'sWorldtreeSourceAccessor— its sole owner — drops.EvalState::acquireWorldtreeZoneSessiondraws a carrier round-robin from a lazily-grown pool bounded by the newtectonix-worldtree-max-connectionssetting (default 64, well under the daemon's 512 cap), and opens the zone's scoped session on it.getZoneStorePathuses it instead of one connection per zone.Per-zone confinement is unchanged: each zone still gets its own
scopedsession (the daemon enforces the cone per session), just hosted on a shared connection. Reads that land on the same carrier serialize on it; cross-zone parallelism now comes from the pool holding several carriers rather than one connection per zone.