Problem
When Zoo needs model metadata for a provider, the request goes through the model cache and out to the provider's catalog endpoint over plain HTTP. Nothing about that outbound request is cancellable: a caller that gives up (a timeout, a cancelled task, a closed preview) cannot stop the HTTP request, and the cache keeps the shared in-flight entry alive until the request settles. The fetcher layer under src/api/providers/fetchers/ accepts no cancellation input.
Observed in version v3.82.1
Evidence
- The catalog refresh is a shared single-flight.
inFlightRefresh is a module-level map keyed per provider cache key (src/api/providers/fetchers/modelCache.ts:45), and dedupedFetch deletes the entry only in a .finally() on the fetch promise (src/api/providers/fetchers/modelCache.ts:363-380). A waiter that stops waiting does not affect the entry or the request; only settle removes it.
- The OpenRouter catalog fetch is a bare
axios.get with no request timeout and no signal (src/api/providers/fetchers/openrouter.ts:102). Against a hung or slow endpoint the request has no bound.
- No caller-supplied cancellation reaches the catalog fetch path. Neither the model-cache entry points nor the OpenRouter fetcher accepts a signal, so whatever cancellation a caller applies at its own layer, the shared cached fetch and the provider HTTP call keep running. (Some other providers' fetchers build their own timeout controllers; none forwards a caller's cancellation.)
Consequence
A hung or slow catalog endpoint leaves the shared in-flight entry pending until the request settles. Callers that stop waiting leave the HTTP request running; repeated triggers against a misconfigured endpoint accumulate stale fetches behind the single-flight map. Previews can advertise stale model capabilities because a superseded fetch is still the one in flight. A timed-out catalog fetch can outlive every caller that wanted it.
Proposed approach
Only the fetcher layer: src/api/providers/fetchers/. Thread the caller's cancellation through the model-cache single-flight down to the fetchers' HTTP calls (including the OpenRouter axios.get), give the catalog request a request timeout, and release the in-flight entry when an aborted request settles. Where multiple waiters share one fetch, reference-count it so the underlying request aborts when the last waiter cancels. Caller-side bounded waits and cancellation checks are outside this issue.
Acceptance criteria
- A waiter that aborts detaches from the shared fetch; when the last waiter is gone, the underlying request is aborted at the network level.
- The in-flight entry is released on abort, not only on settle.
- A timed-out or superseded catalog fetch cannot outlive its callers.
- Tests hold the HTTP layer pending, abort the caller, and assert the request was aborted and the entry released.
Problem
When Zoo needs model metadata for a provider, the request goes through the model cache and out to the provider's catalog endpoint over plain HTTP. Nothing about that outbound request is cancellable: a caller that gives up (a timeout, a cancelled task, a closed preview) cannot stop the HTTP request, and the cache keeps the shared in-flight entry alive until the request settles. The fetcher layer under
src/api/providers/fetchers/accepts no cancellation input.Observed in version v3.82.1
Evidence
inFlightRefreshis a module-level map keyed per provider cache key (src/api/providers/fetchers/modelCache.ts:45), anddedupedFetchdeletes the entry only in a.finally()on the fetch promise (src/api/providers/fetchers/modelCache.ts:363-380). A waiter that stops waiting does not affect the entry or the request; only settle removes it.axios.getwith no request timeout and no signal (src/api/providers/fetchers/openrouter.ts:102). Against a hung or slow endpoint the request has no bound.Consequence
A hung or slow catalog endpoint leaves the shared in-flight entry pending until the request settles. Callers that stop waiting leave the HTTP request running; repeated triggers against a misconfigured endpoint accumulate stale fetches behind the single-flight map. Previews can advertise stale model capabilities because a superseded fetch is still the one in flight. A timed-out catalog fetch can outlive every caller that wanted it.
Proposed approach
Only the fetcher layer:
src/api/providers/fetchers/. Thread the caller's cancellation through the model-cache single-flight down to the fetchers' HTTP calls (including the OpenRouteraxios.get), give the catalog request a request timeout, and release the in-flight entry when an aborted request settles. Where multiple waiters share one fetch, reference-count it so the underlying request aborts when the last waiter cancels. Caller-side bounded waits and cancellation checks are outside this issue.Acceptance criteria