fix(pacing): bypass informer cache in PullSlots to enforce global cap correctly - #123
Merged
Conversation
…bal cap The pacing engine was using the informer-cache-backed client to list active pull Pods. This caused a TOCTOU race: when two CachedImage reconciles ran back-to-back, the second reconcile could read a stale cache that didn't yet reflect the pod just created by the first reconcile, causing both to launch pods and exceed the maxConcurrentPulls global cap. Fix by switching the Engine's reader to client.Reader (backed by mgr.GetAPIReader()), which reads directly from the API server and always returns an up-to-date view of active pods.
Copilot
AI
changed the title
[WIP] Fix failing GitHub Actions job e2e
fix(pacing): bypass informer cache in PullSlots to enforce global cap correctly
Aug 3, 2026
Breee
marked this pull request as ready for review
August 3, 2026 09:05
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
maxConcurrentPullsglobal cap could be exceeded when two CachedImages sharing a PullPolicy were reconciled back-to-back. The pacing engine read pod counts from the informer cache, which lagged behind reality — so the second reconcile saw 0 active pods and created a pod despite one already existing, violating the cap.Changes
internal/pacing/engine.go: ReplaceEngine.Client client.ClientwithEngine.Reader client.Reader. Pod listing now goes through this reader, which must bypass the informer cache.cmd/main.go: Wiremgr.GetAPIReader()(direct API server reads, no cache) instead ofmgr.GetClient()when constructing the pacing engine.The
client.Readerinterface is a subset ofclient.Client, so no call sites beyond the constructor change.