Config server cache backend - #471
Open
steiler wants to merge 44 commits into
Open
Conversation
Wayfinder-driven design pass for a config-server-backed pkg/cache.Client backend: resolves interface-shape, running-vs-real-intent routing, and config-server-local-API questions, consolidated into ADR pkg/cache/docs/adr/0001-config-server-backed-cache-client.md. Also breaks the ADR down into a numbered implementation ticket index (.scratch/config-server-cache-backend/issues/) for a follow-up implementation effort. Co-authored-by: Cursor <cursoragent@cursor.com>
CacheConfig.validateSetDefaults silently rewrote any Cache.Type other than "remote" back to "local", which would have clobbered the new "config-server" backend type before it ever reached the backend-selection switch. Switch explicitly on known types and reject anything else with a clear error instead of coercing it. Closes ticket 02 of the config-server-backed cache client effort (.scratch/config-server-cache-backend/issues/02-fix-cache-type-validation.md). Co-authored-by: Cursor <cursoragent@cursor.com>
…GetAll Add GetOrphan()/GetSensitivePaths() to ImportConfigAdapter so it is the full intent-shaped adapter surface, implemented from real fields on ProtoTreeImporter and stubbed on Json/XmlTreeImporter (never used for real intents). Widen cache.Client's and CacheClientBound's real-intent Get/GetAll methods to return importer.ImportConfigAdapter instead of the concrete *tree_persist.Intent, so a future config-server-backed cache.Client can satisfy the same surface without a shared persisted representation. LocalCache's behavior is unchanged — it now just wraps its existing proto payload in ProtoTreeImporter before returning it. Ticket 03 of the config-server-backed cache client plan; running still flows through the same Get/Modify methods unchanged (ticket 04 splits it out). Co-authored-by: Cursor <cursoragent@cursor.com>
running (the synced device-state pseudo-intent) is structurally different from a real intent and only shared the generic Intent surface because LocalCache happened to store everything the same way. Give it its own InstanceRunningGet/InstanceRunningModify (cache.Client) and RunningGet/RunningModify (CacheClientBound), so a future config-server-backed Client never has to special-case running inside the real-intent Get/List/Modify/Delete path. replaceIntent and writeBackSyncTree switch to the dedicated methods instead of passing consts.RunningIntentName through IntentGet/ IntentModify. LocalCache implements the new methods as thin passthroughs to its existing disk-backed store, so on-disk behavior for running is unchanged. populateSensitivePathIndex now explicitly excludes running from its IntentGetAll scan, replacing the previous incidental exclusion (running never set SensitivePaths). Co-authored-by: Cursor <cursoragent@cursor.com>
sdc-protos now has ConfigReadService (Get/List) + ConfigEntry/ConfigBlob messages covering the ADR's field-mapping table, at commit bca11ef on sdc-protos' config-server-cache-backend branch. Contract-only, per ticket scope. Co-authored-by: Cursor <cursoragent@cursor.com>
Defines LocalConfigReader (Get-by-name / List-by-target), the Target, Document, and ConfigBlob types per the ADR's field-mapping table, and an in-memory FakeLocalConfigReader seedable with fixtures. Lets the config-server-backed cache.Client (ticket 06) be built and unit-tested now, decoupled from the real cross-repo gRPC transport (ticket 01/07/08). Ticket 05 of docs/adr/0001-config-server-backed-cache-client.md; no existing file touched. Co-authored-by: Cursor <cursoragent@cursor.com>
…fake Builds ConfigServerCache (pkg/cache/configserver.go), the new cache.Client for Cache.Type: config-server, behind ticket 05's LocalConfigReader seam: real-intent reads map onto the seam's Get/List and wrap results as importer.ImportConfigAdapter via a new configserver.NewImportAdapter (pkg/cache/configserver/importer.go, merge.go) per the ADR's field-mapping table; writes on real intents are unconditional no-ops; "running" is served from its own independent in-memory, per-instance store, entirely separate from the seam. Ticket 06 done. Co-authored-by: Cursor <cursoragent@cursor.com>
Implements ticket 08: GRPCConfigReader (pkg/cache/configserver/grpc.go) implements the local-read seam by calling the generated config_read.ConfigReadServiceClient (sdc-protos ticket 01), replacing ticket 05's fake behind ticket 06's backend. Connection settings are caller-supplied (grpc.ClientConnInterface / Dial(address)), never hardcoded. Unit-tested against a new generated mock (mocks/mockconfigread) of the real client. Pins sdc-protos to the sibling local checkout (untagged commit a8f3da0, which adds config_read) until that branch is pushed/tagged upstream. No wiring into Server.createCacheClient/CacheConfig — that's ticket 09. Co-authored-by: Cursor <cursoragent@cursor.com>
Add the case "config-server" branch to createCacheClient, dialing the colocated config-server controller and constructing ConfigServerCache against it. Extend CacheConfig with Namespace and require both Address and Namespace for Cache.Type: config-server; local/remote behavior is unchanged. Closes ticket 09. Co-authored-by: Cursor <cursoragent@cursor.com>
…landed Implemented in the config-server checkout (pkg/sdc/configread); see the ticket's Comments section for what landed. All tickets 01-09 are now done. Co-authored-by: Cursor <cursoragent@cursor.com>
…igAdapter running is domain-distinct from Intent, but Client's read-side return type is a mechanical "ready for Tree.ImportConfig" contract, not an intent-specific one. Move the *tree_persist.Intent -> ImportConfigAdapter wrap into each Client implementation (LocalCache, ConfigServerCache), matching InstanceIntentGet, so transaction_rpc.go no longer needs to know running is proto-backed. Amends ADR 0001, which originally specified the now-reversed *tree_persist.Intent signature. Co-authored-by: Cursor <cursoragent@cursor.com>
Decompose cache.Client and cache.CacheClientBound into IntentReader, IntentWriter, RunningStore, and InstanceLifecycle capability interfaces (recomposed via embedding), regenerate mocks per capability, and narrow read-only consumers (forEachIntent, populateSensitivePathIndex) to the bound reader interface. Make config-server's no-op writes explicit via a standalone noopIntentWriter composed with *ConfigServerCache in Server.createCacheClient, instead of burying no-ops inside ConfigServerCache's own methods. Also extract LocalCache.decodeIntent to dedup repeated decode logic, and relocate config-server's ImportConfigAdapter implementation (documentImporter/mergeConfigBlobs) to pkg/tree/importer/configserver alongside its sibling tree importers. Co-authored-by: Cursor <cursoragent@cursor.com>
steiler
force-pushed
the
config-server-cache-backend
branch
from
August 11, 2026 06:47
a570265 to
ccff783
Compare
…rver backend Adds splitDatastoreName + ErrMalformedDatastoreName so ConfigServerCache can later derive per-call target namespace/name from the datastore name instead of a fixed, deployment-wide namespace (ticket 01). Co-authored-by: Cursor <cursoragent@cursor.com>
target() now decodes each call's cacheInstanceName via splitDatastoreName instead of trusting a single, deployment-wide namespace field, so one data-server process correctly serves datastores across multiple Kubernetes namespaces and config-server's ConfigReadService receives the bare target name. Every target() caller, plus InstanceCreate, propagates/surfaces ErrMalformedDatastoreName on a malformed datastore name. Constructors drop their namespace parameter (ticket 02). Co-authored-by: Cursor <cursoragent@cursor.com>
Namespace was never meaningful once ConfigServerCache derives it per-call from the datastore name (ticket 02); config-server cache type now requires only Address (ticket 03). Co-authored-by: Cursor <cursoragent@cursor.com>
CI can't resolve `replace github.com/sdcio/sdc-protos => ../sdc-protos` since no sibling checkout exists on the runner. Push the config-server-cache-backend branch to sdcio/sdc-protos and pin go.mod to its commit via a pseudo-version instead. Also silence two staticcheck QF1011 findings on intentional interface assertions in new tests, uncovered now that lint can actually resolve the module. Co-authored-by: Cursor <cursoragent@cursor.com>
This was referenced Aug 11, 2026
Open
Config-server stores intents as a single JSON object at the YANG root path, which mergeConfigBlobs rejected during LoadAllButRunningIntents and broke performRevert after sync. Co-authored-by: Cursor <cursoragent@cursor.com>
Document that per-intent SensitivePaths markers are scoped separately from schema-defined YANG sensitivity, which ShouldRedact still applies unconditionally. Co-authored-by: Cursor <cursoragent@cursor.com>
Config-server keys TransactionSet intents as namespace.name, but the config-server cache was returning bare Config names. That mismatch made DeviationWatcher split intent owners incorrectly and panic the controller. Co-authored-by: Cursor <cursoragent@cursor.com>
Second single.yml invocation (cache_type: config-server) lands with the cache backend, not on main ahead of it. Existing integration-tests job is unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
3 tasks
Use the integration-tests feature-branch workflow so cache_type=config-server can include 02-crud and 03-deviations alongside 05-cache-backend. Co-authored-by: Cursor <cursoragent@cursor.com>
IntentModify/IntentDelete now write through to config-server's ConfigSnapshotService synchronously at apply time, replacing the noopIntentWriter composition ADR 0001 used. Last-applied tracking under this backend now matches Cache.Type: local's write-at-apply timing, closing the ghost-intent race where a deleted intent stayed last-applied until the next best-effort snapshot refresh. - ConfigServerCache gains a LocalConfigClient seam (read+write) and implements IntentWriter directly; NewConfigServerClient no longer composes noopIntentWriter. - New tree-to-JSON flattener (document_export.go) converts an applied tree_persist.Intent into the flat Document shape the read path already understands, for GRPCConfigClient.Modify's wire payload. - Fix the log-only IntentDelete failure in lowlevelTransactionSet: a failed delete-write now hard-fails the transaction, matching IntentModify's existing behavior (ADR 0003's asymmetry fix). - GRPCConfigReader/FakeLocalConfigReader renamed to GRPCConfigClient/FakeLocalConfigClient; mocks/mockconfigread regenerated against the renamed ConfigSnapshotServiceClient. - Tests: seam-level modify/delete round trips, malformed-datastore-name coverage for the new write callers, and datastore-level regression tests for delete-failure hard-fail and rollback-restores-a-deleted- intent. - Commit the previously-drafted ADR 0003 and CONTEXT.md Last-applied term update.
Spec review of the real-IntentWriter commit found the delete-no-rehydration checklist item was backed by a weaker test than described, and the rollback-restore test only checked the intent's name, not its content. - Add TestConfigServerBackend_DeleteApply_NoRehydration: wires a real *cache.ConfigServerCache over configserver.FakeLocalConfigClient (not a generic mock) through two TransactionSet calls and asserts the exact regression signature named in the ticket — LoadAllButRunningIntents does not rehydrate a deleted intent. - Strengthen TestTransactionRollback_RestoresDeletedIntent to assert the restored leaf content, not just the intent name. - Clarify ConfigServerCache's ignoreNonExisting doc comment: unused by today's LocalConfigWriter implementations, not an inherent design guarantee. - Note in GRPCConfigClient.Modify why the wire payload is plaintext (ADR 0003's "encrypted payload" describes the persisted record; config-server encrypts on receipt, same as northbound SensitiveConfig). Standards review separately flagged Document<->ConfigEntry mapping duplicated across grpc.go/document_export.go; added configEntryFromDocument as the Modify-side counterpart to the existing documentFromEntry to remove the wire-mapping half of that duplication.
…d transaction intents (ticket 05) lowlevelTransactionSet's validation abort check (ValidationResults.HasErrors) was unscoped: a validation error owned by an intent rehydrated only via LoadAllButRunningIntents (a ghost left behind by stale last-applied state) hard-failed the whole transaction, even when the RPC's own intents (e.g. an unrelated delete) had no errors of their own. This is the safety-net ticket (05) on top of ticket 03's apply-time last-applied fix, guarding the 02-CRUD-teardown-hang CI signature described in the spec. Compute the set of intent names loaded only via LoadAllButRunningIntents (not part of transaction.GetNewIntents()) and use the new ValidationResults.HasErrorsExcludingOwners to ignore their errors when deciding whether to abort. Errors owned by an RPC-owned intent still hard-fail exactly as before, and excluded ghost errors are still reported in the response for observability.
golangci-lint-action defaults to verify: true, which runs 'golangci-lint config verify' and fetches the JSON schema for .golangci.yml from https://golangci-lint.run over the network on every run. That remote fetch is flaky in CI and has caused non-deterministic lint-job failures (context deadline exceeded) unrelated to the actual code or lint config. golangci-lint run still validates the config at runtime, so disable the network-dependent early check.
This was referenced Sep 4, 2026
Closed
Merged
…ErrRunningNotFound
InstanceCreate was seeding c.running[name] with nil, so the very first
InstanceRunningGet call (before any writeBackSyncTree had run) returned
ErrRunningNotFound, causing replaceIntent to hard-fail on first-ever
replace transactions.
Fix: seed with &tree_persist.Intent{} so InstanceRunningGet always
returns an empty-but-valid ImportConfigAdapter on first call.
ErrRunningNotFound and the nil-guard in InstanceRunningGet are removed —
the only reachable nil case was the seeding bug this commit fixes.
… constant Ghost-intent exclusion (layer 1 + layer 2) was conceptually wrong: Layer 1 — LoadAll-only intent exclusion: if the combined YANG validation of the highest-precedence intent tree fails, the RPC must not apply — regardless of which intent's content caused the violation. Excluding bystander errors could silently suppress a real cross-intent constraint failure (e.g. intent2 deletes a leaf intent1 has a leafref to; error is attributed to intent1, layer 1 suppresses it, invalid config is pushed). The root fix is eliminating ghosts at the source (ticket 03 + config-server APIReader read-after-write consistency), not masking errors in the abort path. Layer 2 — UnknownOwner suppression: 'unknown' is an error-attribution sentinel for validators that cannot identify a single responsible intent (mandatory child, must on containers). It is never stored in the cache and never returned by LoadAllButRunningIntents, so adding it to the LoadAll-only exclusion set was mixing up two unrelated concepts and could suppress genuine mandatory or must-statement failures. Changes: - Remove loadAllOnlyIntentNames function - Remove the ghost-exclusion block in lowlevelTransactionSet; replace with plain validationResult.HasErrors() - Remove HasErrorsExcludingOwners from ValidationResults; inline loop back into HasErrors() - Replace hardcoded "unknown" string literals in all four validators with the types.UnknownOwner constant (no behaviour change — purely removes magic strings) - Remove TestTransactionSet_LoadAllOnlyValidationError_DoesNotBlockUnrelatedIntent (tested the now-deleted exclusion behaviour) - Replace TestValidationResults_HasErrorsExcludingOwners with TestValidationResults_HasErrors; add explicit case asserting UnknownOwner errors block the transaction rather than being special-cased
Only used by the ghost-intent exclusion test removed in the prior commit.
When a gNMI subscription delivers a JSON-IETF blob at the container level, ExpandUpdate's top-level IsState() guard is never reached for the individual fields inside the blob — those are expanded by ExpandContainerValue, which had no state-leaf filter. This caused read-only operational leaves (e.g. afi-safi/active-routes on SRL) to be written into the sync tree under the running owner. When an intent owning the same container was later deleted, the mandatory-field validator saw the container kept alive by those state leaves and fired, blocking the delete transaction for the full retry window (15 min timeout in CI). Fix: add IsState() checks in ExpandContainerValue for all three branch types — LeafSchema fields, LeafListSchema, and child containers. For fields the redundant duplicate GetSchemaSdcpbPath call (one for state check, one for value conversion) is collapsed into a single call. Regression test: TestExpandContainerValueSkipsStateLeaves in pkg/utils/converter_test.go.
Cache.Type: local is a standalone deployment with no config-server; it was never designed to run in a cluster alongside one, and doing so produced confusing failures unrelated to the local-cache code path (see .scratch/config-server-cache-backend-ci-fix/spec.md). Only the integration-tests (config-server cache backend) job remains as the regression gate for this workflow. Ticket: 02-integration-tests-remove-local-cache-job
ExpandContainerValue already stripped a JSON-IETF module prefix (e.g. "srl_nokia-common:ipv4-unicast") from an identityref list key when that key arrived inside the JSON value, but not when the same key arrived already embedded in the gNMI path (keysInPath) — the common case for GET/Subscribe sync responses whose list keys are delivered as path elements rather than nested in the body. This let the same list entry (e.g. bgp/afi-safi[afi-safi-name=...]) resolve to two different tree nodes depending on whether it was populated by device sync (module-prefixed, via path) or by an intent (unprefixed, via JSON): a running-only "ghost" node that was never tied to the intent's lifecycle, and the intent-owned node that was. Deleting the intent removed only the intent-owned node. The ghost node (kept alive by ordinary config leaves like admin-state, not just state leaves already handled by 767b535) made the ancestor container's RemainsToExist() stay true forever, so mandatory-field validation kept firing on the now-orphaned mandatory children (autonomous-system, router-id) that only the deleted intent had provided — permanently failing the delete transaction and hanging the ConfigSet teardown for the full 15-minute CI retry window (see run https://github.com/sdcio/data-server/actions/runs/34466219951). Fix: strip the identityref module prefix for path-sourced keys the same way as JSON-value-sourced keys, and always write the resulting keySet back onto the path's last element in one place (previously that write only happened when the key came from the JSON value, which was harmless before but would have left path-sourced identityref keys unstripped even with the fix above). Regression test: TestExpandContainerValueStripsIdentityrefModulePrefixFromPathKey in pkg/utils/converter_test.go.
JsonTreeImporterElement.GetKeyValue previously returned
fmt.Sprintf("%v", data) verbatim, so JSON_IETF-encoded identityref
values (e.g. "srl_nokia-common:ipv4-unicast") kept their module
prefix while plain-JSON/proto-imported values for the same identity
stored the bare name ("ipv4-unicast"). The two forms were treated as
different tree keys, splitting one logical identityref-keyed list
entry (e.g. bgp/afi-safi) into two sibling tree nodes.
In production this let a running-only remnant of an afi-safi entry
survive a full intent delete, keeping the parent bgp container
'existing' just long enough for mandatory-child validation to run
against it and reject the transaction with a spurious
'mandatory child [autonomous-system] does not exist' error, even
though the intent (and its mandatory leaves) was being deleted in
full.
Fix GetKeyValue to delegate to GetTVValue + tv.ToString(), mirroring
the XML and proto importers, which normalizes identityref values to
the bare identity name regardless of encoding. Falls back to
fmt.Sprintf when slt is nil, since GetTVValue's underlying
ConvertJsonValueToTv would panic on a nil SchemaLeafType.
Also documents the GetKeyValue contract on ImportConfigAdapterElement:
implementations must return the bare identity name for identityref
types.
Implements issue 01 of the must-leafref-identityref-normalization PRD.
…a race TreeExport and other read paths were reading LeafEntry.Delete/IsNew/IsUpdated/ IsExplicitDelete fields directly, racing with MarkDelete()/MarkUpdate()/ MarkExpliciteDelete() which mutate these fields under LeafEntry.mu (e.g. from the owner-delete-marker pool task running concurrently with TreeExport during writeBackSyncTree/LoadAllButRunningIntents). Switch all direct field reads to the existing lock-guarded accessors (GetDeleteFlag/GetNewFlag/GetUpdateFlag/GetExplicitDeleteFlag), and add a new LeafEntry.ResetFlags helper so LeafVariants.ResetFlags no longer mutates the flags directly while holding only the sibling lesMutex.
CI run 34482168553 (job 102888410883, this PR) failed the config-server cache backend job's 03-deviations suite: 5 "Partially Revert Deviations by Filter Path" tests hit code=Aborted "datastore is locked, other action is ongoing" from kubectl sdc deviation --revert --filter-path. Root cause: legitimate lock contention on data-server's per-datastore TryLock (predates this branch), hit by a code path (config-server's ClearDeviations) that had no retry, unlike the TargetConfigController reconcile loop which already tolerates the same condition. Fixed in the two sibling repos: - sdcio/config-server: executeClearDeviationTx now retries on dsclient.IsRecoverableError (codes.Aborted/ResourceExhausted). - sdcio/integration-tests: completed c69c07e's fix by wrapping the two partial-revert keywords it missed. No data-server code change needed -- this ticket documents the investigation and cross-repo fix for future reference.
steiler
added a commit
to sdcio/config-server
that referenced
this pull request
Sep 11, 2026
…d errors kubectl sdc deviation --revert (including --filter-path partial reverts) reaches executeClearDeviationTx via the cleardeviation subresource, which called TransactionSet/TransactionConfirm once and returned whatever error it got straight through. Under concurrent load (e.g. TargetConfigController reconciling the same datastore) that error is frequently codes.Aborted from data-server's dmutex TryLock losing a race — a condition the reconciler's own TransactionSet calls already tolerate via isRecoverableGRPCError + 500ms RequeueAfter (pkg/reconcilers/targetconfig/reconciler.go), but which ClearDeviations had no equivalent for. - Move isRecoverableGRPCError's classification to pkg/sdc/dataserver/client.IsRecoverableError, the lowest-layer package both apis/config and pkg/sdc/target/manager already depend on (avoids an apis/config <-> pkg/sdc/target/manager import cycle). - Wrap executeClearDeviationTx's TransactionSet/TransactionConfirm calls in retryOnRecoverable: up to 5 attempts, 500ms fixed backoff, matching the reconciler's own RequeueAfter for the identical error condition. codes.Aborted is nothing having been applied server-side (TryLock never acquired), so replaying the same request is always safe. Evidence: data-server CI run 34482168553 (sdcio/data-server#471), 03-deviations suite, "Partially Revert Deviations by Filter Path" group. See data-server .scratch/deviation-partial-revert-lock-contention/spec.md.
steiler
added a commit
to sdcio/integration-tests
that referenced
this pull request
Sep 11, 2026
Completes the fix started in c69c07e ("retry Delete Deviation on transient datastore-locked errors"). That commit's own ticket (.scratch/last-applied-snapshot-write-at-apply/issues/08) named two originally-failing groups -- "Reject Deviations" and "Partially Revert Deviations by Filter Path" -- but only wrapped the keyword behind the first (Delete Deviation). The second group's two keywords were missed and remained unwrapped, causing 5 test failures in CI run 34482168553 (sdcio/data-server#471, config-server-cache-backend job, 03-deviations suite): - Partial Revert Deviations For Intent by Interface (22-srl-nonrevertive.robot) -> extracts Run Partial Revert Deviations by Interface. - Partial Revert Deviations For Intent by Admin State (21-sros-nonrevertive.robot) -> extracts Run Partial Revert Deviations by Admin State. Both now wrap the kubectl sdc deviation --revert --filter-path call in Wait Until Keyword Succeeds ${eventual_timeout} ${retry}, same shape as c69c07e's Run Deviation Revert extraction. This is a CI-level safety net alongside the actual protocol-level fix in sdcio/config-server (executeClearDeviationTx now retries recoverable gRPC errors server-side). See data-server .scratch/deviation-partial-revert-lock-contention/spec.md.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! 🚀 New features to boost your workflow:
|
steiler
force-pushed
the
config-server-cache-backend
branch
from
September 14, 2026 13:37
3916037 to
15b7efd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cache.Clientimplementation that reads last-applied config via a newconfig_read.ConfigReadServicegRPC client instead of the local/dragonfly cache.Pairs-with: sdcio/config-server#476
Pairs-with: sdcio/integration-tests#115
Note on integration coverage
The
integration-testsjob now runstests/05-cache-backend/(CRUD round-trip + restart-recovery) against aCache.Type: config-serverdeployment viasdcio/integration-testsPR #115, which extendssingle.ymlwithcache_type/suites_to_runinputs and adds the suite itself.Pairs-with: sdcio/integration-tests#115supersedes the previous#113reference: #115's branch bundles #113'sconfig-keyringSecret fix directly (a hard, cache-backend-independent prerequisite -- without itdata-server-controllerhangs inContainerCreatingfor any deploy) alongside the actual cache-backend suite/workflow changes, so a single paired ref now covers both concerns.