Repository navigation
feat(service): secrets by reference, web probes, and a release job - #1
Closed
sylvesterdamgaard wants to merge 1 commit into
Closed
sylvesterdamgaard wants to merge 1 commit into
sylvesterdamgaard wants to merge 1 commit into
Conversation
Three things a production service needs that a spec could not say. Secrets by reference. ServiceSpec::$secretRefs (SecretReference: variable, secret, key) compiles to valueFrom.secretKeyRef on every container of the service, at a Secret the operator created. Unlike $envSecret, the value never passes through the spec or anything compiled. Never optional: a missing Secret or key holds the pod in CreateContainerConfigError, which is the right failure. Names Kubernetes would refuse, a variable claimed twice, and a reference to a Secret this compiler writes are refused here. Probes. ServiceSpec::$health (HealthCheck) compiles startup + liveness (httpGet on the liveness path) and readiness for the web Deployment only. A readiness token compiles to an exec probe that reads the variable from the container's own environment and pipes the header to curl -H @-: an httpGet probe can only carry static headers, which would put the token in the pod spec. The release. ServiceSpec::$release and ReleaseJobCompiler compile the service's release command as a run Job named for its pod template's hash, kept OUT of the service's manifest set so an applier can run it to completion before the Deployments roll. The sequence is documented in docs/core-concepts/secrets-probes-releases.md. Existing goldens are byte-for-byte unchanged; service-production is new.
Contributor
Author
|
Closed: Cbox ID's production deployment is defined in cbox-infra (infra/kubernetes/cbox-id), not cbox.yaml, so the platform doesn't need these keys for it. Branch kept. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three things a production service (cbox-id's
cbox.yaml) needs that aServiceSpeccould not say.Secrets by reference —
ServiceSpec::$secretRefs, a list ofSecretReference(variable, secret, key). Compiles tovalueFrom.secretKeyRefon every container of the service: web, every process, runs and the release (they all derive from the web container). The Secret is the operator's; nothing here creates, updates or prunes it, and its value is never in the spec or anything compiled. Neveroptional— a missing Secret/key leaves the pod inCreateContainerConfigError. Refused at compile time: invalid variable/Secret/key names, a variable claimed by a reference and byenv/envSecret/a binding/another reference, and a reference to<service>-envor<service>-registry.Probes —
ServiceSpec::$health, aHealthCheck(readinessPath, livenessPath, tokenVariable). Web Deployment only:startupProbe: httpGet liveness path, every 2s, 150 failures (5 min to boot)livenessProbe: httpGet liveness path, every 10s, timeout 3s, 3 failuresreadinessProbe: httpGet readiness path, or with a token anexecprobe:printf 'Authorization: Bearer %s\n' "${VAR}" | curl -fsS -o /dev/null --max-time 4 -H @- 'http://127.0.0.1:PORT/path'. An httpGet probe can only send static headers, so the token would otherwise be in the pod spec. Header via stdin keeps it out of argv. Needssh+curlin the image (every Cbox base image has both). Refused when the token variable is not set by the spec, and on the CRIU scale-to-zero tier.Process Deployments and run Jobs now also strip
startupProbe.The release —
ServiceSpec::$release(argv) +ReleaseJobCompiler(Contracts\ReleaseCompiler). A run Job named<service>-release-<hash of its pod template>: same release twice = same Job (no double migration), new image/env/command = new Job (Job templates are immutable). Empty while suspended. Kept out of the service's manifest set on purpose, so the applier can sequence: apply the set without Deployments/scalers → run the Job to completion → on failure stop, on success apply the full set. Documented indocs/core-concepts/secrets-probes-releases.md.Golden files
Existing goldens are byte-for-byte unchanged (no new field is set in them). New:
tests/Golden/service-production.golden.yaml— a cbox-id-shaped service (2 replicas, queue + scheduler, a Postgres binding, two Secret references, the token readiness probe) followed by its release Job. What it records that is new: thestartupProbe/livenessProbe/readinessProbeon the web container only, thesecretKeyRefentries atcbox-id-appon all three Deployments, and thecbox-id-release-<hash>Job. I could not run kubeconform locally (no image cached and I did not pull one); the CI schema step coverstests/Golden/*.yaml, so it will validate the new file.Gates
pint --test,phpstan(max),pest(308 passed),license-check,audit— all green locally.Consumers: cbox-engine (
ProjectManifestReader→secrets/health/releaseincbox.yaml, and the deploy sequencing) follows in a separate PR that needs this released as 0.12.0.