Skip to content

feat(project): secrets, health and release in cbox.yaml - #1

Closed
sylvesterdamgaard wants to merge 1 commit into
mainfrom
feat/secrets-health-release
Closed

sylvesterdamgaard wants to merge 1 commit into
mainfrom
feat/secrets-health-release

Conversation

@sylvesterdamgaard

Copy link
Copy Markdown
Contributor

Do not merge before cboxdk/platform#1 is merged and tagged v0.12.0. composer.json currently points at that PR branch (dev-feat/secrets-health-release as 0.12.0 + a vcs repository) so this branch's CI can run. Before merging: set "cboxdk/platform": "^0.12.0", drop the repositories entry, composer update cboxdk/platform, composer sbom, commit.

Three keys cbox-id's production cbox.yaml needs, which the reader refused and the compiler could not express.

secrets

secrets:
  APP_KEY: { secret: cbox-id-app, key: APP_KEY }

Env var from a Secret that already exists in the project namespace. Compiled (platform SecretReference) to valueFrom.secretKeyRef on every container: web, every process, cbox run, the release. Never a value, never optional. Refused here: invalid variable / Secret / key names, settings other than secret/key, a variable also under env, also bound by a resource, or equal to the url variable. A missing Secret leaves the pod in CreateContainerConfigError; the deploy's blocked-pod message now names secrets as the likely cause.

health

health: { path: /health/ready, liveness: /up, token: HEALTH_TOKEN }

Web Deployment only. startup (liveness path, 2s × 150 = 5 min) → liveness (10s, 3 misses) and readiness (10s, 3 misses). token must name a secrets entry (refused under env or missing): Kubernetes httpGet probes take static headers only, so readiness with a token is an exec probe — printf 'Authorization: Bearer %s\n' "${HEALTH_TOKEN}" | curl -fsS -o /dev/null --max-time 4 -H @- 'http://127.0.0.1:PORT/path'. The value never enters the pod spec or argv. Needs curl in the image (every Cbox base image has it).

release

release: ["php", "artisan", "migrate", "--force"]

ProjectDeployer now sequences a deploy with a release (new ReleaseRunner):

  1. apply the compiled set minus the project's Deployment/ScaledObject/HTTPScaledObject — the old release keeps serving
  2. wait for every CNPG Cluster / StatefulSet to be ready (a first deploy's Postgres is seconds old and its app Secret does not exist yet)
  3. apply the release Job (same container as the new web pods) and poll its Complete/Failed condition. A Complete Job of the same name means this release already ran → skip. A Failed one is deleted and re-run. A pod stuck in CreateContainerConfigError/ImagePullBackOff for 5 polls fails the deploy and deletes the Job instead of waiting out the 1h deadline
  4. on failure: return a failed outcome with the Job's last 30 log lines; no workload is applied
  5. on success: apply the full set

Dry run: the Job is included in the single admission-only apply, nothing waits. No release → one apply, exactly as before.

cbox push does not carry the three keys (Cortex's /v1 has no field for them); omitted() now says so and what to do in Cortex.

Tests

tests/SecretsHealthReleaseTest.php — 38 tests: reader acceptance and every refusal, the compiled secretKeyRefs and probes, push omissions, and the deploy order (release before workloads, failure rolls nothing, stuck pod, already-ran, retry-after-failure, timeout, dry run, no-release unchanged). Also checked by hand: cbox-id's cbox.yaml from docs/k8s-deployment reads cleanly and compiles.

Gates locally: pint, phpstan (max), pest 253 passed, license-check. sbom.json regenerated (only the platform entry moved, and will move again with the constraint swap).

Three keys a production manifest needs, read, compiled and deployed.

secrets: `VAR: { secret, key }` — an environment variable from a Secret
that already exists on the platform. Validated here (variable, Secret and
key names; no `optional`; no collision with env, a resource binding or
the url variable) and compiled to a secretKeyRef on every container by
cboxdk/platform's SecretReference. Never a value; a missing Secret stops
the pod with CreateContainerConfigError, which deploy now explains.

health: `{ path, liveness, token }` — the web process's probes. The token
must be a `secrets` entry: Kubernetes cannot put a Secret into a probe
header, so the platform compiles an exec probe that reads it from the
container's environment.

release: a command run to completion before anything rolls. The deploy
applies everything but the workloads, waits for the databases, runs the
release Job, and only on success applies the Deployments. A failure stops
the deploy with the release's output and leaves the running release
untouched; a pod that cannot start is reported within seconds rather than
after the Job's hour-long deadline. The Job is named for what it runs, so
the same release is not migrated twice and a failed one is retried.

Needs cboxdk/platform 0.12.0 (SecretReference, HealthCheck,
ReleaseJobCompiler). Until that is tagged, composer.json points at the
platform PR branch aliased as 0.12.0.
@sylvesterdamgaard

Copy link
Copy Markdown
Contributor Author

Closed: Cbox ID's production deployment is defined in cbox-infra (infra/kubernetes/cbox-id), not cbox.yaml, so the platform doesn't need these keys for it. Branch kept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant