Skip to content

CI: run the six deadline-health cells in turn on one runner - #748

Open
rmcdaniel wants to merge 3 commits into
mainfrom
rust-325-ci-codec-one-job
Open

rmcdaniel wants to merge 3 commits into
mainfrom
rust-325-ci-codec-one-job

Conversation

@rmcdaniel

Copy link
Copy Markdown
Member

Review class: R0 (CI efficiency; same checks, same failure semantics).

Why

#746's measured run (40104569657) shows the six codec-deadline-health matrix cells taking 1m03s–1m27s each, while the database parts beside them take 20–31 minutes. Each cell still held a runner. Under org-wide queueing (18 queued jobs observed at 14:33Z), that is five extra runners per run for no wall-time gain.

What

  • One codec-deadline-health job runs the six cells in turn: sqlite, postgres-16, postgres-17, mysql-8.0, mysql-8.4, mariadb-10.11.
  • New scripts/conformance/server-parity/codec-deadline-cells.sh runs each cell under its own scope, $RESOURCE_SCOPE-<database>. A failed cell still lets every later cell run, and the job fails after all of them, as the matrix did with fail-fast: false. Per-cell exit statuses are kept as codec-evidence/<database>.status.
  • The always() step removes every cell's exact owned resources by name.
  • The step summary lists each database's audit, and one artifact keeps all six cells' diagnostics.
  • The job keeps needs: [published-sdk-adapters, codec-native-build], so the aggregate and structure tests are unchanged.

Evidence

  • New tests/Unit/ServerParityCodecDeadlineCellsTest.mjs (3/3) runs the runner with a stub cell and proves:
    • all six cells run once, each with its own scope and output, and leave no state;
    • a failed cell (postgres-16, exit 3) still lets the later cells run, and the runner exits 1 with exactly one ::error::;
    • a missing scope is refused.
  • The Docker isolation policy passes; actionlint 1.7.7 passes. ServerParityAggregateTest and ServerParityPartsTest pass.
  • Expected: about 6–8 minutes for the one job, still well inside the 31-minute critical path, and five fewer runners per run.

Each cell takes about a minute beside database parts that take half an
hour, so the matrix only held five extra runners per run. One job now runs
every cell in its own scope; a failed cell still lets the others run, and
the job fails after all of them, as the matrix did. A shell test pins that.
@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel

Copy link
Copy Markdown
Member Author

Independent CI wrapper review of 0e550a96d6da235076a1422b649abf2447b11d42: one failure-path correction is needed before this consolidation is accepted.

codec-deadline-cells.sh records a failed child and immediately starts the next database. The child codec-operations-cell.sh has an EXIT trap that captures logs; its server/database stop calls occur only on the successful path. The workflow's always() cleanup now runs after the whole six-cell loop. A failed cell can therefore leave its database/server consuming resources while later cells qualify. The previous matrix cleaned each failed cell in its own job.

Please retire each cell's exact resources on both success and failure before the next cell starts, preserving diagnostics, aggregate failure and the final always() cleanup fallback. Remove its state only after the actors are stopped. Add a counterfactual in which a failed child leaves a live-resource marker and the next child refuses overlap; the current tests only cover state directories/status propagation.

Verification: the three existing wrapper tests pass in an isolated Node container. An additional stub-only control-flow counterfactual returns wrapper exit1 while confirming the failed-cell marker remains present when the next cell starts. No native service, compiler, workload or native log/artifact retrieval was involved. This identifies a source failure-path gap; it does not attribute the hosted deadline-health failure to that gap.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant