Background
While investigating CI speed in Postgres-Extensions/cat_tools, pulled real job/step-level timing from several recent ci.yml runs (gh api .../actions/runs/<id>/jobs, per-step timestamps, not estimates). Findings specific to that repo aren't relevant here, but one pattern is generic to any pgxntool-based repo using the pgxn/pgxn-tools container across a PG-version matrix:
- "Initialize containers" (pulling + booting
pgxn/pgxn-tools) costs roughly 20-30s and recurs on essentially every container-based CI job -- in cat_tools's matrix that's ~30 jobs per run.
- Several jobs additionally
apt-get install build dependencies (e.g. flex bison libkrb5-dev for building pg_tle from source) on top of that.
- None of this sits on the critical path in cat_tools's case (its matrix runs in parallel, and the true bottleneck there is a single long sequential job, unrelated to this) -- so this is a COST optimization (aggregate billable compute-minutes), not a latency one. Rough estimate for cat_tools alone: perhaps 3-7 minutes of aggregate compute-minutes per CI run.
Proposal
A pre-baked container image, built on top of (or alongside) pgxn/pgxn-tools, with common build-time dependencies already installed, so individual jobs across pgxntool-based repos skip re-installing them every single run. Candidates to pre-install, based on what's currently installed ad hoc in cat_tools's CI: rsync, postgresql-server-dev-* for the supported PG major range, and pg_tle's own build dependencies (flex, bison, libkrb5-dev) -- likely more once other pgxntool-based repos' CI is surveyed the same way.
This is deliberately scoped at the pgxntool level rather than per-repo: the value is in every repo built on pgxntool benefiting from one shared image, not in cat_tools maintaining its own.
Not in scope for this issue
- Consolidating a PG-version test matrix into one container instead of one-container-per-version -- investigated separately for cat_tools and found to be a wash-to-negative on wall-clock (parallel matrix already hides the per-job container-boot cost from latency; consolidating would trade a modest compute-cost saving for a slower critical path). That's a per-repo CI-structure decision, not a pgxntool-level one, and isn't what this issue is proposing.
- Actually building/maintaining the image -- this issue is to record the idea and the rough cost justification; scoping the image contents and where it'd be hosted/published is follow-up work.
Background
While investigating CI speed in
Postgres-Extensions/cat_tools, pulled real job/step-level timing from several recentci.ymlruns (gh api .../actions/runs/<id>/jobs, per-step timestamps, not estimates). Findings specific to that repo aren't relevant here, but one pattern is generic to any pgxntool-based repo using thepgxn/pgxn-toolscontainer across a PG-version matrix:pgxn/pgxn-tools) costs roughly 20-30s and recurs on essentially every container-based CI job -- in cat_tools's matrix that's ~30 jobs per run.apt-get installbuild dependencies (e.g.flex bison libkrb5-devfor buildingpg_tlefrom source) on top of that.Proposal
A pre-baked container image, built on top of (or alongside)
pgxn/pgxn-tools, with common build-time dependencies already installed, so individual jobs across pgxntool-based repos skip re-installing them every single run. Candidates to pre-install, based on what's currently installed ad hoc in cat_tools's CI:rsync,postgresql-server-dev-*for the supported PG major range, and pg_tle's own build dependencies (flex,bison,libkrb5-dev) -- likely more once other pgxntool-based repos' CI is surveyed the same way.This is deliberately scoped at the pgxntool level rather than per-repo: the value is in every repo built on pgxntool benefiting from one shared image, not in cat_tools maintaining its own.
Not in scope for this issue