Skip to content

Move release metadata from BigQuery to PostgreSQL#3674

Closed
mstaeble wants to merge 1 commit into
openshift:mainfrom
mstaeble:TRT-2734-releases-to-postgres
Closed

Move release metadata from BigQuery to PostgreSQL#3674
mstaeble wants to merge 1 commit into
openshift:mainfrom
mstaeble:TRT-2734-releases-to-postgres

Conversation

@mstaeble

@mstaeble mstaeble commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Creates a release_definitions table in PostgreSQL to store release metadata (GA dates, development start dates, previous release, capabilities, product, status) previously only available in BigQuery
  • During the data load cycle, releases fetched from BQ are synced to PostgreSQL via a new release-definitions loader (--loader release-definitions)
  • All release data consumers (/api/releases, component readiness, metrics, triage, job run risk analysis, MCP tools, Jira automation) now read from PostgreSQL
  • Removes QueryReleases and QueryReleaseDates from the DataProvider interface — release data is no longer a provider concern
  • Removes the hardcoded releaseMetadata map from the PostgreSQL data provider
  • Removes GetReleasesFromBigQuery, GetReleases, transformRelease, and the BQ releasedates.go file
  • All internal consumers use models.ReleaseDefinition directly instead of the v1.Release intermediary
  • Capability constants defined in pkg/db/models alongside ReleaseDefinition

Test plan

  • go build ./... passes
  • go vet ./... passes
  • make lint passes
  • go test ./pkg/... ./cmd/... passes
  • sippy serve with seed data serves /api/releases from PostgreSQL with correct GA dates, capabilities, and previous release chain

Staging verification

Deployed the branch to sippy-staging and ran the release-definitions loader as a one-off job in the sippy namespace:

sippy load --loader release-definitions --init-database

This synced 36 release definitions from BigQuery to the staging PostgreSQL database (all OCP releases 3.11 through 5.0, OKD, ROSA, ARO, HyperShift, and CAPI entries).

Verified the following endpoints on sippy-staging.dptools.openshift.org:

Endpoint Result
GET /api/releases 36 releases with correct GA dates, capabilities, previous release chain, product, and status
GET /api/releases/health?release=4.22 6 payload health reports returned correctly
GET /api/component_readiness?baseRelease=4.21&sampleRelease=4.22 (with full params) 72 components, 38 columns
GET /api/component_readiness/regressions?release=4.22 4,545 regressions returned

All release data served from PostgreSQL with no BigQuery calls.

Ref: TRT-2734

@coderabbitai ignore

🤖 Generated with Claude Code

[WIP] — do not merge until fully reviewed

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 23, 2026
@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mstaeble
Once this PR has been reviewed and has the lgtm label, please assign stbenjam for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mstaeble mstaeble force-pushed the TRT-2734-releases-to-postgres branch from bdc3fe8 to 8ec21c3 Compare June 23, 2026 22:10
@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 23, 2026
@mstaeble mstaeble force-pushed the TRT-2734-releases-to-postgres branch 2 times, most recently from 3c2fb56 to 35f162a Compare June 24, 2026 00:49
@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 24, 2026
@mstaeble mstaeble force-pushed the TRT-2734-releases-to-postgres branch from 35f162a to 3c097a9 Compare June 24, 2026 12:58
Create a release_definitions table to store release metadata (GA dates,
development start dates, previous release, capabilities, etc.) that was
previously only available in BigQuery. During the data load cycle,
releases fetched from BQ are converted directly from ReleaseRow to
ReleaseDefinition and synced to PostgreSQL.

All release data consumers now read from PostgreSQL via GetReleasesFromDB
and GetReleaseDatesFromDB. The BigQuery release functions
(GetReleasesFromBigQuery, GetReleases, QueryReleaseDates) are removed,
along with the hardcoded releaseMetadata map from the PostgreSQL data
provider and the QueryReleases/QueryReleaseDates methods from the
DataProvider interface.

Internally, all consumers use models.ReleaseDefinition directly instead
of converting through the v1.Release intermediary. Capability constants
are defined in pkg/db/models alongside the ReleaseDefinition type.

Ref: TRT-2734

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mstaeble mstaeble force-pushed the TRT-2734-releases-to-postgres branch from 3c097a9 to f432180 Compare June 24, 2026 13:57
@mstaeble

Copy link
Copy Markdown
Contributor Author

Superseded by #3679 (minimal version). The v1.Release → ReleaseDefinition migration will be a follow-up PR.

@mstaeble mstaeble closed this Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant