Skip to content

Align push-to-stage PCC with per-OCP approach from multi-push-to-stage#20190

Open
crackcodecamp wants to merge 1 commit into
mainfrom
fix/push-to-stage-per-ocp-pcc
Open

Align push-to-stage PCC with per-OCP approach from multi-push-to-stage#20190
crackcodecamp wants to merge 1 commit into
mainfrom
fix/push-to-stage-per-ocp-pcc

Conversation

@crackcodecamp
Copy link
Copy Markdown
Contributor

@crackcodecamp crackcodecamp commented Apr 21, 2026

Problem

The single-branch push-to-stage.yaml workflow generates PCC (Pre-Computed Catalog) from a fixed v4.17 operator index. Since RHOAI 3.x only targets v4.19+, the v4.17 index has no 3.x entries. When this workflow is used for a 2.x release (e.g., 2.25.5), the output catalog for overlapping OCP versions (v4.19–v4.21) is missing all 3.x bundles and channels — including already-shipped versions like 3.3.2.

This caused the catalog override incident with 2.25.3/3.4EA1 where pushing 2.25.3 wiped 3.x entries from the production operator index.

Root Cause

push-to-stage.yaml line 191:

CATALOG_GENERATION_REF_OCP_VERSION=v4.17  # Fixed — no 3.x entries

While multi-push-to-stage.yaml already generates per-OCP PCC from each version's actual index:

opm migrate registry.redhat.io/redhat/redhat-operator-index:${OCP_VERSION}

Changes

Three targeted changes to align push-to-stage.yaml with the working multi-push-to-stage.yaml:

  1. Sparse checkout — add config/ (needed for global OCP version list)
  2. PCC Regeneration — replace single v4.17 migration with per-OCP loop from config/config.yaml, identical to multi-push-to-stage.yaml lines 334–363
  3. Push To Stage step — use catalog-${OPENSHIFT_VERSION}.yaml instead of csv_meta_catalog.yaml/bundle_object_catalog.yaml

Local Test Results

Main test: 2.25.5 push for v4.19 (the override scenario)

PCC Source 2.25.5 3.3.2 (prod) 3.4EA2 stable-3.x
OLD (v4.17) ✅ 6 refs ❌ MISSING ❌ MISSING ❌ MISSING
NEW (v4.19) ✅ 6 refs ✅ 4 refs — (not released) ✅ 2 channels

Corner cases (all using NEW per-OCP PCC)

Scenario 2.25.5 3.3.2 3.4EA2 stable-3.x Result
v4.16 — only 2.25 targets, no 3.x expected ✅ No 3.x leakage
v4.19 — 2.25.5 standalone ✅ Shipped 3.x preserved
v4.19 — 3.4EA2 standalone ✅ Shipped 2.x preserved
v4.20 — 2.25.5 standalone ✅ Same as v4.19
v4.20 — 3.4EA2 standalone ✅ Same as v4.19
v4.21 — 2.25.5 standalone ✅ Works on newest OCP
v4.22 — fallback PCC (copied from v4.21) ✅ Graceful fallback
Missing PCC file (no fallback) ✅ Fails with clear FileNotFoundError

Multi-version same-day test (chained, simulating multi-push-to-stage)

Order 2.25.5 3.3.2 3.4EA2 stable-3.x
2.25.5 first → 3.4EA2 second
3.4EA2 first → 2.25.5 second

Scope & Limitations

  • This fix protects shipped versions from being wiped by single-branch pushes
  • For same-day multi-version releases (e.g., 2.25.5 + 3.4EA2 on April 23rd), continue using multi-push-to-stage.yaml which chains outputs — this fix does not add chaining to push-to-stage.yaml
  • Logic is identical to what's already running in production via multi-push-to-stage.yaml
  • No changes to stage_promoter.py or any other script

When to use which workflow

Scenario Workflow
Single release, no same-day overlap push-to-stage.yaml (this PR)
Multiple releases same day, overlapping OCP multi-push-to-stage.yaml

Tracker: https://redhat.atlassian.net/browse/RHOAIENG-59007

The single-branch push-to-stage workflow was generating PCC from a fixed
v4.17 operator index, which lacks 3.x entries (3.x only targets v4.19+).
This caused catalog overrides when pushing 2.x releases to overlapping
OCP versions — wiping shipped 3.x bundles and channels from the index.

Aligns with multi-push-to-stage.yaml by:
- Generating per-OCP PCC files from each version's actual production index
- Using catalog-{OCP_VERSION}.yaml instead of the single v4.17-based PCC

Signed-off-by: Akshay Ghodake <aghodake@redhat.com>
@crackcodecamp
Copy link
Copy Markdown
Contributor Author

Hello @dchourasia, Whenever you get some time please help review these changes, Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant