Skip to content

fix: unified spec-first official image import for ocp jobs#5213

Open
deepsm007 wants to merge 1 commit into
openshift:mainfrom
deepsm007:fix-ocp-input-import-spec
Open

fix: unified spec-first official image import for ocp jobs#5213
deepsm007 wants to merge 1 commit into
openshift:mainfrom
deepsm007:fix-ocp-input-import-spec

Conversation

@deepsm007
Copy link
Copy Markdown
Contributor

@deepsm007 deepsm007 commented May 28, 2026

/hold
/cc @openshift/test-platform

https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-api-2854-openshift-installer-10566-openshift-origin-31211-openshift-machine-config-operator-6076-ci-5.0-e2e-aws-ovn-techpreview-serial-3of3/2060029597415641088#1:build-log.txt%3A150-157

https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_continuous-release-jobs/1797/pull-ci-openshift-continuous-release-jobs-master-images/2060040001508347904

Unified spec-first official image import for OpenShift CI jobs

This PR changes how ci-operator detects and imports OpenShift base images for builds so the tool prefers ImageStream/spec-defined sources ("spec-first") and avoids duplicate Dockerfile-detected imports.

What changed (practical terms)

  • Dockerfile scanning: extraction of registry references was simplified to produce a de-duplicated, first-seen list and no longer requires a caller-provided "from" value. This reduces accidental duplicate detections.
  • Dockerfile input detection: DetectInputsFromDockerfile now accepts the existing baseImages map so detection can skip creating inputs for base images already configured.
  • Official-image resolution: new, spec-aware helpers (ResolveOfficialInputFrom / OfficialImageTagFrom / FindSpecTag) implement a spec-first resolution flow that:
    • prefers a "stable" ImageStream in the job namespace when applicable,
    • otherwise prefers the source ImageStream spec (ImageStreamTag) if present,
    • falls back to a DockerImage (quay) pull spec only when the ImageStream/spec path does not apply.
  • InputImageTag step: uses the new resolver to decide whether to create pipeline ImageStreamTag From references as ImageStreamTag vs DockerImage and sets the appropriate TagReferencePolicy; the step now records the resolved source pull spec in events/metrics.
  • Promotion/mirroring: promotion scripts were refactored to use explicit retry helper logic and per-tag retry loops; several promotion test fixture YAMLs were reformatted to match the new retry patterns.
  • Tests: multiple unit tests added and updated to cover Dockerfile detection, spec-first resolution (stable-first behavior), InputImageTag outcomes, and the mirror-retry helper.

Components affected

  • ci-operator Dockerfile parsing and input-detection (pkg/dockerfile, pkg/defaults)
  • Input image creation and image utilities (pkg/steps/input_image_tag.go, pkg/steps/utils/image.go)
  • Release promotion/mirroring logic and promotion Pod fixtures (pkg/steps/release and testdata)
  • Unit tests under pkg/dockerfile, pkg/steps/*, and pkg/steps/utils

Behavior change for CI users/operators

  • ci-operator will preferentially use ImageStream/spec-defined sources (including stable ImageStreams in the job namespace) for official OCP base images, reducing unexpected imports from quay when a spec is authoritative.
  • Dockerfile-based input detection will avoid creating duplicate import entries when a base image is already present in configured baseImages, producing fewer redundant pipeline ImageStreamTag inputs.
  • Promotion/mirror operations have clearer, more robust retry behavior; fixtures/tests reflect the new retry structure.
  • Improved observability: InputImageTag records the resolved source pull spec in events/metrics, helping operators understand whether an import used a local ImageStreamTag or a DockerImage pull.

Risk / review notes

  • Several function signatures changed and new exported helpers were added; callers were updated accordingly—reviewers should confirm API changes are correct and used consistently.
  • The resolver's ordering (stable-first → source spec → quay) determines import semantics; carefully review ResolveOfficialInputFrom and related unit tests to validate expected behavior across consolidated OCP promotion, legacy streams, and edge cases.
  • Promotion script and fixture changes affect retry semantics; validate cluster behavior for long-running mirror/tag operations.

Additional PR metadata: the author added a hold and CC to @openshift/test-platform and included Prow CI log references for related failures.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 28, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 3c77575e-105c-498c-8778-c2aab07f1d29

📥 Commits

Reviewing files that changed from the base of the PR and between e382c3e and caf8970.

📒 Files selected for processing (19)
  • cmd/registry-replacer/main.go
  • pkg/defaults/defaults.go
  • pkg/dockerfile/extract.go
  • pkg/dockerfile/inputs.go
  • pkg/dockerfile/inputs_test.go
  • pkg/steps/input_image_tag.go
  • pkg/steps/input_image_tag_test.go
  • pkg/steps/release/promote.go
  • pkg/steps/release/promote_test.go
  • pkg/steps/release/snapshot.go
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case__arm64_only.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case__multi_architecture.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_4.12.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_multiple_tags.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_non_release_namespace.yaml
  • pkg/steps/utils/image.go
  • pkg/steps/utils/image_test.go
✅ Files skipped from review due to trivial changes (1)
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case__arm64_only.yaml
🚧 Files skipped from review as they are similar to previous changes (15)
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case__multi_architecture.yaml
  • cmd/registry-replacer/main.go
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case.yaml
  • pkg/defaults/defaults.go
  • pkg/dockerfile/extract.go
  • pkg/steps/release/promote_test.go
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay.yaml
  • pkg/dockerfile/inputs_test.go
  • pkg/steps/release/snapshot.go
  • pkg/steps/input_image_tag_test.go
  • pkg/steps/release/promote.go
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_multiple_tags.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_4.12.yaml
  • pkg/steps/input_image_tag.go
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_non_release_namespace.yaml

📝 Walkthrough

Walkthrough

Simplify Dockerfile registry extraction, make Dockerfile input detection aware of existing base images, add official-image resolution helpers and integrate them into input-tag creation, and refactor promotion/snapshot retry logic and fixtures.

Changes

Official Image Resolution and Dockerfile Input Detection

Layer / File(s) Summary
Registry extraction API simplification
pkg/dockerfile/extract.go, cmd/registry-replacer/main.go
ExtractRegistryReferences now accepts only Dockerfile bytes; de-duplication uses a seen set and first-seen order; callers updated to new signature.
Dockerfile input detection with base image awareness
pkg/dockerfile/inputs.go, pkg/defaults/defaults.go, pkg/dockerfile/inputs_test.go
DetectInputsFromDockerfile gains a baseImages parameter, uses simplified extraction, and skips detected refs that match entries in baseImages. Tests updated/added for multi-stage and COPY --from=... cases.
Official image resolution utilities
pkg/steps/utils/image.go, pkg/steps/utils/image_test.go
Export FindSpecTag, add OfficialImageTagFrom and ResolveOfficialInputFrom to prefer stable ImageStreamTags, fall back to source ImageStream spec/status, or use Quay-derived DockerImage when appropriate; tests added for resolver scenarios.
InputImageTagStep official resolution integration
pkg/steps/input_image_tag.go, pkg/steps/input_image_tag_test.go
Add resolveOfficialImport, unify computation of From, ReferencePolicy.Type, and sourcePullSpec across cases, update metrics and tests for official-spec, legacy-stream, and stable-first outcomes.
Snapshot and promotion refactors + fixtures
pkg/steps/release/snapshot.go, pkg/steps/release/promote.go, pkg/steps/release/*_fixture*.yaml, pkg/steps/release/promote_test.go
Shared source loading in snapshotStream, introduce getMirrorRetryShell, reorganize promotion command assembly to use retry helpers and per-tag digest resolution, and update fixtures to explicit multi-attempt retry loops with randomized backoff; tests added/updated accordingly.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested labels

lgtm

🚥 Pre-merge checks | ✅ 16 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 38.89% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (16 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: unified spec-first official image import for OCP jobs. The multi-file refactor implements this exact objective across registry extraction, input detection, and image resolution logic.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Go Error Handling ✅ Passed All error handling follows proper patterns: errors wrapped with fmt.Errorf %w, no panic() calls, nil pointers checked before dereferencing, ignored values are non-errors.
Test Coverage For New Features ✅ Passed All new exported functions have corresponding tests; function signature changes covered by updated test cases; private helpers tested indirectly; refactored code covered by existing tests.
Stable And Deterministic Test Names ✅ Passed No Ginkgo tests in PR. All test modifications use standard Go testing package with table-driven tests, not Ginkgo BDD framework. Check is not applicable.
Test Structure And Quality ✅ Passed The custom check is for Ginkgo test code, but this PR contains only standard Go tests with no Ginkgo framework. The check is not applicable.
Microshift Test Compatibility ✅ Passed PR adds no new Ginkgo e2e tests. All new test functions (7 total) are standard Go unit tests using testing.T, not Ginkgo e2e tests. Check not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR adds no Ginkgo e2e tests. Changes are internal refactoring to pkg/ and unit tests only; check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed PR refactors image handling logic without adding deployment manifests or topology-unaware scheduling constraints like affinity or PDBs.
Ote Binary Stdout Contract ✅ Passed PR modifies ci-tools image handling, dockerfile processing, and promotion code—not OTE test extension binaries. No JSON stdout communication or test framework setup found.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR does not add any Ginkgo e2e tests. It only modifies standard Go unit/integration tests and refactors production code. The check is not applicable.
No-Weak-Crypto ✅ Passed No weak cryptography patterns found. PR changes registry/Dockerfile handling and image stream logic without introducing MD5, SHA1, DES, RC4, custom crypto, or insecure token comparisons.
Container-Privileges ✅ Passed No container privilege escalation settings found in 7 modified YAML fixtures or 8 Go files. No privileged: true, hostPID/Network/IPC, allowPrivilegeEscalation, or SYS_ADMIN capabilities detected.
No-Sensitive-Data-In-Logs ✅ Passed All logging statements log only image pull specs, tag names, and operation status. No credentials, tokens, passwords, API keys, PII, or sensitive configuration are exposed.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 28, 2026

Actionable comments posted: 0

@deepsm007 deepsm007 force-pushed the fix-ocp-input-import-spec branch from 7ef25f4 to e382c3e Compare May 28, 2026 19:31
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 28, 2026
@deepsm007
Copy link
Copy Markdown
Contributor Author

/test e2e

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/steps/utils/image.go`:
- Around line 46-47: The current check in ImageDigestFor uses FindSpecTag(is,
tag) == nil to test tag existence but FindSpecTag returns the spec tag's From
reference so a spec entry with From == nil is treated as missing; add a helper
like hasSpecTag(is *imagev1.ImageStream, tag string) that scans is.Spec.Tags for
a name match, then change the conditional in ImageDigestFor to: if ref == nil &&
!hasSpecTag(is, tag) { return error } so spec-only tags with nil From are not
reported as absent (leave FindSpecTag behaviour unchanged).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 26cf2690-339d-4051-901b-393950fabf6a

📥 Commits

Reviewing files that changed from the base of the PR and between 7ef25f4 and e382c3e.

📒 Files selected for processing (19)
  • cmd/registry-replacer/main.go
  • pkg/defaults/defaults.go
  • pkg/dockerfile/extract.go
  • pkg/dockerfile/inputs.go
  • pkg/dockerfile/inputs_test.go
  • pkg/steps/input_image_tag.go
  • pkg/steps/input_image_tag_test.go
  • pkg/steps/release/promote.go
  • pkg/steps/release/promote_test.go
  • pkg/steps/release/snapshot.go
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case__arm64_only.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case__multi_architecture.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_4.12.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_multiple_tags.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_promotion_quay_non_release_namespace.yaml
  • pkg/steps/utils/image.go
  • pkg/steps/utils/image_test.go
✅ Files skipped from review due to trivial changes (2)
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case.yaml
  • pkg/steps/release/testdata/zz_fixture_TestGetPromotionPod_basic_case__arm64_only.yaml
🚧 Files skipped from review as they are similar to previous changes (9)
  • cmd/registry-replacer/main.go
  • pkg/defaults/defaults.go
  • pkg/steps/release/snapshot.go
  • pkg/steps/utils/image_test.go
  • pkg/dockerfile/extract.go
  • pkg/dockerfile/inputs_test.go
  • pkg/steps/input_image_tag_test.go
  • pkg/steps/input_image_tag.go
  • pkg/dockerfile/inputs.go

Comment thread pkg/steps/utils/image.go Outdated
@deepsm007 deepsm007 force-pushed the fix-ocp-input-import-spec branch from e382c3e to caf8970 Compare May 28, 2026 19:45
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 28, 2026

Actionable comments posted: 0

@deepsm007
Copy link
Copy Markdown
Contributor Author

/assign @danilo-gemoli

@deepsm007
Copy link
Copy Markdown
Contributor Author

/retest

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e

@jupierce
Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 29, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 29, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deepsm007, jupierce

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 7b6e7b8 and 2 for PR HEAD caf8970 in total

@jupierce
Copy link
Copy Markdown
Contributor

/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 29, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 29, 2026

@deepsm007: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e caf8970 link true /test e2e

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@deepsm007
Copy link
Copy Markdown
Contributor Author

/close in favor of #5215 #5216 and #5217

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants