Skip to content

Use AdminPolicyBasedExternalRoute CR for external gateway test#31293

Open
arkadeepsen wants to merge 1 commit into
openshift:mainfrom
arkadeepsen:use-apber-for-ex-gw
Open

Use AdminPolicyBasedExternalRoute CR for external gateway test#31293
arkadeepsen wants to merge 1 commit into
openshift:mainfrom
arkadeepsen:use-apber-for-ex-gw

Conversation

@arkadeepsen

@arkadeepsen arkadeepsen commented Jun 11, 2026

Copy link
Copy Markdown
Member

This PR changes existing external gateway tests to use AdminPolicyBasedExternalRoute CR instead of annotations.

Summary by CodeRabbit

  • Tests
    • Rewrote external-gateway tests to validate IPv4, IPv6, and Dual-Stack behaviors using fixed pods and ovnkube-node log-based verification of policy application or expected sync failures.
    • Added assertions ensuring pod IP-family matches cluster IP-family for Dual-Stack scenarios.
  • Refactor
    • Improved IP-family detection by deriving family from pod status.
    • Added utilities to apply namespace-scoped external-gateway policies via rendered manifests.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: f5441f88-234e-49ae-b1d6-93e2016e7915

📥 Commits

Reviewing files that changed from the base of the PR and between dc6898d and 707c2f4.

📒 Files selected for processing (2)
  • test/extended/networking/external_gateway.go
  • test/extended/networking/util.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/extended/networking/external_gateway.go

Walkthrough

Refactors external gateway tests to render and apply AdminPolicyBasedExternalRoute manifests, poll ovnkube-node logs for APB policy sync outcomes, and derive pod IP-family from created Pod.Status.PodIPs.

Changes

External Gateway Address Test Refactoring

Layer / File(s) Summary
APB manifest templating & imports
test/extended/networking/util.go
Add bytes and text/template imports, introduce AdminPolicyBasedExternalRoute Go template, params struct, and renderer that executes the template into an in-memory manifest for application.
Pod creation and IP-family detection
test/extended/networking/util.go
Change createPod to return *corev1.Pod (poll until PodRunning) and update GetIPFamilyForCluster to determine IP family from pod.Status.PodIPs.
Imports and test setup
test/extended/networking/external_gateway.go, test/extended/networking/util.go
Update test imports for log polling and pod log retrieval; create CLI without namespace, set privileged pod security at framework level, create labeled namespace, add ovnkube-node log helper, and derive cluster pod IP family.
IPv6 test scenario
test/extended/networking/external_gateway.go
Apply an explicitly named IPv6 external-gateway APB to the labeled namespace, create a fixed-name IPv6 pod, and poll ovnkube-node logs until success or the expected error log is observed depending on cluster pod IP family.
IPv4 test scenario
test/extended/networking/external_gateway.go
Apply an explicitly named IPv4 APB, create a fixed-name IPv4 pod, and poll ovnkube-node logs until success or the expected error log is observed depending on cluster pod IP family.
Dual Stack test scenario
test/extended/networking/external_gateway.go
Apply an explicitly named dual-stack APB, create a fixed-name dual-stack pod, assert the pod’s IP family equals the cluster pod IP family, and poll ovnkube-node logs for dual-stack policy success.

🎯 3 (Moderate) | ⏱️ ~20 minutes


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error, 4 warnings)

Check name Status Explanation Resolution
Container-Privileges ❌ Error util.go creates a HostNetwork pod running as root (RunAsUser=0) with CAP_NET_ADMIN; external_gateway.go sets NamespacePodSecurityLevel=LevelPrivileged. Remove/avoid HostNetwork + root/capabilities (or run non-root) and avoid privileged pod-security level unless you add an explicit justification and constraints.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Structure And Quality ⚠️ Warning FAIL: external_gateway.go’s single It block tests IPv6/IPv4/DualStack together, has no BeforeEach/AfterEach, creates a namespace without explicit cleanup, and uses expectNoError(err) without messages. Split IPv6/IPv4/DualStack into separate It blocks; move setup/cleanup to BeforeEach/AfterEach (namespace + APB + pods); add messages to expectNoError/ExpectWithOffset and ensure cluster calls use bounded timeouts/contexts.
Microshift Test Compatibility ⚠️ Warning Fails MicroShift check: test calls InOVNKubernetesContext -> networkPluginName() which runs oc get network cluster (config.openshift.io Networks), not guarded for MicroShift. Add MicroShift runtime skip (exutil.IsMicroShiftCluster) around the test or around networkPluginName(), or tag with [Skipped:MicroShift]/[apigroup:config.openshift.io].
Ipv6 And Disconnected Network Test Compatibility ⚠️ Warning external_gateway.go hardcodes IPv4 gateway IP 10.10.10.1 for IPv4 and dual-stack APB scenarios, an explicit IPv4 assumption. Avoid hardcoded IPv4 (10.10.10.1): derive IPv4 gateway/cidr from cluster pod IP family or guard/skip IPv4 sub-scenarios on IPv6-only CI (e.g., add [Skipped:Disconnected] / context skipping).
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: migrating external gateway tests from annotations to using AdminPolicyBasedExternalRoute CR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Ginkgo test titles are static: only '[sig-network] external gateway address' and 'should match the address family of the pod'; no pod/node/namespace/IP/suffix/timestamps in any It/Describe/Context/...
Single Node Openshift (Sno) Test Compatibility ✅ Passed external_gateway.go’s single Ginkgo It creates pods and checks ovnkube-node logs for pod.Spec.NodeName; no multi-node/HA assumptions (no node-counting, rescheduling, topology-spread).
Topology-Aware Scheduling Compatibility ✅ Passed PR only updates external gateway tests/util (APB templating, pod log polling). Diff contains no pod scheduling constraints (no affinity/topologySpread/nodeSelector/anti-affinity) targeting HA topol...
Ote Binary Stdout Contract ✅ Passed Scanned PR-touched networking test files for process-level stdout writers (fmt.Print*/os.Stdout/klog.*) and suite setup entrypoints; none found.
No-Weak-Crypto ✅ Passed GitHub PR diff for external_gateway.go/util.go contains no md5/SHA1/DES/RC4/3DES/Blowfish/ECB matches and no crypto-related imports were found in the changed code.
No-Sensitive-Data-In-Logs ✅ Passed PR reads non-sensitive OVN system logs (policy names, IP addresses) for test validation; does not log passwords, tokens, API keys, PII, or authentication credentials.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci

openshift-ci Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: arkadeepsen
Once this PR has been reviewed and has the lgtm label, please assign jcaamano for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot requested review from jcaamano and pperiyasamy June 11, 2026 18:42

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
test/extended/networking/util.go (1)

322-336: 💤 Low value

Consider quoting LabelKey in the template to prevent potential injection.

At line 331, {{ .LabelKey }} is not quoted, while {{ .LabelValue | quote }} is. If LabelKey were to come from user input or external sources in the future, this could allow template injection or YAML structure manipulation.

While the current usage hardcodes labelKey as "test" (line 26 in external_gateway.go), applying the defense-in-depth principle suggests quoting all dynamic values in templates.

Proposed defense-in-depth improvement
   from:
     namespaceSelector:
       matchLabels:
-        {{ .LabelKey }}: {{ .LabelValue | quote }}
+        {{ .LabelKey | quote }}: {{ .LabelValue | quote }}

Note: YAML keys should remain unquoted for standard label names, so this may require careful testing to ensure label selectors work correctly with quoted keys.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/networking/util.go` around lines 322 - 336, The template
adminPolicyBasedExternalRouteTemplate uses an unquoted dynamic label key
(LabelKey) which can allow template/YAML injection; update the template to quote
or escape the LabelKey the same way LabelValue is handled (apply the template's
quoting/escaping filter to .LabelKey), run tests to confirm the
AdminPolicyBasedExternalRoute's namespaceSelector.matchLabels still works as
expected, and validate with the code that constructs labelKey
(external_gateway.go) to ensure no functional change.
test/extended/networking/external_gateway.go (1)

48-48: 💤 Low value

Consider using a proper context instead of context.TODO().

While context.TODO() is acceptable for test code, using a proper context with timeout would improve clarity and allow better control over the polling behavior. The context could be created at the test function level and passed through.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/networking/external_gateway.go` at line 48, The
wait.PollUntilContextTimeout call currently uses context.TODO(); replace it with
a proper context created at the test level (e.g., ctx, cancel :=
context.WithTimeout(context.Background(), 2*time.Minute)) and pass that ctx into
wait.PollUntilContextTimeout instead of context.TODO(), and ensure you defer
cancel() to release resources; locate the call to
wait.PollUntilContextTimeout(...) in external_gateway.go and update callers to
use the new ctx so the poll uses the intended timeout and cancellation
semantics.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended/networking/external_gateway.go`:
- Around line 48-69: The DualStack case is unhandled causing timeouts; update
the polling lambda used with wait.PollUntilContextTimeout to treat DualStack
like the IPv6 success path: in the switch on podIPFamily (where DualStack
currently has an empty case) check logs with strings.Contains for
fmt.Sprintf(successLog, apbPolicyName, f.Namespace.Name, podName) and return
true,nil if found (use the same check as the IPv6 branch); do this for both
occurrences of the switch so DualStack clusters detect the success pattern
instead of falling through to a timeout.
- Around line 110-112: Accessing pod.Status.PodIPs before checking the result of
createPod can panic if createPod returned an error; change the order to verify
the error from createPod (the variable err returned by createPod) with
expectNoError(err) or an explicit nil check before touching pod, then only read
pod.Status.PodIPs into podIPs after the success check—look for the
createPod(...) call, the pod variable, pod.Status.PodIPs and the expectNoError
invocation to update.

In `@test/extended/networking/util.go`:
- Around line 530-544: In createPod, the inner closure shadows the outer
retrievedPod by using :=; change the declaration inside the wait.PollImmediate
closure to assign to the outer variable (use = instead of :=) so the
retrievedPod set inside the loop is returned by the function; ensure the closure
still returns the correct (bool, error) values and references
execPod.Namespace/execPod.Name as it currently does.

---

Nitpick comments:
In `@test/extended/networking/external_gateway.go`:
- Line 48: The wait.PollUntilContextTimeout call currently uses context.TODO();
replace it with a proper context created at the test level (e.g., ctx, cancel :=
context.WithTimeout(context.Background(), 2*time.Minute)) and pass that ctx into
wait.PollUntilContextTimeout instead of context.TODO(), and ensure you defer
cancel() to release resources; locate the call to
wait.PollUntilContextTimeout(...) in external_gateway.go and update callers to
use the new ctx so the poll uses the intended timeout and cancellation
semantics.

In `@test/extended/networking/util.go`:
- Around line 322-336: The template adminPolicyBasedExternalRouteTemplate uses
an unquoted dynamic label key (LabelKey) which can allow template/YAML
injection; update the template to quote or escape the LabelKey the same way
LabelValue is handled (apply the template's quoting/escaping filter to
.LabelKey), run tests to confirm the AdminPolicyBasedExternalRoute's
namespaceSelector.matchLabels still works as expected, and validate with the
code that constructs labelKey (external_gateway.go) to ensure no functional
change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 6b72cb40-a8ea-48aa-bbe0-9feba28cf622

📥 Commits

Reviewing files that changed from the base of the PR and between bde16f5 and dd910f5.

📒 Files selected for processing (2)
  • test/extended/networking/external_gateway.go
  • test/extended/networking/util.go

Comment thread test/extended/networking/external_gateway.go Outdated
Comment thread test/extended/networking/external_gateway.go Outdated
Comment thread test/extended/networking/util.go
@arkadeepsen arkadeepsen force-pushed the use-apber-for-ex-gw branch from dd910f5 to ded79e4 Compare June 11, 2026 18:55
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@arkadeepsen arkadeepsen force-pushed the use-apber-for-ex-gw branch from ded79e4 to 69561e8 Compare June 11, 2026 19:42
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@arkadeepsen arkadeepsen force-pushed the use-apber-for-ex-gw branch from 69561e8 to dc6898d Compare June 12, 2026 02:05
@openshift-ci openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Jun 12, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@arkadeepsen

Copy link
Copy Markdown
Member Author

/retest-required

@arkadeepsen arkadeepsen force-pushed the use-apber-for-ex-gw branch from dc6898d to 9744ea2 Compare June 12, 2026 05:50
@arkadeepsen

Copy link
Copy Markdown
Member Author

/retest

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@arkadeepsen arkadeepsen force-pushed the use-apber-for-ex-gw branch from 9744ea2 to 707c2f4 Compare June 12, 2026 17:25
@arkadeepsen

Copy link
Copy Markdown
Member Author

@coderabbitai resume
@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

@arkadeepsen: Sure! Resuming reviews and kicking off a review of the PR now.

✅ Action performed

Reviews resumed.


Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-ci

openshift-ci Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

@arkadeepsen: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-fips 707c2f4 link true /test e2e-aws-ovn-fips
ci/prow/e2e-metal-ipi-ovn-ipv6 707c2f4 link true /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-trt

openshift-trt Bot commented Jun 13, 2026

Copy link
Copy Markdown

Job Failure Risk Analysis for sha: 707c2f4

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-gcp-ovn Low
[Feature:NetworkSegmentation][ovn-kubernetes-ote][sig-network] Network Segmentation: services on a user defined primary network should be reachable through their cluster IP, node port and load balancer L2 primary UDN with custom network, cluster-networked pods, NodePort service [Suite:openshift/conformance/parallel]
This test has passed 0.00% of 7 runs on release 5.0 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:standard Network:ovn NetworkStack:ipv4 OS:rhcos9 Owner:eng Platform:gcp Procedure:none SecurityMode:default Topology:ha Upgrade:micro] in the last week.
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High
[sig-network] external gateway address when using openshift ovn-kubernetes should match the address family of the pod [Suite:openshift/conformance/parallel]
This test has passed 100.00% of 6 runs on release 5.0 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:standard Network:ovn NetworkStack:ipv6 OS:rhcos9 Owner:eng Platform:metal Procedure:none SecurityMode:default Topology:ha Upgrade:micro] in the last week.
pull-ci-openshift-origin-main-e2e-vsphere-ovn Low
[Feature:NetworkSegmentation][ovn-kubernetes-ote][sig-network] Network Segmentation: services on a user defined primary network should be reachable through their cluster IP, node port and load balancer L2 primary UDN with custom network, cluster-networked pods, NodePort service [Suite:openshift/conformance/parallel]
This test has passed 0.00% of 19 runs on release 5.0 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:standard Network:ovn NetworkStack:ipv4 OS:rhcos9 Owner:eng Platform:vsphere Procedure:none SecurityMode:default Topology:ha Upgrade:none] in the last week.
pull-ci-openshift-origin-main-e2e-vsphere-ovn-upi Low
[Feature:NetworkSegmentation][ovn-kubernetes-ote][sig-network] Network Segmentation: services on a user defined primary network should be reachable through their cluster IP, node port and load balancer L2 primary UDN with custom network, cluster-networked pods, NodePort service [Suite:openshift/conformance/parallel]
This test has passed 0.00% of 12 runs on release 5.0 [Architecture:amd64 FeatureSet:default Installer:upi JobTier:standard Network:ovn NetworkStack:ipv4 OS:rhcos9 Owner:eng Platform:vsphere Procedure:none SecurityMode:default Topology:ha Upgrade:none] in the last week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant