Skip to content

chore: sync upstream e2e tests to downstream#1179

Open
varshab1210 wants to merge 4 commits into
redhat-developer:masterfrom
varshab1210:test-updates-1.21
Open

chore: sync upstream e2e tests to downstream#1179
varshab1210 wants to merge 4 commits into
redhat-developer:masterfrom
varshab1210:test-updates-1.21

Conversation

@varshab1210

Copy link
Copy Markdown
Member

What type of PR is this?

/kind enhancement

What does this PR do / why we need it:

sync upstream argocd-operator e2e tests to downstream
Tested locally against 1.21 RC

Have you updated the necessary documentation?

  • Documentation update is required by this PR.
  • Documentation has been updated.

Which issue(s) this PR fixes:

Fixes #?

Test acceptance criteria:

  • Unit Test
  • E2E Test

How to test changes / Special notes to the reviewer:

Signed-off-by: Varsha B <vab@redhat.com>
@openshift-ci openshift-ci Bot requested review from Naveena-058 and svghadi June 16, 2026 10:43
@openshift-ci

openshift-ci Bot commented Jun 16, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign varshab1210 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: d7198549-1f9d-484b-97aa-b227c3608603

📥 Commits

Reviewing files that changed from the base of the PR and between fc863da and 5c949f9.

📒 Files selected for processing (1)
  • test/openshift/e2e/ginkgo/sequential/1-037_validate_applicationset_in_any_namespace_test.go
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • argoproj-labs/argocd-operator (manual)
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/openshift/e2e/ginkgo/sequential/1-037_validate_applicationset_in_any_namespace_test.go

📝 Walkthrough

Summary by CodeRabbit

  • New Features
    • Extended the Argo CD CRD with per-component Prometheus ServiceMonitor tuning (interval/scrapeTimeout), Application Controller sharding algorithm selection, and improved logging configuration (logFormat/logLevel with deprecated logformat).
    • Added declarative webhookSecrets syncing with Azure DevOps username/password pairing validation, plus Agent resource configuration and new top-level options (clusterDomain, cmdParams, webTerminalEnabled).
  • Tests
    • Expanded OpenShift E2E coverage for logformat backward compatibility, controller sharding env var behavior, Dex token/secret handling, ApplicationSet cleanup/reconcile, server serving-cert annotation restoration, webhook secret sync/preserve/remove, metrics ServiceMonitor enablement, and RoleBinding annotation non-inheritance.

Walkthrough

Adds comprehensive Ginkgo E2E tests across parallel and sequential suites covering operator runtime behaviors and schema support, plus expands ArgoCD CRD definitions. E2E additions validate: deprecated lowercase logformat field propagation, controller sharding algorithm env-var lifecycle, Dex short-lived token creation and legacy cleanup, ApplicationSet resource deletion on spec omission, OpenShift serving-cert annotation add/remove/restore, declarative webhook secret sync and removal, Prometheus ServiceMonitor metrics propagation, RoleBinding annotation isolation, ApplicationSet RBAC revocation recovery, and agent/principal ServiceMonitor toggles. CRD enhancements add metrics configuration blocks, resource limits, webhook provider references, log format standardization, sharding algorithm selection, root-level configuration fields, and NetworkPolicy default handling.

Changes

E2E Test Coverage Expansion and CRD Schema Enhancement

Layer / File(s) Summary
Deprecated logformat field and sharding algorithm env-var coverage
test/openshift/e2e/ginkgo/parallel/1-043_validate_log_level_format_test.go, test/openshift/e2e/ginkgo/parallel/1-048_validate_controller_sharding_test.go
Adds test validating deprecated spec.logformat fields on ApplicationSet and Notifications propagate --logformat json to Deployment commands. Extends sharding test to assert ARGOCD_CONTROLLER_SHARDING_ALGORITHM env var is set on StatefulSet when DistributionAlgorithm is configured and removed when omitted.
Dex token secret lifecycle and legacy cleanup
test/openshift/e2e/ginkgo/parallel/1-095_validate_dex_clientsecret_test.go
Replaces single Dex test with two It blocks: first validates short-lived operator-created token secret with RFC3339 future expiry, $oidc.dex.clientSecret placeholder in argocd-cm, and synchronization into argocd-secret; second injects legacy non-expiring token, triggers cleanup via annotation, and verifies deletion and continued Argo CD health.
ApplicationSet omission cleanup and serving-cert annotation lifecycle
test/openshift/e2e/ginkgo/parallel/1-125_validate_applicationset_cleanup_when_spec_field_omitted_test.go, test/openshift/e2e/ginkgo/parallel/1-125_validate_server_serving_cert_annotation_restore_test.go
New test verifying ApplicationSet controller resources (Deployment, ServiceAccount, Role, RoleBinding, Service) are deleted when spec.applicationSet is set to nil. Two-spec test validates OpenShift serving-cert annotation creation under default TLS, removal on passthrough, restoration on revert to default, and preservation in user-managed TLS-secret scenarios.
Declarative webhook secrets sync, retention, removal, and validation
test/openshift/e2e/ginkgo/parallel/1-126_validate_declarative_webhook_secrets_test.go
Large suite exercising spec.webhookSecretsargocd-secret propagation for GitHub, GitLab, Azure DevOps (username+password pair), Bitbucket, and Gogs. Tests retention when stanza is absent or cleared, provider-driven key removal, atomic Azure DevOps pair cleanup on invalid references, and API rejection of partial Azure DevOps configuration.
ServiceMonitor metrics endpoint configuration propagation
test/openshift/e2e/ginkgo/parallel/1-126_validate_servicemonitor_metrics_config_test.go
Suite with three test blocks verifying interval and scrapeTimeout propagation from component/principal/agent metrics specs into Prometheus ServiceMonitor endpoints, including update and clear lifecycle steps.
RoleBinding operator annotation isolation
test/openshift/e2e/ginkgo/parallel/1-131_validate_rolebinding_no_tracking_annotation_propagation_test.go
Test asserting operator-generated RoleBindings in managed and Argo CD instance namespaces retain only operator default annotation and do not inherit CR-level tracking annotations.
ApplicationSet RBAC revocation and reconcile-driven restoration
test/openshift/e2e/ginkgo/sequential/1-037_validate_applicationset_in_any_namespace_test.go
Extends sequential suite with error namespaces in cluster config and debug output. Adds test that snapshots and removes ClusterRole list namespaces permission, verifies ApplicationSet controller Deployment is blocked, restores rules via annotation-triggered reconcile, and confirms Deployment reappears with --applicationset-namespaces flag. Includes helpers to identify operator ClusterRoles and snapshot/restore namespace-list permissions.
Principal and agent ServiceMonitor lifecycle via Prometheus flag
test/openshift/e2e/ginkgo/sequential/1-051_validate_argocd_agent_principal_test.go, test/openshift/e2e/ginkgo/sequential/1-052_validate_argocd_agent_agent_test.go
Adds Prometheus Operator imports to principal and agent sequential suites. Principal test toggles spec.prometheus.enabled and asserts ServiceMonitor creation/deletion/recreation and removal on principal disable. Agent test toggles Prometheus and agent enabled flags and asserts ServiceMonitor lifecycle.
CRD metrics configuration for all monitored components
bundle/manifests/argoproj.io_argocds.yaml, config/crd/bases/argoproj.io_argocds.yaml
Both CRD manifests expand with metrics schema blocks (containing interval and scrapeTimeout) for Agent, Principal, Application Controller, Notifications, Repo Server, and Server components across multiple schema sections.
CRD log format standardization and controller sharding algorithm
bundle/manifests/argoproj.io_argocds.yaml, config/crd/bases/argoproj.io_argocds.yaml
Both CRD manifests add logFormat (enum: text/json), logLevel, and deprecated logformat to Notifications and ApplicationSet configurations. Application Controller sharding configuration introduces algorithm enum (legacy, round-robin, consistent-hashing) in multiple schema sections.
CRD resource configuration, webhook secrets, and root-level fields
bundle/manifests/argoproj.io_argocds.yaml, config/crd/bases/argoproj.io_argocds.yaml
Both CRD manifests add resources blocks for Agent container and principal component supporting Kubernetes claims, limits, and requests. New webhookSecrets schema with provider-specific secret references (Azure DevOps, Bitbucket, GitHub, GitLab, Gogs) and validation ensuring Azure DevOps refs are paired. Root-level fields clusterDomain, cmdParams, and webTerminalEnabled are introduced.
CRD NetworkPolicy default removal and bundle metadata update
bundle/manifests/argoproj.io_argocds.yaml, config/crd/bases/argoproj.io_argocds.yaml, bundle/manifests/gitops-operator.clusterserviceversion.yaml
Both CRD manifests remove explicit default: true from networkPolicy.enabled across multiple schema locations. CSV metadata createdAt timestamp is updated.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~70 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: syncing upstream e2e tests to downstream, which is directly reflected in the file additions and test changes throughout the PR.
Description check ✅ Passed The description relates to the changeset by indicating it syncs upstream e2e tests to downstream and mentions testing against 1.21 RC, which aligns with the test file additions present in the PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

defer cleanupFunc()

argoCD := &argov1beta1api.ArgoCD{
ObjectMeta: metav1.ObjectMeta{Name: "example-argocd-clear-gh", Namespace: ns.Name},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aali309 I had to change the name as I saw the error

Route.route.openshift.io "example-argocd-clear-github-server" is invalid: spec.host: Invalid value: "example-argocd-clear-github-server-gitops-e2e-test-0b13d83e-eda2.apps.psi-421-002.ocp-gitops-qe.com": must be no more than 63 characters

Looks like we need something similar to https://redhat.atlassian.net/issues?jql=reporter%20IN%20(62332b7550cceb00707ad539)%20AND%20type%20%3D%20Bug%20AND%20summary%20~%20%22host%22&selectedIssue=GITOPS-7315

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@test/openshift/e2e/ginkgo/parallel/1-048_validate_controller_sharding_test.go`:
- Around line 119-122: The assertion at line 121 negates the
HaveContainerWithEnvVar matcher with a specific key and value, which only passes
if the environment variable exists with a different value. This is too weak and
can miss regressions. Instead of using ShouldNot with HaveContainerWithEnvVar
matching a specific key-value pair, use a matcher that asserts the
ARGOCD_CONTROLLER_SHARDING_ALGORITHM environment variable key is completely
absent from the app controller StatefulSet, regardless of its value. Replace the
negated key-value matcher with an assertion that explicitly checks for the
absence of the environment variable key itself.

In
`@test/openshift/e2e/ginkgo/sequential/1-037_validate_applicationset_in_any_namespace_test.go`:
- Around line 643-656: The nested loop structure selecting the
operatorClusterRoleName picks the first matching ClusterRoleBinding for the
ServiceAccount without verifying it grants the required permissions, which can
lead to selecting the wrong RBAC object. Instead of selecting based only on
matching the ServiceAccount name and namespace, implement a deterministic
selection criterion by resolving the candidate ClusterRoles and validating that
the selected ClusterRole grants the necessary permissions (such as `list` on
`namespaces`) before using it for mutation. This ensures the correct operator
ClusterRole is chosen for the test rather than any arbitrary matching binding.
- Around line 663-691: The rule rewriting logic in the loop iterating through
operatorClusterRole.Rules does not handle PolicyRules containing multiple
resources correctly. When a rule contains both "namespaces" and other resources,
the current code removes "list" from the verbs but applies this modified verb
set to all resources in the rule, not just namespaces. To fix this, when you
find "namespaces" in the resources list, check if there are other resources
present as well. If it is a mixed rule with multiple resources, create two
separate PolicyRule objects: one with only "namespaces" as the resource and the
modified verbs (without "list"), and another with the non-namespaces resources
and the original verbs. If "namespaces" is the only resource, proceed with the
existing logic of removing "list" and adding the modified rule to modifiedRules.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: ad187eb0-4a14-46b0-ae80-5a89cd221a7f

📥 Commits

Reviewing files that changed from the base of the PR and between 426874a and e35b5df.

📒 Files selected for processing (11)
  • test/openshift/e2e/ginkgo/parallel/1-043_validate_log_level_format_test.go
  • test/openshift/e2e/ginkgo/parallel/1-048_validate_controller_sharding_test.go
  • test/openshift/e2e/ginkgo/parallel/1-095_validate_dex_clientsecret_test.go
  • test/openshift/e2e/ginkgo/parallel/1-125_validate_applicationset_cleanup_when_spec_field_omitted_test.go
  • test/openshift/e2e/ginkgo/parallel/1-125_validate_server_serving_cert_annotation_restore_test.go
  • test/openshift/e2e/ginkgo/parallel/1-126_validate_declarative_webhook_secrets_test.go
  • test/openshift/e2e/ginkgo/parallel/1-126_validate_servicemonitor_metrics_config_test.go
  • test/openshift/e2e/ginkgo/parallel/1-131_validate_rolebinding_no_tracking_annotation_propagation_test.go
  • test/openshift/e2e/ginkgo/sequential/1-037_validate_applicationset_in_any_namespace_test.go
  • test/openshift/e2e/ginkgo/sequential/1-051_validate_argocd_agent_principal_test.go
  • test/openshift/e2e/ginkgo/sequential/1-052_validate_argocd_agent_agent_test.go
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • argoproj-labs/argocd-operator (manual)

Comment thread test/openshift/e2e/ginkgo/parallel/1-048_validate_controller_sharding_test.go Outdated
CRD changes
Assisted-by: cursor

@cjcocokrisp cjcocokrisp left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM to me overall. Addressed what I think about the coderabbit comment as well. Once this is merged I can also sync this change with the upstream test.


By("checking if ARGOCD_CONTROLLER_SHARDING_ALGORITHM env var is not set in app controller StatefulSet")
Eventually(statefulSet).Should(k8sFixture.ExistByName())
Eventually(statefulSet, "60s", "5s").ShouldNot(statefulset.HaveContainerWithEnvVar("ARGOCD_CONTROLLER_SHARDING_ALGORITHM", "round-robin", 0), "Statefulset should not have ARGOCD_CONTROLLER_SHARDING_ALGORITHM")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with coderabbit's comment. The suggested fix also seems good to me as well and I have commented below the version that I used to test it.

			Eventually(func() bool {
				if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(statefulSet), statefulSet); err != nil {
					return false
				}
				if len(statefulSet.Spec.Template.Spec.Containers) == 0 {
					return false
				}
				for _, env := range statefulSet.Spec.Template.Spec.Containers[0].Env {
					if env.Name == "ARGOCD_CONTROLLER_SHARDING_ALGORITHM" {
						return false
					}
				}
				return true
			}, "60s", "5").Should(BeTrue(), "StatefulSet should have no env variable named ARGOCD_CONTROLLER_SHARDING_ALGORITHM")

Signed-off-by: Varsha B <vab@redhat.com>
@varshab1210

Copy link
Copy Markdown
Member Author

/test v4.14-ci-index-gitops-operator-bundle

@varshab1210

Copy link
Copy Markdown
Member Author

/test v4.14-kuttl-parallel

@aali309

aali309 commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

LGTM

Signed-off-by: Varsha B <vab@redhat.com>
@varshab1210

Copy link
Copy Markdown
Member Author

/retest

@varshab1210

Copy link
Copy Markdown
Member Author
/test v4.14-images

@varshab1210

Copy link
Copy Markdown
Member Author

/retest

2 similar comments
@varshab1210

Copy link
Copy Markdown
Member Author

/retest

@varshab1210

Copy link
Copy Markdown
Member Author

/retest

@varshab1210

Copy link
Copy Markdown
Member Author

/test v4.14-kuttl-parallel

@openshift-ci

openshift-ci Bot commented Jun 21, 2026

Copy link
Copy Markdown

@varshab1210: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/v4.14-kuttl-parallel 5c949f9 link false /test v4.14-kuttl-parallel

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants