[release-4.22] OCPBUGS-104615: E2E: LLC: Restore uncore cache annotation to true

[openshift-cherrypick-robot]

This is an automated cherry-pick of #1570

/assign mrniranjan

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 117ef6e1-da29-4b92-aaa1-11a888d5bee0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-ci-robot]

@openshift-cherrypick-robot: Jira Issue OCPBUGS-100115 has been cloned as Jira Issue OCPBUGS-104615. Will retitle bug to link to clone.
/retitle [release-4.22] OCPBUGS-104615: E2E: LLC: Restore uncore cache annotation to true

Details

In response to this:

This is an automated cherry-pick of #1570

/assign mrniranjan

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@openshift-cherrypick-robot: This pull request references Jira Issue OCPBUGS-104615, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.22.0) matches configured target version for branch (4.22.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
• release note type set to "Release Note Not Required"
• dependent bug Jira Issue OCPBUGS-100115 is in the state Verified, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
• dependent Jira Issue OCPBUGS-100115 targets the "5.0.0" version, which is one of the valid target versions: 5.0.0
• bug has dependents

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

This is an automated cherry-pick of #1570

/assign mrniranjan

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[qodo-for-rh-openshift]

PR Summary by Qodo

E2E LLC test: restore uncore-cache kubelet annotation via DeferCleanup

🐞 Bug fix 🧪 Tests 🕐 10-20 Minutes

AI Description

• Add DeferCleanup to re-enable prefer-align-cpus-by-uncorecache after disabling it in test 77724.
• Prevent subsequent LLC E2E tests from inheriting a permanently-disabled kubelet policy option.
• Ensure profile update + tuning wait happens during cleanup to fully apply the restored setting.

Diagram

graph TD
  T["LLC E2E test 77724"] --> D["Set annotation false"] --> PP[("PerformanceProfile CR")] --> U["UpdateWithRetry"] --> W["Wait tuning updated"] --> V["Verify kubelet option"]
  T --> C["DeferCleanup"] --> R["Restore annotation true"] --> PP --> U --> W

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Use AfterEach/AfterAll to restore the annotation

• ➕ Centralizes restoration logic for the context/suite
• ➕ Makes cleanup behavior uniform even if future tests modify the same annotation
• ➖ Broader scope than needed; may mask per-test ownership of state changes
• ➖ Can make ordering/ownership unclear when multiple tests mutate the same profile

2. Introduce a helper to toggle annotation with automatic rollback

• ➕ Reduces duplication (update + wait) and standardizes test hygiene
• ➕ Reusable for other profile-annotation toggles
• ➖ More refactor than necessary for a targeted bugfix/cherry-pick
• ➖ Adds indirection for a straightforward lifecycle fix

Recommendation: Keep the current DeferCleanup approach: it tightly scopes restoration to the test that mutates cluster state, and it ensures the restore is fully reconciled (update + wait) before the test ends. The helper/refactor option is useful later, but is likely out-of-scope for this bugfix cherry-pick.

Files changed (1) +16 / -0

Bug fix (1) +16 / -0

llc.goRestore LLC kubelet policy option after test disables it +16/-0

Restore LLC kubelet policy option after test disables it

• Registers a DeferCleanup handler in test_id:77724 to reset kubeletconfig.experimental back to the LLC-enabled policy (prefer-align-cpus-by-uncorecache=true). The cleanup reapplies the PerformanceProfile and waits for tuning to begin and finish updating, preventing subsequent tests from running with the option unintentionally left disabled.

test/e2e/performanceprofile/functests/13_llc/llc.go

[qodo-for-rh-openshift]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

1. Cleanup rollout wait no-op 🐞 Bug ☼ Reliability

Description

The new DeferCleanup unconditionally waits for a tuning rollout (Updating then Updated) after
setting the annotation back to llcPolicy, even when the test may have failed before applying the
disable policy and the restore becomes a no-op. In that case, WaitForTuningUpdating can wait until
its long MCP timeout, obscuring the original failure and slowing/flaking the e2e suite.

Code

test/e2e/performanceprofile/functests/13_llc/llc.go[R262-265]

+					profiles.UpdateWithRetry(perfProfile)
+
+					By(fmt.Sprintf("Applying changes in performance profile and waiting until %s will start updating", poolName))
+					profilesupdate.WaitForTuningUpdating(cleanupCtx, perfProfile)

Relevance

●●● Strong

Team has accepted tightening DeferCleanup to avoid long/stalled waits; aligns with reliability/flake
reduction patterns.

PR-#1556

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The cleanup is registered before the disabling update and always performs rollout waits. The wait
helper blocks until MCP Updating becomes true, and MCP waiting uses long timeouts based on node
count, so a no-op restore can cause a prolonged wait/failure.

test/e2e/performanceprofile/functests/13_llc/llc.go[253-283]
test/e2e/performanceprofile/functests/utils/profilesupdate/profile_update.go[87-108]
test/e2e/performanceprofile/functests/utils/mcps/mcps.go[159-199]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The new `DeferCleanup` always calls `WaitForTuningUpdating/Updated` after attempting to restore the annotation. If the spec fails before the disable policy is applied (or if the annotation is already `llcPolicy`), the restore may not trigger any MCP/nodepool rollout; in that case `WaitForTuningUpdating` can block until timeout.

### Issue Context
`DeferCleanup` is registered before the disable policy update occurs, so it runs even on early failures. `WaitForTuningUpdating` waits for the MCP Updating condition to become true, which will never happen when no rollout is triggered.

### Fix Focus Areas
- test/e2e/performanceprofile/functests/13_llc/llc.go[255-269]

### Suggested fix
In the cleanup function:
1. Fetch the current profile.
2. If `Annotations["kubeletconfig.experimental"]` is already `llcPolicy` (or the desired state), **skip** the update and rollout waits.
3. Otherwise, apply the update and then wait.

Optionally, track a boolean like `disableApplied` (set it only after the disable update + rollout succeeds) and only perform restore waits when `disableApplied == true`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

_{To customize comments, go to the Qodo configuration screen, or learn more in the docs.}

[qodo-for-rh-openshift]

1. Cleanup rollout wait no-op 🐞 Bug ☼ Reliability

The new DeferCleanup unconditionally waits for a tuning rollout (Updating then Updated) after
setting the annotation back to llcPolicy, even when the test may have failed before applying the
disable policy and the restore becomes a no-op. In that case, WaitForTuningUpdating can wait until
its long MCP timeout, obscuring the original failure and slowing/flaking the e2e suite.

Agent Prompt

### Issue description
The new `DeferCleanup` always calls `WaitForTuningUpdating/Updated` after attempting to restore the annotation. If the spec fails before the disable policy is applied (or if the annotation is already `llcPolicy`), the restore may not trigger any MCP/nodepool rollout; in that case `WaitForTuningUpdating` can block until timeout.

### Issue Context
`DeferCleanup` is registered before the disable policy update occurs, so it runs even on early failures. `WaitForTuningUpdating` waits for the MCP Updating condition to become true, which will never happen when no rollout is triggered.

### Fix Focus Areas
- test/e2e/performanceprofile/functests/13_llc/llc.go[255-269]

### Suggested fix
In the cleanup function:
1. Fetch the current profile.
2. If `Annotations["kubeletconfig.experimental"]` is already `llcPolicy` (or the desired state), **skip** the update and rollout waits.
3. Otherwise, apply the update and then wait.

Optionally, track a boolean like `disableApplied` (set it only after the disable update + rollout succeeds) and only perform restore waits when `disableApplied == true`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[mrniranjan]

This is automatic backport PR and this issue needs a new PR on the main branch.

[mrniranjan]

/retest-required

[mrniranjan]

/retest-required

[yanirq]

/lgtm

[yanirq]

/approve

[openshift-merge-bot]

Scheduling required tests:
/test e2e-aws-operator
/test e2e-aws-ovn
/test e2e-aws-ovn-techpreview
/test e2e-gcp-pao
/test e2e-gcp-pao-updating-profile
/test e2e-gcp-pao-workloadhints
/test e2e-hypershift
/test e2e-hypershift-pao
/test e2e-no-cluster
/test e2e-upgrade

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: openshift-cherrypick-robot, yanirq

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [yanirq]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@openshift-cherrypick-robot: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-gcp-pao	adc082c	link	true	/test e2e-gcp-pao

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.