fix(windows): avoid false CSE failure during kubelet recovery

[r2k1]

What this PR does / why we need it:

Windows CSE can fail after node reset because NSSM reports kubelet as running before it is healthy. Kubelet may exit while containerd creates its pipe, then recover on the next NSSM attempt.

This PR:

• waits for the reset task, then polls kubelet's effective /healthz endpoint;
• preserves the service-state fallback when healthz is disabled;
• surfaces direct kubelet startup errors and restores HNS remediation after kubelet startup;
• preserves Windows E2E diagnostics across AzCopy timeouts and locked logs.

The endpoint and helper ship together in CustomData, including for old and PIS-cached VHDs.

Which issue(s) this PR fixes:

N/A. Related: #8970, #9344

Testing:

• 53 Windows CSE helper tests passed
• go test ./pkg/agent/datamodel ./pkg/agent
• Windows Server 2025 Gen2 VHD E2E passed

[github-actions]

Windows Unit Test Results

  3 files   13 suites   55s ⏱️
409 tests 409 ✅ 0 💤 0 ❌
412 runs  412 ✅ 0 💤 0 ❌

Results for commit 2acbc86.

♻️ This comment has been updated with latest results.

[Copilot]

🔵 Needs a closer look

Standalone fallback logs should be downloaded before the large archive can exhaust their shared timeout.

Review details

Suppressed comments (1)

e2e/vmss.go:1011

• 🟡 Medium Risk — 🔧 Script Logic: The large archive is attempted first while all downloads share one two-minute context. If that download consumes the budget, the newly added kubelet/containerd fallback downloads start with an expired context and cannot provide partial diagnostics; download the small standalone artifacts before the archive.

	downloadBlob("collected-node-logs.zip")
	downloadBlob("cse.log")
	downloadBlob("provision.complete")
	downloadBlob("kubelet.err.log")
	downloadBlob("containerd.err.log")

• Files reviewed: 7/7 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

It changes production-critical Windows provisioning across independently released CSE and VHD components, warranting final human validation.

Review details

• Files reviewed: 9/9 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Disabled health checks can hide kubelet failure, and uploading the large archive first can prevent partial diagnostic recovery.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

parts/windows/windowscsehelper.ps1:337

• Severity: 🟡 Medium Risk — Script Logic / Backward Compatibility. When --healthz-port=0, this removes the previous post-task service-state validation. Start-Service can return successfully and kubelet can then exit—the failure mode this change is addressing—so CSE now reports success with a stopped kubelet for this supported configuration. Keep a bounded Get-Service/start-state fallback when the endpoint is disabled, and update this branch's Pester case accordingly.

• Files reviewed: 9/9 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Disabled health checks can conceal kubelet failure, and diagnostic collection still has ordering and coverage gaps.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

staging/cse/windows/debug/collect-windows-logs.ps1:46

• This file explicitly requires corresponding updates to provisioningscripts/loggenerator.ps1 when adding collected logs. That script includes active azure-vnet.log but only rotated azure-vnet-ipam.log.*, so its periodic diagnostic bundle still omits the currently locked IPAM log. Add azure-vnet-ipam.log to its C:\k log list as part of this change.

e2e/vmss.go:890

• 🟡 Medium Risk — 🏗️ Architecture: The aggregate ZIP is still the first AzCopy upload. If that large transfer hangs or exhausts the RunCommand window, none of the small standalone diagnostics exists for the new detached download phase to recover. Upload the small files first and leave the ZIP last, matching the download priority.

if ($collectedLogs) {
    Upload-Log $collectedLogs "$arg1/collected-node-logs.zip"
}
Upload-Log "C:\azuredata\CustomDataSetupScript.log" "$arg1/cse.log"
Upload-Log "C:\AzureData\provision.complete" "$arg1/provision.complete"
Upload-Log "C:\k\kubelet.err.log" "$arg1/kubelet.err.log"
Upload-Log "C:\k\containerd.err.log" "$arg1/containerd.err.log"
Upload-Log "network_config.txt" "$arg1/network_config.txt"

• Files reviewed: 9/9 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

It changes critical Windows node provisioning and recovery behavior across current, legacy, and PIS-cached VHDs.

Review details

• Files reviewed: 10/10 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

It changes production-critical Windows provisioning and cached-VHD recovery behavior across Go templates and PowerShell lifecycle scripts.

Review details

• Files reviewed: 10/10 changed files
• Comments generated: 0 new
• Review effort level: Balanced