Repository navigation
fix(nightly): run Strix Halo Windows E2E on the devlab-dispatch pool #682
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
656ad76
7b4794f
345b398
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -923,11 +923,13 @@ jobs: | |
| # Windows counterpart to e2e-gpu-nightly-strix. Keep the PowerShell-native | ||
| # setup aligned with e2e-selfhosted.yml's proven Strix Windows job while | ||
| # opting into the same platform-adaptive @nightly large-model scenario as the | ||
| # Linux runner. | ||
| # Linux runner. Same DevLab Dispatch pool as e2e-wsl-nightly below and the | ||
| # per-PR e2e-gpu-strix-windows/e2e-wsl jobs in e2e-selfhosted.yml — there is | ||
| # no separate dedicated "native" Windows host registered for this anymore. | ||
|
Comment on lines
+926
to
+928
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No test backs the "same pool as per-PR" claim. |
||
| e2e-gpu-nightly-strix-windows: | ||
| name: E2E tests (Strix Halo, Windows, incl. nightly-only) | ||
| timeout-minutes: 90 | ||
| runs-on: [self-hosted, windows, strix-halo, native] | ||
| runs-on: [self-hosted, windows, devlab-dispatch, strix-halo] | ||
| continue-on-error: true | ||
| strategy: | ||
| fail-fast: false | ||
|
|
@@ -1125,18 +1127,20 @@ jobs: | |
| shell: powershell | ||
| run: | | ||
| # Install the shared runtime in place because its manifest contains | ||
| # absolute paths. The persistent runner workspace keeps those paths valid | ||
| # across scenarios and subsequent runs. | ||
| # absolute paths. On the ephemeral DevLab Dispatch pool this host is | ||
| # destroyed after the job, so the paths only need to stay valid for | ||
| # this run's scenarios, not across runs. | ||
| $prewarm = "$env:RUNNER_WORKSPACE\e2e-prewarm-multi-arch-v2" | ||
| $env:E2E_SHARED_RUNTIMES_DIR = "$prewarm\data\runtimes" | ||
|
|
||
| # See e2e-gpu-strix-windows in e2e-selfhosted.yml: the harness's own | ||
| # isolate_env() derives HF_HOME/PIP_CACHE_DIR/ROCM_CLI_CACHE_DIR from | ||
| # this per scenario. Without it every serve scenario here re-downloads | ||
| # its model checkpoint cold, which raced serve-08's readiness timeout | ||
| # (EAI-8031 / #260) on every nightly run — this host was never actually | ||
| # warm for it, since Windows doesn't set HOME by default and the | ||
| # engine's cache lookup didn't fall back to USERPROFILE. | ||
| # (EAI-8031 / #260) on every nightly run — on the ephemeral pool there's | ||
| # no cache carried over from a prior job, so this is cold every | ||
| # scenario, and Windows doesn't set HOME by default so the engine's | ||
| # cache lookup didn't fall back to USERPROFILE either. | ||
| $env:E2E_SHARED_CACHE_DIR = "$env:RUNNER_WORKSPACE\e2e-shared" | ||
|
|
||
| # See the e2e-gpu lane in e2e-selfhosted.yml for why the | ||
|
|
@@ -1164,12 +1168,11 @@ jobs: | |
|
|
||
| # WSL2 counterpart to e2e-gpu-nightly-strix: an Ubuntu-24.04 distro installed | ||
| # fresh inside WSL2 on a DevLab Dispatch pool Windows host — the same pool | ||
| # the per-PR `e2e-gpu-strix-windows`/`e2e-wsl` jobs use, not a separate | ||
| # `wsl`-labeled runner and not the `native` host e2e-gpu-nightly-strix-windows | ||
| # still runs on. Mirrors the per-PR `e2e-wsl` lane in e2e-selfhosted.yml step | ||
| # for step (see that job's header comment for the pool maintainer's | ||
| # sign-off), opting into the same @nightly large-model scenario as the other | ||
| # nightly lanes. | ||
| # `e2e-gpu-nightly-strix-windows` above and the per-PR `e2e-gpu-strix-windows`/ | ||
| # `e2e-wsl` jobs use, not a separate `wsl`-labeled runner. Mirrors the per-PR | ||
| # `e2e-wsl` lane in e2e-selfhosted.yml step for step (see that job's header | ||
| # comment for the pool maintainer's sign-off), opting into the same @nightly | ||
| # large-model scenario as the other nightly lanes. | ||
| e2e-wsl-nightly: | ||
| name: E2E tests (Strix Halo, WSL2, incl. nightly-only) | ||
| # See e2e-wsl in e2e-selfhosted.yml: every job here is cold (fresh distro, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nightly.yml:1129-1131and1138-1141still describe a persistent host. They say "The persistent runner workspace keeps those paths valid across scenarios and subsequent runs" and "this host was never actually warm for it". On the DevLab Dispatch pool each job gets a fresh runner that is destroyed afterwards, so nothing persists to a later run. This PR is what makes the comments false. Whene2e-gpu-strix-windowsmoved, its equivalent comment was rewritten (e2e-selfhosted.yml:897-908). Please reword both the same way.