Skip to content

Remove windows server 2022 from ci#1319

Open
ludfjig wants to merge 1 commit intohyperlight-dev:mainfrom
ludfjig:remove_ws2022
Open

Remove windows server 2022 from ci#1319
ludfjig wants to merge 1 commit intohyperlight-dev:mainfrom
ludfjig:remove_ws2022

Conversation

@ludfjig
Copy link
Contributor

@ludfjig ludfjig commented Mar 17, 2026

This PR removes windows server 2022 from CI. The main reason for doing so is that WS2022 does not support WHvResetPartition, which is an API we will start to rely on for windows.

Note that historic benchmarks show that 2022 is faster than 2025, for example benchmarks artifact from hyperlight release v0.13 show the following.

I'm not sure the reason for this. Doing some digging I am guessing that differences in building the images are probably the main cause for this. For 2022 we use Azure Pipelines - Windows 2022 which as far as I understand is based on https://github.com/actions/runner-images, whereas for 2025, we build our own from scratch based on 2025-datacenter-g2. Some differences could be:

  • VBS/HVCI: The 2022 image runs on Gen1 VMs (VBS not supported). The 2025 image uses 2025-datacenter-g2 where VBS is enabled by default, adding overhead to every VM exit.
  • Defender: The 2022 image fully disables Defender but we don't for the 2025 image
  • Power plan: The 2022 image sets high-performance. The 2025 image uses the OS default.

Another thing to point out is that our 2022 runners use Standard D8ads v5 and Standard D8ds v5 whereas 2025 uses Standard D16ads v5 and Standard D16s v5, but I would not expect this to cause any such difference.

It would be good to confirm these

=== AMD — Server 2022 vs 2025 (v0.13.0) ===
Benchmark                                                       2022 (ms)    2025 (ms)   2025 slower by
--------------------------------------------------------------------------------------------------------
function_call_serialization/deserialize_function_call              9.2128       9.5946            +4.1%
function_call_serialization/serialize_function_call                7.4765       8.0028            +7.0%
guest_calls/call_default                                           0.0371       0.0919          +147.5%
guest_calls/call_large                                             0.0363       0.1130          +210.9%
guest_calls/call_medium                                            0.0358       0.1092          +205.1%
guest_calls/call_small                                             0.0360       0.1087          +201.8%
guest_calls/call_with_host_function_default                        0.0623       0.1806          +189.9%
guest_calls/call_with_host_function_large                          0.0623       0.1911          +206.9%
guest_calls/call_with_host_function_medium                         0.0613       0.1856          +202.9%
guest_calls/call_with_host_function_small                          0.0616       0.1776          +188.1%
guest_calls/call_with_restore_default                              0.1548       0.2985           +92.8%
guest_calls/call_with_restore_large                               56.3103      61.7075            +9.6%
guest_calls/call_with_restore_medium                               5.6287       6.0249            +7.0%
guest_calls/call_with_restore_small                                0.5027       0.7817           +55.5%
guest_calls/different_thread                                       0.0640       0.1477          +130.8%
guest_calls/interrupt_latency                                      0.0336       0.1576          +368.6%
guest_functions_with_large_parameters/guest_call_with_large_parameters    2789.4683    6582.2529          +136.0%
sample_workloads/24k_in_8k_out_c                                   0.0604       0.1481          +145.4%
sample_workloads/24k_in_8k_out_rust                                0.0599       0.1550          +158.6%
sandboxes/create_initialized_and_drop_default                      4.5490       9.6005          +111.0%
sandboxes/create_initialized_and_drop_large                      317.2101     324.4793            +2.3%
sandboxes/create_initialized_and_drop_medium                      84.4375      91.3262            +8.2%
sandboxes/create_initialized_and_drop_small                       14.9649      22.1703           +48.1%
sandboxes/create_initialized_default                               3.6851       7.9838          +116.7%
sandboxes/create_initialized_large                               280.3505     289.7702            +3.4%
sandboxes/create_initialized_medium                               74.1300      81.0521            +9.3%
sandboxes/create_initialized_small                                12.8177      18.1391           +41.5%
sandboxes/create_uninitialized_and_drop_default                    0.9678       1.1021           +13.9%
sandboxes/create_uninitialized_and_drop_large                    301.0708     309.1558            +2.7%
sandboxes/create_uninitialized_and_drop_medium                    76.4937      77.6991            +1.6%
sandboxes/create_uninitialized_and_drop_small                      9.9913      10.2366            +2.5%
sandboxes/create_uninitialized_default                             0.9297       1.0019            +7.8%
sandboxes/create_uninitialized_large                             273.2304     275.5433            +0.8%
sandboxes/create_uninitialized_medium                             70.0725      70.6195            +0.8%
sandboxes/create_uninitialized_small                               9.4343       9.9035            +5.0%
shared_memory/copy_from_slice/1mb                                  0.0421       0.0425            +1.0%
shared_memory/copy_from_slice/64mb                                 7.4984       7.3961   -1.4% (faster)
shared_memory/copy_to_slice/1mb                                    0.0436       0.0436   -0.1% (faster)
shared_memory/copy_to_slice/64mb                                   7.9380       8.0312            +1.2%
shared_memory/fill/1mb                                             0.0414       0.0421            +1.7%
shared_memory/fill/64mb                                            5.3448       5.1563   -3.5% (faster)
snapshots/create_default                                           0.6090       0.7027           +15.4%
snapshots/create_large                                           240.1440     243.5992            +1.4%
snapshots/create_medium                                           61.4519      60.8286   -1.0% (faster)
snapshots/create_small                                             7.9811       8.0179            +0.5%
snapshots/restore_default                                          0.0876       0.1420           +62.1%
snapshots/restore_large                                          139.8504     156.2203           +11.7%
snapshots/restore_medium                                           7.3281       7.8804            +7.5%
snapshots/restore_small                                            0.4055       0.5124           +26.4%

=== Intel — Server 2022 vs 2025 (v0.13.0) ===
Benchmark                                                       2022 (ms)    2025 (ms)   2025 slower by
--------------------------------------------------------------------------------------------------------
function_call_serialization/deserialize_function_call             13.7435      14.6867            +6.9%
function_call_serialization/serialize_function_call               11.7201      12.2146            +4.2%
guest_calls/call_default                                           0.0703       0.1252           +78.2%
guest_calls/call_large                                             0.0714       0.1299           +81.8%
guest_calls/call_medium                                            0.0720       0.1332           +84.9%
guest_calls/call_small                                             0.0703       0.1310           +86.4%
guest_calls/call_with_host_function_default                        0.1066       0.2160          +102.5%
guest_calls/call_with_host_function_large                          0.1095       0.2389          +118.1%
guest_calls/call_with_host_function_medium                         0.1099       0.2402          +118.5%
guest_calls/call_with_host_function_small                          0.1069       0.2231          +108.7%
guest_calls/call_with_restore_default                              0.3072       0.4211           +37.1%
guest_calls/call_with_restore_large                               90.7141     116.1676           +28.1%
guest_calls/call_with_restore_medium                              10.7809      13.9802           +29.7%
guest_calls/call_with_restore_small                                1.2907       1.2146   -5.9% (faster)
guest_calls/different_thread                                       0.0947       0.1591           +68.1%
guest_calls/interrupt_latency                                      0.0473       0.1609          +240.1%
guest_functions_with_large_parameters/guest_call_with_large_parameters    3847.2519    6978.3283           +81.4%
sample_workloads/24k_in_8k_out_c                                   0.0961       0.1491           +55.1%
sample_workloads/24k_in_8k_out_rust                                0.0927       0.1445           +55.8%
sandboxes/create_initialized_and_drop_default                      5.3577       8.8508           +65.2%
sandboxes/create_initialized_and_drop_large                      366.8559     423.9386           +15.6%
sandboxes/create_initialized_and_drop_medium                      95.2664     112.4438           +18.0%
sandboxes/create_initialized_and_drop_small                       16.8000      24.4038           +45.3%
sandboxes/create_initialized_default                               4.5819       7.9857           +74.3%
sandboxes/create_initialized_large                               331.8065     354.0294            +6.7%
sandboxes/create_initialized_medium                               88.5752      96.9159            +9.4%
sandboxes/create_initialized_small                                15.3522      19.6374           +27.9%
sandboxes/create_uninitialized_and_drop_default                    1.1593       1.3144           +13.4%
sandboxes/create_uninitialized_and_drop_large                    349.9504     400.1288           +14.3%
sandboxes/create_uninitialized_and_drop_medium                    90.3956      99.3707            +9.9%
sandboxes/create_uninitialized_and_drop_small                     12.0447      13.5676           +12.6%
sandboxes/create_uninitialized_default                             1.1130       1.2499           +12.3%
sandboxes/create_uninitialized_large                             325.8819     346.6382            +6.4%
sandboxes/create_uninitialized_medium                             82.6306      89.5236            +8.3%
sandboxes/create_uninitialized_small                              11.0915      12.2664           +10.6%
shared_memory/copy_from_slice/1mb                                  0.0718       0.0714   -0.6% (faster)
shared_memory/copy_from_slice/64mb                                13.0746      15.7547           +20.5%
shared_memory/copy_to_slice/1mb                                    0.0744       0.0737   -1.0% (faster)
shared_memory/copy_to_slice/64mb                                  14.3308      17.5272           +22.3%
shared_memory/fill/1mb                                             0.0374       0.0359   -4.1% (faster)
shared_memory/fill/64mb                                            8.0912      11.3835           +40.7%
snapshots/create_default                                           0.6898       0.8863           +28.5%
snapshots/create_large                                           341.6443     366.7739            +7.4%
snapshots/create_medium                                           84.2382      91.5338            +8.7%
snapshots/create_small                                            10.6006      11.4260            +7.8%
snapshots/restore_default                                          0.1868       0.1927            +3.2%
snapshots/restore_large                                          151.6614     198.2303           +30.7%
snapshots/restore_medium                                          14.3601      17.7376           +23.5%
snapshots/restore_small                                            1.0710       1.0710            +0.0%

Updated: Indeed the image was the main culprit as you can see below. The 2025 azure pipeline image is almost on part with perf compared to 2022

=== Intel — WS2022 vs WS2025 Azure Pipelines image ===
Benchmark                                                                      2022 (ms) 2025-AP (ms)         Difference
------------------------------------------------------------------------------------------------------------------------
function_call_serialization/deserialize_function_call                            14.3761      15.5543              +8.2%
function_call_serialization/serialize_function_call                              11.9188      12.7160              +6.7%
guest_calls/call_default                                                          0.0725       0.0758              +4.6%
guest_calls/call_large                                                            0.0729       0.0770              +5.7%
guest_calls/call_medium                                                           0.0729       0.0770              +5.6%
guest_calls/call_small                                                            0.0727       0.0775              +6.5%
guest_calls/call_with_host_function_default                                       0.1126       0.1204              +6.9%
guest_calls/call_with_host_function_large                                         0.1124       0.1211              +7.8%
guest_calls/call_with_host_function_medium                                        0.1119       0.1220              +9.1%
guest_calls/call_with_host_function_small                                         0.1122       0.1260             +12.3%
guest_calls/call_with_restore_default                                             0.3175       0.3355              +5.7%
guest_calls/call_with_restore_large                                              91.0777     107.8129             +18.4%
guest_calls/call_with_restore_medium                                             11.7507      13.0839             +11.3%
guest_calls/call_with_restore_small                                               1.1505       1.2026              +4.5%
guest_calls/different_thread                                                      0.0933       0.1030             +10.4%
guest_calls/interrupt_latency                                                     0.0571       0.0475    -16.7% (faster)
guest_functions_with_large_parameters/guest_call_with_large_parameters         3900.1984    4684.4873             +20.1%
sample_workloads/24k_in_8k_out_c                                                  0.0996       0.1063              +6.7%
sample_workloads/24k_in_8k_out_rust                                               0.0993       0.1058              +6.5%
sandboxes/create_initialized_and_drop_default                                     5.9768       6.5628              +9.8%
sandboxes/create_initialized_and_drop_large                                     371.3387     400.6305              +7.9%
sandboxes/create_initialized_and_drop_medium                                     97.0073     105.8259              +9.1%
sandboxes/create_initialized_and_drop_small                                      17.7256      19.4701              +9.8%
sandboxes/create_initialized_default                                              4.7922       5.6452             +17.8%
sandboxes/create_initialized_large                                              343.4991     346.4078              +0.8%
sandboxes/create_initialized_medium                                              89.2919      91.5602              +2.5%
sandboxes/create_initialized_small                                               15.8775      17.0517              +7.4%
sandboxes/create_uninitialized_and_drop_default                                   1.2175       1.2755              +4.8%
sandboxes/create_uninitialized_and_drop_large                                   365.0983     380.0893              +4.1%
sandboxes/create_uninitialized_and_drop_medium                                   90.6570      96.5923              +6.5%
sandboxes/create_uninitialized_and_drop_small                                    12.3070      12.8387              +4.3%
sandboxes/create_uninitialized_default                                            1.2790       1.5575             +21.8%
sandboxes/create_uninitialized_large                                            334.0717     337.4903              +1.0%
sandboxes/create_uninitialized_medium                                            85.3904      85.3373     -0.1% (faster)
sandboxes/create_uninitialized_small                                             11.7366      11.5292     -1.8% (faster)
shared_memory/copy_from_slice/1mb                                                 0.0706       0.0739              +4.6%
shared_memory/copy_from_slice/64mb                                               12.6016      15.5809             +23.6%
shared_memory/copy_to_slice/1mb                                                   0.0767       0.0762     -0.6% (faster)
shared_memory/copy_to_slice/64mb                                                 14.3348      18.1202             +26.4%
shared_memory/fill/1mb                                                            0.0373       0.0361     -3.2% (faster)
shared_memory/fill/64mb                                                           8.4049      10.7703             +28.1%
snapshots/create_default                                                          0.7687       0.8790             +14.4%
snapshots/create_large                                                          347.1510     342.4304     -1.4% (faster)
snapshots/create_medium                                                          86.0649      87.1121              +1.2%
snapshots/create_small                                                           11.0103      11.2687              +2.3%
snapshots/restore_default                                                         0.1930       0.2032              +5.3%
snapshots/restore_large                                                         151.7616     189.0782             +24.6%
snapshots/restore_medium                                                         14.6476      16.7113             +14.1%
snapshots/restore_small                                                           1.0924       1.1737              +7.4%

@ludfjig ludfjig added area/infrastructure Concerns infrastructure rather than core functionality kind/refactor For PRs that restructure or remove code without adding new functionality. labels Mar 17, 2026
Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig ludfjig marked this pull request as ready for review March 18, 2026 23:13
@ludfjig ludfjig requested a review from Copilot March 19, 2026 00:32
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes Windows Server 2022 from CI workflows and standardizes Windows CI to use the WS2025 Hyper-V runner selection.

Changes:

  • Remove hyperv (WS2022) from hypervisor matrices (PR validation, daily benchmarks, release benchmarks).
  • Update reusable workflow runner selection logic to only treat hyperv-ws2025 as Windows.
  • Switch the release Windows build job runner from windows-2022 to windows-2025 and drop WS2022 benchmark artifacts from releases.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
.github/workflows/dep_run_examples.yml Updates runner label selection to no longer map hyperv to Windows/WS2022 pools.
.github/workflows/dep_build_test.yml Same runner selection update for the build/test reusable workflow.
.github/workflows/dep_benchmarks.yml Same runner selection update for the benchmarks reusable workflow.
.github/workflows/ValidatePullRequest.yml Removes hyperv from PR validation matrix.
.github/workflows/DailyBenchmarks.yml Removes hyperv from daily benchmarks matrix.
.github/workflows/CreateRelease.yml Moves Windows release build runner to windows-2025 and removes WS2022 benchmark artifacts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/infrastructure Concerns infrastructure rather than core functionality kind/refactor For PRs that restructure or remove code without adding new functionality.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants