Fix and re-enable skipped and flaky tests

[rosebyte]

Restore test coverage lost to temporary skips, known failures, missing test assets, and unresolved platform differences.

This issue tracks the work to investigate the remaining exclusions, fix their underlying causes, and re-enable the affected tests. Existing issues should remain the home for detailed investigation and fixes, with progress tracked here.

Skipped tests

• Tests disabled for the helix integration #5845: Resolve the remaining Helix-related exclusions, including TestDownloadError, MatrixFactorizationSimpleTrainAndPredict, quantile regression, and factorisation-machine initialisation tests. Reconcile the older checklist with current source.
• PR Validation Flaky Test #6684: Re-enable MulticlassLRTest on affected architectures and TestGridSearchTrialRunner2. The latter currently has an unconditional skip despite its architecture-specific explanation. Investigate the related failure reports in Failure to load lib_lightgbm in Microsoft.ML.Fairlearn.Tests.GridSearchTest.TestGridSearchTrialRunner2 #6841, ArgumentOutOfRangeException when running Microsoft.ML.Fairlearn.Tests.GridSearchTest.TestGridSearchTrialRunner2 #6842, and Microsoft.ML.Fairlearn.Tests.GridSearchTest.TestGridSearchTrialRunner2 failed with bestResult.Metric comparison #6843 without assuming they share a root cause.
• TestBackAndForthConversionWithAlphaNoInterleave disabled #6043: Restore the image conversion tests excluded on ARM for stability.
• Add Windows ARM baseline coverage for SymSGD predictor test #7688: Add the missing Windows ARM baseline and remove the corresponding early return in BinaryClassifierSymSgdTest. Investigate its separate Linux early return as well.
• Microsoft.ML.TorchSharp.Tests.QATests.TestSimpleQA followed by process killed / return 137 #6978: Address the memory requirements of QATests.TestSimpleQA and restore automated coverage.
• Need real time series dataset for tests #1120: Provide suitable time-series data for EntryPointSsaChangePoint and EntryPointSsaSpikeDetector; address the equivalent dataset-related skip in SavePipeSsaSpike.
• BenchmarksProjectIsNotBroken fails on x86 #1391: Resolve or explicitly retire the outstanding x86 exclusion for BenchmarksProjectIsNotBroken, according to the supported benchmark configurations.

Flaky tests

• TestAppendMany_SizeMoreThanMaxBufferCapacity triggers OOM killer #7419: TestAppendMany_SizeMoreThanMaxBufferCapacity triggers the OOM killer.
• AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async fails in CI #7418: AutoML experiment cancellation test fails in CI.
• Resource contention in tests DownloadImageSet #7393: Concurrent DownloadImageSet calls contend for shared files.
• AutoML tests writing test output outside of a test which causes test process to fail with no failing test cases #7067: AutoML writes test output after the test has finished.
• Microsoft.ML.Tests.TrainerEstimators.TrainerEstimators.TestEstimatorMulticlassNaiveBayesTrainer failed with assert #7060: Assertion failure in TestEstimatorMulticlassNaiveBayesTrainer.
• Microsoft.ML.Tests.OnnxConversionTest.NaiveCalibratorOnnxConversionTest Failed #7057: Incorrect results in NaiveCalibratorOnnxConversionTest.
• Possible deadlock when running Microsoft.ML.Tests #6996: Possible deadlock in Microsoft.ML.Tests.
• SqlException Timeout when running DatabaseLoaderTests #6971: SQL connection timeouts in DatabaseLoaderTests.
• Microsoft.ML.Tests crashing after [LightGBM] [Warning] bad allocation #6961: LightGBM allocation failure and process crash.
• AccessViolationException when running Microsoft.ML.Scenarios.ScenariosTests.TestRegressionScenario #6960: Access violation during TestRegressionScenario.
• FastRank MSM sparse test baselines are suspicious  #2299: TBD
• Test system doesn't actually run in MTA apartment state #772

[github-actions]

🔍 No similar issues found with high confidence.

---

AI-powered similar issue detection