Skip to content

Document dotnet test whole-run zero-tests verdict and global/per-module --minimum-expected-tests - #55938

Open
meaghanlewis with Copilot wants to merge 4 commits into
mainfrom
copilot/document-dotnet-test-verdicts
Open

Document dotnet test whole-run zero-tests verdict and global/per-module --minimum-expected-tests#55938
meaghanlewis with Copilot wants to merge 4 commits into
mainfrom
copilot/document-dotnet-test-verdicts

Conversation

Copilot AI commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

dotnet test on the Microsoft.Testing.Platform path gained a whole-run "zero tests ran" verdict plus a global vs. per-module minimum-expected-tests model (shipped in the .NET 11 SDK, 11.0.1xx). None of it was on Learn.

docs/core/tools/dotnet-test-mtp.md

  • --minimum-expected-tests is now described as a whole-run option, with a pointer to the per-module form and an SDK availability note.

  • New Whole-run and per-module minimums section covering:

    • the -- argument split (before = global/orchestrator, after = local/forwarded to each module)
    • the whole-run verdict being decided once from the total test count, with a module's exit code 8 (ZeroTests) normalized to success before aggregation while keeping its Exit code: 8 diagnostic
    • exit code 9 (MinimumExpectedTestsPolicyViolation) for an unmet explicit minimum
    • --minimum-expected-tests 0 is not an alias for --ignore-exit-code 8
  • New example:

    dotnet test --minimum-expected-tests 5 -- --minimum-expected-tests 2
    

docs/core/testing/microsoft-testing-platform-troubleshooting.md

  • Exit code table entry for 9 now says explicit --minimum-expected-tests, and a follow-up sentence states why 8 and 9 stay distinct.
  • New Zero tests in a multi-module run subsection summarizing per-module vs. whole-run behavior and cross-linking to the command reference (kept short to avoid duplicating the prose there).

docs/core/testing/microsoft-testing-platform-cli-options.md

  • --minimum-expected-tests entry notes the global vs. per-module scope under dotnet test and the --ignore-exit-code 8 clarification.

Content is derived from the issue description (microsoft/testfx#9709, dotnet/sdk#55362); the behavioral claims and the SDK version gating are worth a maintainer check.


Internal previews

File Preview link
docs/core/testing/microsoft-testing-platform-cli-options.md Learn preview
docs/core/testing/microsoft-testing-platform-troubleshooting.md Learn preview
docs/core/tools/dotnet-test-mtp.md Learn preview

Build report

Copilot AI lite review requested due to automatic review settings September 10, 2026 16:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

…xpected-tests

Co-authored-by: meaghanlewis <10103121+meaghanlewis@users.noreply.github.com>
…ated prose

Co-authored-by: meaghanlewis <10103121+meaghanlewis@users.noreply.github.com>
Copilot AI changed the title [WIP] Document dotnet test zero-tests verdict and minimum expected tests Document dotnet test whole-run zero-tests verdict and global/per-module --minimum-expected-tests Sep 10, 2026
Copilot AI requested a review from meaghanlewis September 10, 2026 16:31
@meaghanlewis
meaghanlewis marked this pull request as ready for review September 10, 2026 19:47
@meaghanlewis
meaghanlewis requested review from a team as code owners September 10, 2026 19:47
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
Member

Validated the documentation against the current dotnet/sdk and microsoft/testfx implementations and pushed corrections in 550a670.

Key corrections:

  • Global --minimum-expected-tests aggregation is available in .NET SDK 10.0.100; .NET 11 introduced the separate whole-run zero-tests verdict.
  • The global minimum includes skipped tests, while a forwarded per-module minimum excludes skipped tests.
  • A zero-test per-module minimum reliably returns exit code 9 with Microsoft.Testing.Platform 4.4.0 or later.
  • --zero-tests-policy <allow-skipped|strict> was added in MTP 4.3.0, defaults to allow-skipped, and applies per module when forwarded after --.
  • Without a global minimum, the .NET 11 SDK currently treats an all-skipped whole run as a zero-test run regardless of the forwarded per-module policy.
  • --minimum-expected-tests 0 is invalid; the minimum must be positive.

I also corrected the existing MTP version note for --zero-tests-policy and clarified exit code 8 for no-discovery versus all-skipped strict-policy runs.

@meaghanlewis

Copy link
Copy Markdown
Contributor

@Evangelink thanks for your corrections.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document dotnet test whole-run zero-tests verdict and global/per-module --minimum-expected-tests (.NET 11 SDK)

4 participants