Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions docs/core/testing/microsoft-testing-platform-cli-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) CLI options reference
description: Find platform and extension command-line options for MTP in one place.
author: Evangelink
ms.author: amauryleve
ms.date: 09/02/2026
ms.date: 09/11/2026
ai-usage: ai-assisted
---

Expand Down Expand Up @@ -142,7 +142,12 @@ This article gives a central entry point for MTP command-line options.

- **`--minimum-expected-tests`**

Specifies the minimum number of tests that must run. When the run executes fewer tests, including zero, it exits with code `9`. An explicit minimum supersedes `--zero-tests-policy`.
Specifies a positive minimum number of tests that must run. When the run executes fewer tests, including zero, it exits with code `9`. An explicit minimum supersedes `--zero-tests-policy`.

With `dotnet test`, this option applies to the whole run when it's specified before `--`, and to each test module when it's specified after `--`. For more information, see [Whole-run and per-module minimums](../tools/dotnet-test-mtp.md#whole-run-and-per-module-minimums).

> [!NOTE]
> `--minimum-expected-tests 0` is invalid. To suppress the zero-tests exit code, use `--ignore-exit-code 8`.

- **`--no-banner`**

Expand Down Expand Up @@ -182,7 +187,7 @@ This article gives a central entry point for MTP command-line options.
Controls whether a run that executes no tests because every test was skipped is treated as a failure. Valid values are `allow-skipped` (default) and `strict`. With `allow-skipped`, an all-skipped run succeeds. With `strict`, it fails with exit code `8`. An explicit `--minimum-expected-tests` value supersedes this policy and uses exit code `9` when the minimum isn't met.

> [!NOTE]
> This option is available in MTP starting with version 2.3.0.
> This option is available in MTP starting with version 4.3.0. With `dotnet test`, pass the option after `--` to forward it to each test module. When you don't set a global minimum, the .NET 11 SDK determines the whole-run zero-tests verdict separately. For more information, see [Whole-run and per-module minimums](../tools/dotnet-test-mtp.md#whole-run-and-per-module-minimums).

## Extension options by scenario

Expand Down
15 changes: 11 additions & 4 deletions docs/core/testing/microsoft-testing-platform-troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) troubleshooting
description: Troubleshoot MTP issues, exit codes, and known problems.
author: Evangelink
ms.author: amauryleve
ms.date: 09/02/2026
ms.date: 09/11/2026
ai-usage: ai-assisted
---

Expand All @@ -25,18 +25,25 @@ MTP uses known exit codes to communicate test failure or app errors. Exit codes
| `5` | The exit code `5` indicates that the command-line arguments passed to the test app were invalid. |
| `6` (no longer used) | Exit code `6` is no longer produced by the platform; it previously indicated that the test session was using a non-implemented feature. |
| `7` | The exit code `7` indicates that a test session was unable to complete successfully, and likely crashed. It's possible that this was caused by a test session that was run via a test controller's extension point. |
| `8` | The exit code `8` indicates that the test session ran zero tests under the strict `--zero-tests-policy`. |
| `9` | The exit code `9` indicates that the run executed fewer tests than `--minimum-expected-tests` requires, including zero tests. |
| `8` | The exit code `8` indicates that the test session discovered no tests, or that every selected test was skipped under the strict `--zero-tests-policy`. |
| `9` | The exit code `9` indicates that the run executed fewer tests than an explicit `--minimum-expected-tests` value requires, including zero tests. |
| `10` | The exit code `10` indicates that the test adapter, Testing.Platform Test Framework, MSTest, NUnit, or xUnit, failed to run tests for an infrastructure reason unrelated to the test's self. An example is failing to create a fixture needed by tests. |
| `11` | The exit code `11` indicates that the test process will exit if dependent process exits. |
| `12` | The exit code `12` indicates that the test session was unable to run because the client does not support any of the supported protocol versions. |
| `13` | The exit code `13` indicates that the test session was stopped due to reaching the specified number of maximum failed tests using `--maximum-failed-tests` command-line option. For more information, see [the Options section in MTP CLI options reference](microsoft-testing-platform-cli-options.md) |
| `14` | The exit code `14` indicates that a compatible coverage collector published a failed coverage threshold evaluation. |

An explicit `--minimum-expected-tests` value supersedes `--zero-tests-policy`. Without the minimum option, strict zero-test handling continues to use exit code `8`.
An explicit `--minimum-expected-tests` value supersedes `--zero-tests-policy`. Without the minimum option, strict zero-test handling continues to use exit code `8`. Exit codes `8` and `9` remain distinct so that an unmet minimum isn't confused with a module that ran no tests.

To enable verbose logging and troubleshoot issues, see [Diagnostic logging](#diagnostic-logging).

### Zero tests in a multi-module run

When `dotnet test` runs several test modules, exit code `8` is a per-module signal, while the zero-tests verdict for the whole run is decided once from the aggregated results. A single empty module therefore doesn't fail the whole run, although the module keeps its `Exit code: 8` diagnostic in the output. When you don't set a global minimum, an all-skipped whole run is treated as a zero-test run regardless of the per-module `--zero-tests-policy` value. For more information, see [Whole-run and per-module minimums](../tools/dotnet-test-mtp.md#whole-run-and-per-module-minimums).

> [!NOTE]
> This whole-run zero-tests verdict requires the .NET 11 SDK or a later version.

### Ignore specific exit codes

MTP is designed to be strict by default but allows for configurability. As such, it's possible for users to decide which exit codes should be ignored (an exit code of `0` will be returned instead of the original exit code).
Expand Down
52 changes: 50 additions & 2 deletions docs/core/tools/dotnet-test-mtp.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: dotnet test command with Microsoft.Testing.Platform (MTP)
description: The dotnet test command is used to execute unit tests in a given project using MTP.
ms.date: 09/02/2026
ms.date: 09/11/2026
ai-usage: ai-assisted
---
# dotnet test with Microsoft.Testing.Platform (MTP)
Expand Down Expand Up @@ -98,7 +98,12 @@ With MTP, `dotnet test` operates faster than with VSTest. The test-related argum

- **`--minimum-expected-tests <NUMBER>`**

Specifies the minimum number of tests that must be executed. If the actual number of tests is less than the specified minimum, the test run fails with exit code 9. For more information about exit codes, see [MTP exit codes](../testing/microsoft-testing-platform-troubleshooting.md#exit-codes).
Specifies a positive minimum number of tests for the whole run. If the aggregated test count is less than the specified minimum, the test run fails with exit code 9. The global count includes skipped tests. For more information about exit codes, see [MTP exit codes](../testing/microsoft-testing-platform-troubleshooting.md#exit-codes).

Because this option appears before `--`, it's a global (whole-run) option. To require a minimum for each test module instead, pass the option after `--` so that it's forwarded to every test module. For more information, see [Whole-run and per-module minimums](#whole-run-and-per-module-minimums).

> [!NOTE]
> The global minimum requires the .NET 10 SDK (10.0.100) or a later version.

- [!INCLUDE [arch](includes/cli-arch.md)]

Expand Down Expand Up @@ -190,6 +195,43 @@ The preceding example requires the [`Microsoft.Testing.Extensions.TrxReport`](ht

The same parser behavior applies to `dotnet run` and `dotnet build`. For a detailed example, see [Forward arguments to the application](dotnet-run.md#forward-arguments-to-the-application) in the `dotnet run` reference.

## Whole-run and per-module minimums

For `--minimum-expected-tests`, the `--` separator determines the option's scope:

- Arguments *before* `--` are global. The `dotnet test` orchestrator interprets them for the whole run.
- Arguments *after* `--` are local. `dotnet test` forwards them to each test module, so each module applies them independently.

Because `--minimum-expected-tests` is available in both scopes, you can require a minimum for the whole run, for each module, or both:

```dotnetcli
dotnet test --minimum-expected-tests 5 -- --minimum-expected-tests 2
```

The preceding command requires at least 5 tests across the whole run and at least 2 tests in each test module.

The two scopes count skipped tests differently:

| Scope | Do skipped tests count toward the minimum? |
| --- | --- |
| Global | Yes. The `dotnet test` aggregated total includes skipped tests. |
| Per module | No. MTP excludes skipped tests from the number of tests that ran. |

Starting with the .NET 11 SDK, the zero-tests verdict for the whole run is decided once from the aggregated results. A module that matches no tests, for example because of `--test-modules` or a global `--filter`, exits with code 8 (`ZeroTests`), but that code is normalized to success before the results are aggregated. As a result, a single empty module doesn't fail the whole run, although the module keeps its `Exit code: 8` diagnostic in the output for visibility.

MTP 4.3.0 and later versions provide `--zero-tests-policy <allow-skipped|strict>`. The default value, `allow-skipped`, lets an all-skipped module succeed. The `strict` value treats skipped tests as not run, so an all-skipped module exits with code 8. Pass the option after `--` to forward it to each test module:

```dotnetcli
dotnet test -- --zero-tests-policy strict
```

When you don't set a global minimum, the .NET 11 SDK determines the whole-run zero-tests verdict separately. An all-skipped whole run exits with code 8 regardless of the per-module `--zero-tests-policy` value.

When you specify `--minimum-expected-tests` and the minimum isn't met, the run fails with exit code 9 (`MinimumExpectedTestsPolicyViolation`). This code is distinct from 8 so that a stricter global or per-module minimum isn't confused with an empty module. For a per-module minimum to return code 9 when the module runs zero tests, the test module must use MTP 4.4.0 or a later version.

> [!NOTE]
> `--minimum-expected-tests 0` is invalid. To suppress the zero-tests exit code, use `--ignore-exit-code 8`.

## Examples

- Run the tests in the project or solution in the current directory:
Expand Down Expand Up @@ -246,6 +288,12 @@ The same parser behavior applies to `dotnet run` and `dotnet build`. For a detai
dotnet test --minimum-expected-tests 10
```

- Require at least 5 tests across the whole run and at least 2 tests in each test module:

```dotnetcli
dotnet test --minimum-expected-tests 5 -- --minimum-expected-tests 2
```

- Run the tests in the `TestProject` project, providing the `-bl` (binary log) argument to `msbuild`:

```dotnetcli
Expand Down
Loading