D1.6 MTP config: dotnet test reports zero tests locally while CI reports 21, same SDK

[ptr727]

Found migrating ptr727/NxWitness onto WORKFLOW.md D1.6 (ptr727/NxWitness#549). Filing before the remaining dotnet repos migrate, because the failure is quiet in the direction that matters: it looks like a broken repo rather than a broken driver.

What happens

The D1.6 configuration exactly as specified: global.json with {"test": {"runner": "Microsoft.Testing.Platform"}}, Microsoft.Testing.Extensions.CodeCoverage 18.9.0, xunit.v3 4.0.0, xunit.runner.visualstudio dropped, Microsoft.NET.Test.Sdk 18.9.0 retained.

In CI it works. From the job log on resync/dotnet-mtp at dc48c3b:

Test run summary: Passed!
  total: 21
  succeeded: 21

On my machine, a pristine checkout of that same commit reports zero.

$ dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage
CreateMatrixTests/bin/Debug/net10.0/CreateMatrixTests.dll (net10.0) Zero tests ran
Exit code: 5

Same SDK on both sides. The CI log shows .NET Core SDK with version '10.0.400' is already installed and The dotnet runtime version is '10.0.11', which is byte for byte what dotnet --version and the build report locally. So this is not the version floor D1.6 already warns about, and it is not the 18.0.x TypeLoadException case either.

The part that localizes it

The test application itself is fine. Run directly, it discovers and passes everything:

$ dotnet CreateMatrixTests/bin/Debug/net10.0/CreateMatrixTests.dll
Test run summary: Passed! - .../CreateMatrixTests.dll (net10.0|x64)
  total: 21

Note the target string. The direct run reports net10.0|x64. Every failing dotnet test run reports net10.0, with no architecture. CI's passing run also reports net10.0|x64. So the driver is failing to resolve an architecture and then discovering nothing, rather than the test project failing to register its framework.

Ruled out

• Stale bin/obj. Reproduces after a full wipe and dotnet restore --force.
• Incremental versus clean build, and a preceding explicit dotnet build (which is what CI's "Check code style step" does before the test step).
• A worktree-specific artifact. Reproduces in a fresh git worktree add --detach of the same SHA that had never been built in.
• dotnet test argument shape. dotnet test, dotnet test <solution>, dotnet test <test csproj>, --no-build, and -- --results-directory all give zero.
• Microsoft.NET.Test.Sdk. Removing it does not fix discovery, it breaks the build outright, so it is required rather than dead weight.
• <UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>. This one is worth recording: it genuinely converts the app from xunit's own console runner into an MTP host (before it, the built executable rejects --results-directory as an unknown option; after, it accepts MTP options and prints the MTP help). But it does not fix dotnet test, and it broke the --coverage path in one of my runs. D1.6 does not mention the property, and given that the executable is not an MTP host without it, it is worth deciding deliberately whether the fleet wants it rather than leaving it unset by default.

Why it matters for the rollout

1. A zero-test run is not silent, which is good, but it is misleading. Exit 5 and Zero tests ran will read to the next migrator as "my migration is wrong", and they will go change their csproj, as I did, when the configuration is correct and CI proves it.
2. D1.6 states the CI invocation and nothing about the local one. A migrating repo's contributors need a local command that works, and the one this repo's own docs asserted (plain dotnet test) is unverified anywhere and demonstrably fails on at least one machine. Worth D1.6 saying what the supported local invocation is, and worth it naming the direct dotnet <TestProject>.dll run as the fallback that isolates driver from project.
3. The net10.0 versus net10.0|x64 difference is a cheap diagnostic and is not written down anywhere. It separates "the driver did not resolve an architecture" from "the tests did not register", which are the two failures that present identically.

Happy to test a hypothesis on this machine if it is useful, since it reproduces reliably here and apparently not on the runner.

[ptr727]

A negative data point, in case it helps localize this: ptr727/PlexCleaner runs the same D1.6 configuration on the same host and SDK and does not reproduce the zero-tests failure.

Migrated in ptr727/PlexCleaner#943, with global.json {"test": {"runner": "Microsoft.Testing.Platform"}}, xunit.v3 4.0.0, xunit.runner.visualstudio dropped, Microsoft.NET.Test.Sdk 18.9.0 retained, and Microsoft.Testing.Extensions.CodeCoverage 18.10.0 rather than 18.9.0. Locally, the exact D1.6 invocation:

$ dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage
Running tests from .../PlexCleanerTests.dll (net10.0|x64)
Test run summary: Passed!
  total: 239
  succeeded: 239

Note the target string is net10.0|x64, the architecture-resolved form, from dotnet test rather than from a direct run of the test assembly. So on this host the driver resolves an architecture and discovers tests normally, which is the behavior your CI sees and your local run does not.

Two differences from NxWitness worth checking, since they are the only ones I can see between the two setups:

• Extension version. PlexCleaner is on 18.10.0 (MTP 2.3.3), NxWitness on 18.9.0 (MTP 2.3.0). If the resolution is version-sensitive, that is a cheap thing to bisect.
• Project count and layout. PlexCleaner has one test project in a solution with five projects, referenced through PlexCleaner.slnx.

Both repos resolve Microsoft.NET.Test.Sdk 18.9.0, so that is common ground rather than a difference.

I have not tried to reproduce your failure, so this only narrows it to something other than the D1.6 configuration itself.