[samtrion]

Problem

The latest mutation run of this repo (analyzer 59.95%: 793 killed, 491 survived, 2110 created) reports string mutants on the registered CodeAction titles and equivalence keys of the code-fix providers as survived. Counting the individual lines (OneTypePerFile 64/66/84/86, AvoidInvisibleCharacters 88/90/99/101, RequireNumericLiteralSuffix 71/73/96, UseIsNotNull 49 x2/55, UseIsNull 50/52, UseIsNotNullOverIsObject 54/56, AvoidRedundantCollectionClear 43/45, NamespaceMatchesFolder 45/47, SingleNamespacePerFile 68/70, AvoidRegionDirectives 54/57, NativeTypeCref 68, UseLangword 57, RequireCancellationCheck 54/70, RequireCancellationTokenParameter 58/61, plus the Fix All title in SequentialFixAllProvider line 53) this is about 34 string mutants, roughly half titles and half keys. The log only gives per-file survivor totals, so the exact string-only count is an estimate.

Root cause

The CSharpCodeFixVerifier wrappers only assert the fixed source, never which action produced it. Only RequireCancellationCheckCodeFixVerifier passes an equivalence key. Nothing asserts the lightbulb text (Title) or the Fix All grouping (EquivalenceKey). Setting CodeActionEquivalenceKey in the verifier would only kill key mutants (Roslyn fails when no action has that key) and never touches titles, so a test that inspects the registered actions directly is the primary fix.

Change

Tests only, no production code changes.

• Verifiers/CodeActionProbe.cs: runs an analyzer plus code fix over a source file in an AdhocWorkspace, registers the fix for the first diagnostic of a given id and returns Title [EquivalenceKey] of every registered action.
• CodeActionMetadataTests.cs: 18 tests pinning exact title and key per provider (NE0001 to NE0012, NE0014, NE0015). Covers both actions of NE0001 (rename/move), NE0009 and NE0014 (rename/text span), the two titles of NE0005 (C# 9 and pre-C# 9), and the keyless per-suffix actions of NE0012. Interpolated titles are asserted against concrete expected strings.
• FixAllRunner.FixAllTitleAsync plus one test in OneTypePerFileFixAllTests pin the SequentialFixAllProvider title (Fix all 'NE0001' occurrences).

Not done: a codeActionEquivalenceKey parameter on CSharpCodeFixVerifier. The probe asserts the keys directly, so the overload (which would be ambiguous next to the existing int? codeActionIndex overload) is not needed. NE0013 (UseNumericLiteralSuffixOverCast) is not in the finding and is not covered here.

Local check: dotnet test -f net10.0 on the unit project passes (475 tests). Replacing the NE0004 title with an empty string makes NE0004_UseIsNull fail as expected.

Before / After

metric	before	after
mutation score	59.95%	62.81% (+2.86 points)
killed	793	831 (+38)
survived	491	453 (-38)
created / compile errors / no coverage	2110 / 474 / 42	2110 / 474 / 42 (unchanged)
timeout	not recorded	5

Risk / what this does not change

• No production code is touched; only the unit test project grows (about 19 fast tests).
• The tests pin user-visible texts, so a deliberate title change now needs a test update. That is the intent.
• The Integration test project and the other mutation targets (Roslyn variants) are unchanged.

Measured on https://github.com/dailydevops/analyzer/actions/runs/40816302143 (workflow_dispatch on this branch, success, about 27 min). The +38 killed mutants match the estimate of about 34 string mutants plus a few incidental ones. Which individual mutants died was not broken down further, since the report only lists per-file totals.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.54%. Comparing base (8c86c2b) to head (b691b99).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #233      +/-   ##
==========================================
+ Coverage   94.72%   95.54%   +0.82%     
==========================================
  Files          48       48              
  Lines        2557     2557              
  Branches      449      449              
==========================================
+ Hits         2422     2443      +21     
+ Misses         58       41      -17     
+ Partials       77       73       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.