[msbuild] refactor `BenchmarkDotNet.Weaver.targets` to support mobile platforms by jonathanpeppers · Pull Request #2931 · dotnet/BenchmarkDotNet

jonathanpeppers · 2025-12-16T15:38:38Z

Context: #2929
Context: https://github.com/dotnet/android/blob/4aa9af89102af2e745a8507992187d3c5993d638/Documentation/guides/MSBuildBestPractices.md

In initially testing BenchmarkDotNet with .NET MAUI, I found that the weaver was not being executed because the Publish target is not called during the build process for Android and iOS projects. I think the target could actually run much sooner during builds and achieve better results.

To address this, I refactored the BenchmarkDotNet.Weaver.targets file to run after CoreCompile on the @(IntermediateAssembly) in the obj directory. This is similar to what other targets do, such as the XamlC compiler:

https://github.com/dotnet/maui/blob/8224becbb3a8a6bb1caaca4bbe70c56e88875506/src/Controls/src/Build.Tasks/nuget/buildTransitive/netstandard2.0/Microsoft.Maui.Controls.targets#L213-L255

Other general MSBuild improvements:

Defined an MSBuild property for everything that seems useful. This allows consuming projects to configure the behavior (or workarounds!) as needed.
Made the MSBuild target incremental by using inputs and outputs. If the .dll file is an input, we can write a .stamp file as an output to indicate that the weaver has already been run for that assembly. If the .dll file changes, the weaver will run again.

I tested this change with a .NET MAUI on 4 platforms and also the existing BenchmarkDotNet.Samples console app.

… platforms Context: dotnet#2929 Context: https://github.com/dotnet/android/blob/4aa9af89102af2e745a8507992187d3c5993d638/Documentation/guides/MSBuildBestPractices.md In initially testing BenchmarkDotNet with .NET MAUI, I found that the weaver was not being executed because the `Publish` target is not called during the build process for Android and iOS projects. I think the target could actually run much sooner during builds and achieve better results. To address this, I refactored the `BenchmarkDotNet.Weaver.targets` file to run after `CoreCompile` on the `@(IntermediateAssembly)` in the `obj` directory. This is similar to what other targets do, such as the XamlC compiler: https://github.com/dotnet/maui/blob/8224becbb3a8a6bb1caaca4bbe70c56e88875506/src/Controls/src/Build.Tasks/nuget/buildTransitive/netstandard2.0/Microsoft.Maui.Controls.targets#L213-L255 Other general MSBuild improvements: * Defined an MSBuild property for everything that seems useful. This allows consuming projects to configure the behavior (or workarounds!) as needed. * Made the MSBuild target incremental by using inputs and outputs. If the `.dll` file is an input, we can write a `.stamp` file as an output to indicate that the weaver has already been run for that assembly. If the `.dll` file changes, the weaver will run again. I tested this change with a .NET MAUI on 4 platforms and also the existing `BenchmarkDotNet.Samples` console app.

timcassell · 2025-12-16T16:13:24Z

Thanks. I initially tried to do it after CoreCompile target, but it had some issues that I don't recall. Looks like this fixes that.

DrewScoggins · 2025-12-18T19:18:17Z

I think this might have broken us. We have now started to see NETSDK1152 in our performance runs. Chatting with Copilot, it believes that the issue is from this change. This is its logic

Why It Breaks

BenchmarkDotNet passes [/p:ArtifactsPath="..."] to MSBuild for the autogenerated project
The SDK computes IntermediateOutputPath based on [ArtifactsPath], placing the weaved DLL in [{ArtifactsPath}/obj/.../]
BenchmarkDotNet also sets custom OutDir/OutputPath/PublishDir paths
During publish, the SDK's conflict resolution sees two DLLs with the same filename:
The intermediate assembly in obj/
The output assembly in the publish directory
This triggers NETSDK1152

[2025/12/18 10:33:54][INFO] /home/helixbot/work/942D0884/w/A92E09B4/e/performance/tools/dotnet/x64/sdk/11.0.100-alpha.1.25618.105/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.ConflictResolution.targets(112,5): error NETSDK1152: Found multiple publish output files with the same relative path: /home/helixbot/work/942D0884/w/A92E09B4/e/performance/artifacts/obj/BenchmarkDotNet.Autogenerated/Release/MicroBenchmarks-Job-ALNHUU-1.dll, /home/helixbot/work/942D0884/w/A92E09B4/e/performance/artifacts/bin/MicroBenchmarks/Release/net11.0/MicroBenchmarks-Job-ALNHUU-1/bin/Release/net11.0/publish/MicroBenchmarks-Job-ALNHUU-1.dll. [/home/helixbot/work/942D0884/w/A92E09B4/e/performance/artifacts/bin/MicroBenchmarks/Release/net11.0/MicroBenchmarks-Job-ALNHUU-1/BenchmarkDotNet.Autogenerated.csproj::TargetFramework=net11.0]

DrewScoggins · 2025-12-18T19:23:36Z

@timcassell @jonathanpeppers

jonathanpeppers · 2025-12-18T22:11:35Z

@DrewScoggins can you share a .binlog of the error? https://aka.ms/binlog

timcassell · 2025-12-18T22:11:40Z

@DrewScoggins How to repro? We have some toolchains that do publish, but the error doesn't occur in our tests. I'd like to add a test case for that if possible.

Also, can you see if #2935 fixes it?

DrewScoggins · 2025-12-18T23:04:41Z

@jonathanpeppers Here is the binlog, https://pvscmdupload.z22.web.core.windows.net/docs/MicroBenchmarks-Job-ALNHUU-1-build-no-restore.binlog

DrewScoggins · 2025-12-18T23:11:20Z

@timcassell Right now, we have only been able to repro it in our internal CI :( We will try tomorrow to get you something more targeted.

jonathanpeppers · 2025-12-20T17:07:42Z

@DrewScoggins that .binlog above has 0 errors, and it seems like it successfully weaves the assembly:

Do you have a .binlog that includes the error?

I also authored this PR, so you should be able to get back the previous behavior (untested):

<PropertyGroup>
    <BenchmarkDotNetWeaveAssembliesAfterTargets>Build</BenchmarkDotNetWeaveAssembliesAfterTargets>
    <BenchmarkDotNetWeaveAssembliesBeforeTargets>Publish</BenchmarkDotNetWeaveAssembliesBeforeTargets>
    <BenchmarkDotNetWeaveAssemblyPath>$(TargetDir)$(TargetFileName)</BenchmarkDotNetWeaveAssemblyPath>
</PropertyGroup>

(Try putting in a Directory.Build.targets, so $(TargetDir)$(TargetFileName) will be set)

Share either a .binlog with the error (or if you have trouble with the workaround).

timcassell · 2026-02-24T10:03:24Z

with .NET MAUI, I found that the weaver was not being executed because the Publish target is not called during the build process for Android and iOS projects.

I thought BeforeTargets="Publish" just enforced that Publish runs after this target if it's going to run, not that Publish is required to run.

The change to run after CoreCompile causes issues that were revealed by latest AsmResolver (Washi1337/AsmResolver#716). Is there a way to make it run after Build and before Publish more reliably? @jonathanpeppers

[Edit] Would AfterTargets="Build" BeforeTargets="AfterBuild" work?

js6pak · 2026-02-24T16:08:07Z

@timcassell Why not just pass ReferencePath to the task?

jonathanpeppers · 2026-02-24T16:14:20Z

@timcassell if someone can share a project that doesn't work, that would help me understand what is going on.

The changes here run after CoreCompile now, so that is before Build is even done. But that is what I would recommend, so that you update the .dll file in the $(IntermediateOutputPath) right when Roslyn has finished with it.

timcassell · 2026-02-24T16:43:51Z

@timcassell Why not just pass ReferencePath to the task?

I don't know what you mean by that. We already pass the path to the assembly. The issue is the references of the target assembly can be nuget packages or otherwise that we cannot resolve after CoreCompile.

@timcassell if someone can share a project that doesn't work, that would help me understand what is going on.

The changes here run after CoreCompile now, so that is before Build is even done. But that is what I would recommend, so that you update the .dll file in the $(IntermediateOutputPath) right when Roslyn has finished with it.

You can try it out with the update-asmresolver branch. If you simply build BenchmarkDotNet.slnx you will see the warnings. CI run showing the failures due to the issue: https://github.com/dotnet/BenchmarkDotNet/actions/runs/22322058229

jonathanpeppers · 2026-02-24T17:08:15Z

https://github.com/dotnet/BenchmarkDotNet/actions/runs/22322058229

Is there a specific line I should look at? This has unrelated errors.

timcassell · 2026-02-24T17:10:33Z

https://github.com/dotnet/BenchmarkDotNet/actions/runs/22322058229

Is there a specific line I should look at? This has unrelated errors.

The test failures are downstream issues from the weaver failing. You will see the real error (as a warning) when you build the project (the error is pasted in Washi1337/AsmResolver#716).

timcassell · 2026-02-24T17:12:50Z

https://github.com/dotnet/BenchmarkDotNet/actions/runs/22322058229

Is there a specific line I should look at? This has unrelated errors.

Actually if you look at the build step (not the in-tests-core step) you will see the error (line 80 in test-windows-core).

jonathanpeppers · 2026-02-24T17:19:06Z

Does the "weaver" need to be able to find all assembly references? Because they won't be found in the same folder.

C:\Users\runneradmin\.nuget\packages\benchmarkdotnet.weaver\0.16.0-ci-2\buildTransitive\netstandard2.0\BenchmarkDotNet.Weaver.targets(20,5): warning : Assembly weaving failed. Benchmark methods found requiring NoInlining: False. Error: [D:\a\BenchmarkDotNet\BenchmarkDotNet\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj]
C:\Users\runneradmin\.nuget\packages\benchmarkdotnet.weaver\0.16.0-ci-2\buildTransitive\netstandard2.0\BenchmarkDotNet.Weaver.targets(20,5): warning : System.IO.FileNotFoundException: Could not find the file containing the declaring assembly BenchmarkDotNet, Version=0.16.0.0, Culture=neutral, PublicKeyToken=aa0ca2f9092cefc4 (0x23000002) of type JetBrains.Annotations.PublicAPIAttribute [D:\a\BenchmarkDotNet\BenchmarkDotNet\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj]
C:\Users\runneradmin\.nuget\packages\benchmarkdotnet.weaver\0.16.0-ci-2\buildTransitive\netstandard2.0\BenchmarkDotNet.Weaver.targets(20,5): warning :    at AsmResolver.DotNet.TypeDescriptorExtensions.<Resolve>g__ThrowStatusError|4_0(ITypeDescriptor type, ResolutionStatus status) [D:\a\BenchmarkDotNet\BenchmarkDotNet\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj]
C:\Users\runneradmin\.nuget\packages\benchmarkdotnet.weaver\0.16.0-ci-2\buildTransitive\netstandard2.0\BenchmarkDotNet.Weaver.targets(20,5): warning :    at AsmResolver.DotNet.TypeDescriptorExtensions.Resolve(ITypeDescriptor type, RuntimeContext context) [D:\a\BenchmarkDotNet\BenchmarkDotNet\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj]
C:\Users\runneradmin\.nuget\packages\benchmarkdotnet.weaver\0.16.0-ci-2\buildTransitive\netstandard2.0\BenchmarkDotNet.Weaver.targets(20,5): warning :    at BenchmarkDotNet.Weaver.WeaveAssemblyTask.IsBenchmarkAttribute(CustomAttribute attribute, RuntimeContext context) [D:\a\BenchmarkDotNet\BenchmarkDotNet\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj]
C:\Users\runneradmin\.nuget\packages\benchmarkdotnet.weaver\0.16.0-ci-2\buildTransitive\netstandard2.0\BenchmarkDotNet.Weaver.targets(20,5): warning :    at BenchmarkDotNet.Weaver.WeaveAssemblyTask.<>c__DisplayClass4_0.<Execute>b__0(CustomAttribute a) [D:\a\BenchmarkDotNet\BenchmarkDotNet\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj]
C:\Users\runneradmin\.nuget\packages\benchmarkdotnet.weaver\0.16.0-ci-2\buildTransitive\netstandard2.0\BenchmarkDotNet.Weaver.targets(20,5): warning :    at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source, Func`2 predicate) [D:\a\BenchmarkDotNet\BenchmarkDotNet\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj]
C:\Users\runneradmin\.nuget\packages\benchmarkdotnet.weaver\0.16.0-ci-2\buildTransitive\netstandard2.0\BenchmarkDotNet.Weaver.targets(20,5): warning :    at BenchmarkDotNet.Weaver.WeaveAssemblyTask.Execute() [D:\a\BenchmarkDotNet\BenchmarkDotNet\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj]

timcassell · 2026-02-24T17:25:07Z

Does the "weaver" need to be able to find all assembly references? Because they won't be found in the same folder.

It needs to walk the hierarchy of attributes on methods, and to do that it must resolve the assemblies. I'm trying a custom resolver to search the OutDir now to see if we can keep it after CoreCompile.

jonathanpeppers · 2026-02-24T17:30:46Z

I think @js6pak is right that you can look at the @(ReferencePath) item group that will have the assemblies in whatever random folder they might be in.

js6pak · 2026-02-24T17:40:05Z

I don't know what you mean by that. We already pass the path to the assembly. The issue is the references of the target assembly can be nuget packages or otherwise that we cannot resolve after CoreCompile.

@(ReferencePath) already contains paths to all the references you need. XamlC task that was originally linked in this pull request also does it: https://github.com/dotnet/maui/blob/8224becbb3a8a6bb1caaca4bbe70c56e88875506/src/Controls/src/Build.Tasks/nuget/buildTransitive/netstandard2.0/Microsoft.Maui.Controls.targets#L233
You can just make a simple custom IAssemblyResolver that resolves the assemblies by file name.

timcassell · 2026-02-24T19:18:36Z

@(ReferencePath) worked, thanks.

jonathanpeppers commented Dec 16, 2025

View reviewed changes

Comment thread src/BenchmarkDotNet.Weaver/buildTransitive/netstandard2.0/BenchmarkDotNet.Weaver.targets

Prefix BenchmarkDotNet

1dbc5e3

timcassell approved these changes Dec 16, 2025

View reviewed changes

timcassell merged commit 7cad308 into dotnet:master Dec 16, 2025
19 of 21 checks passed

timcassell added this to the v0.16.0 milestone Dec 16, 2025

timcassell mentioned this pull request Dec 18, 2025

Fix NETSDK1152 in auto-generated projects #2935

Merged

timcassell mentioned this pull request Jan 16, 2026

Add support for composite r2r runs #2967

Merged

This comment was marked as outdated.

Sign in to view

Uh oh!

Conversation

jonathanpeppers commented Dec 16, 2025

Uh oh!

Uh oh!

timcassell commented Dec 16, 2025

Uh oh!

Uh oh!

DrewScoggins commented Dec 18, 2025

Uh oh!

DrewScoggins commented Dec 18, 2025

Uh oh!

jonathanpeppers commented Dec 18, 2025

Uh oh!

timcassell commented Dec 18, 2025

Uh oh!

DrewScoggins commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrewScoggins commented Dec 18, 2025

Uh oh!

jonathanpeppers commented Dec 20, 2025

Uh oh!

timcassell commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

js6pak commented Feb 24, 2026

Uh oh!

jonathanpeppers commented Feb 24, 2026

Uh oh!

timcassell commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

jonathanpeppers commented Feb 24, 2026

Uh oh!

timcassell commented Feb 24, 2026

Uh oh!

timcassell commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonathanpeppers commented Feb 24, 2026

Uh oh!

timcassell commented Feb 24, 2026

Uh oh!

jonathanpeppers commented Feb 24, 2026

Uh oh!

js6pak commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

timcassell commented Feb 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

DrewScoggins commented Dec 18, 2025 •

edited

Loading

timcassell commented Feb 24, 2026 •

edited

Loading

timcassell commented Feb 24, 2026 •

edited

Loading

timcassell commented Feb 24, 2026 •

edited

Loading

js6pak commented Feb 24, 2026 •

edited

Loading