.NET: Agent Framework - Structured Output prevent tool calls for many providers

[kacisse]

hey,

I need help, I'm facing a very strange behavior. I'm expecting a json response Format (structured output) through AgentRunOptions.ResponseFormat. It works perfectly for some Gemini models (3.1 flash lite, 2.5 flash lite for ex) but fail for literally all Chinese ones
deepseek v4 pro and flash
mimo pro
minimax 3
etc...

Gemini models call successfully the tools and return the structured output. The failing ones above mostly return an empty structured output right away !

I'm using a ChatclientAgent with Openrouter.

Is that a bug from Agent Framework or a model behavior ? Hard to believe so many models fail at a basic task. Important to note that of course if structured output is not used, all modes are able to call multi turn tools successfully. It's only for structured output.

thanks a lot

[westey-m]

@kacisse, which ChatClient / nuget are you using?

I've tried deepseek v4 pro on foundry over responses with the AIProjectClient, and tool calls and structured output are working for me together. One thing I have observed though is that deepseek v4 pro is not consistent in only returning structured output when using tools. It will often reply with regular text while processing the tools and only the final message has the structured output. This requires manual handling, since by default Agent Framework assumes that only json will be returned when requesting structured output.

To help pinpoint if the problem is in the service or on the client, you can enable OpelTelemetry tracing on the http client used (assuming your ChatClient is using httpclient).

Just add the following to your app, and check the exe folder for the trace file to see what exactly the service returned:

        var traceLogPath = Path.Combine(AppContext.BaseDirectory, $"traces_{DateTime.UtcNow:yyyyMMdd_HHmmss}_{Guid.NewGuid()}.log");

        using Sdk.CreateTracerProviderBuilder()
            .AddHttpClientInstrumentation((options) =>
            {
                options.EnrichWithHttpRequestMessage = (activity, request) =>
                {
                    activity.SetTag("http.request.headers", request.Headers.ToString());
                    if (request.Content != null)
                    {
                        activity.SetTag("http.request.content.headers", request.Content.Headers.ToString());
                        var content = request.Content.ReadAsStringAsync().GetAwaiter().GetResult();
                        activity.SetTag("http.request.content.body", content);
                    }
                };

                options.EnrichWithHttpResponseMessage = (activity, response) =>
                {
                    activity.SetTag("http.response.headers", response.Headers.ToString());
                    if (response.Content != null)
                    {
                        activity.SetTag("http.response.content.headers", response.Content.Headers.ToString());
                        var content = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
                        activity.SetTag("http.response.content.body", content);
                    }
                };
            })
            .AddProcessor(new SimpleActivityExportProcessor(new FileSpanExporter(traceLogPath)))
            .Build();

[kacisse]

@kacisse, which ChatClient / nuget are you using?

I've tried deepseek v4 pro on foundry over responses with the AIProjectClient, and tool calls and structured output are working for me together. One thing I have observed though is that deepseek v4 pro is not consistent in only returning structured output when using tools. It will often reply with regular text while processing the tools and only the final message has the structured output. This requires manual handling, since by default Agent Framework assumes that only json will be returned when requesting structured output.

To help pinpoint if the problem is in the service or on the client, you can enable OpelTelemetry tracing on the http client used (assuming your ChatClient is using httpclient).

Just add the following to your app, and check the exe folder for the trace file to see what exactly the service returned:

        var traceLogPath = Path.Combine(AppContext.BaseDirectory, $"traces_{DateTime.UtcNow:yyyyMMdd_HHmmss}_{Guid.NewGuid()}.log");

        using Sdk.CreateTracerProviderBuilder()
            .AddHttpClientInstrumentation((options) =>
            {
                options.EnrichWithHttpRequestMessage = (activity, request) =>
                {
                    activity.SetTag("http.request.headers", request.Headers.ToString());
                    if (request.Content != null)
                    {
                        activity.SetTag("http.request.content.headers", request.Content.Headers.ToString());
                        var content = request.Content.ReadAsStringAsync().GetAwaiter().GetResult();
                        activity.SetTag("http.request.content.body", content);
                    }
                };

                options.EnrichWithHttpResponseMessage = (activity, response) =>
                {
                    activity.SetTag("http.response.headers", response.Headers.ToString());
                    if (response.Content != null)
                    {
                        activity.SetTag("http.response.content.headers", response.Content.Headers.ToString());
                        var content = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
                        activity.SetTag("http.response.content.body", content);
                    }
                };
            })
            .AddProcessor(new SimpleActivityExportProcessor(new FileSpanExporter(traceLogPath)))
            .Build();

OH that's a good insight, it might come OpenRouter. I'll try creating an API on Deepseek platform directly and see the difference. I will also try the telemetry as you suggested ASAP.
thanks a lot for your time :)