-
Notifications
You must be signed in to change notification settings - Fork 1.2k
.NET: Add AgentWithApproval hosted agent sample with HITL [WIP] #4077
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
rogerbarreto
wants to merge
1
commit into
microsoft:main
Choose a base branch
from
rogerbarreto:feature/hostedagents-hitl-entra
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+248
−0
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
68 changes: 68 additions & 0 deletions
68
dotnet/samples/HostedAgents/AgentWithApproval/AgentWithApproval.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <TargetFrameworks>net10.0</TargetFrameworks> | ||
|
|
||
| <Nullable>enable</Nullable> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
|
|
||
| <!-- | ||
| Disable central package management for this project. | ||
| This project requires explicit package references with versions specified inline rather than | ||
| inheriting them from Directory.Packages.props. This is necessary because a Docker image will | ||
| be created from this project, and the Docker build process only has access to this folder | ||
| and cannot access parent folders where Directory.Packages.props resides. | ||
| --> | ||
| <ManagePackageVersionsCentrally>false</ManagePackageVersionsCentrally> | ||
| </PropertyGroup> | ||
|
|
||
| <!-- | ||
| Remove analyzer PackageReference items inherited from Directory.Packages.props. | ||
| Note: ManagePackageVersionsCentrally only controls PackageVersion items, not PackageReference items. | ||
| Directory.Packages.props contains both PackageVersion and PackageReference entries for analyzers, | ||
| and the PackageReference items are always inherited through MSBuild imports regardless of the | ||
| ManagePackageVersionsCentrally setting. We must explicitly remove them before adding our own versions. | ||
| --> | ||
| <ItemGroup> | ||
| <PackageReference Remove="Microsoft.CodeAnalysis.NetAnalyzers" /> | ||
| <PackageReference Remove="Microsoft.VisualStudio.Threading.Analyzers" /> | ||
| <PackageReference Remove="xunit.analyzers" /> | ||
| <PackageReference Remove="Moq.Analyzers" /> | ||
| <PackageReference Remove="Roslynator.Analyzers" /> | ||
| <PackageReference Remove="Roslynator.CodeAnalysis.Analyzers" /> | ||
| <PackageReference Remove="Roslynator.Formatting.Analyzers" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="Azure.AI.AgentServer.AgentFramework" Version="1.0.0-beta.6" /> | ||
| <PackageReference Include="Azure.AI.OpenAI" Version="2.8.0-beta.1" /> | ||
| <PackageReference Include="Azure.Identity" Version="1.17.1" /> | ||
| <PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="10.3.0" /> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Add analyzers with compatible versions --> | ||
| <ItemGroup> | ||
| <PackageReference Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="10.0.100"> | ||
| <PrivateAssets>all</PrivateAssets> | ||
| <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> | ||
| </PackageReference> | ||
| <PackageReference Include="Microsoft.VisualStudio.Threading.Analyzers" Version="17.14.15"> | ||
| <PrivateAssets>all</PrivateAssets> | ||
| <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> | ||
| </PackageReference> | ||
| <PackageReference Include="Roslynator.Analyzers" Version="4.14.1"> | ||
| <PrivateAssets>all</PrivateAssets> | ||
| <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> | ||
| </PackageReference> | ||
| <PackageReference Include="Roslynator.CodeAnalysis.Analyzers" Version="4.14.1"> | ||
| <PrivateAssets>all</PrivateAssets> | ||
| <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> | ||
| </PackageReference> | ||
| <PackageReference Include="Roslynator.Formatting.Analyzers" Version="4.14.1"> | ||
| <PrivateAssets>all</PrivateAssets> | ||
| <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> | ||
| </PackageReference> | ||
| </ItemGroup> | ||
|
|
||
| </Project> | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Build the application | ||
| FROM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS build | ||
| WORKDIR /src | ||
|
|
||
| # Copy files from the current directory on the host to the working directory in the container | ||
| COPY . . | ||
|
|
||
| RUN dotnet restore | ||
| RUN dotnet build -c Release --no-restore | ||
| RUN dotnet publish -c Release --no-build -o /app -f net10.0 | ||
|
|
||
| # Run the application | ||
| FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final | ||
| WORKDIR /app | ||
|
|
||
| # Copy everything needed to run the app from the "build" stage. | ||
| COPY --from=build /app . | ||
|
|
||
| EXPOSE 8088 | ||
| ENTRYPOINT ["dotnet", "AgentWithApproval.dll"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| // Copyright (c) Microsoft. All rights reserved. | ||
|
|
||
| // This sample shows how to create and use a simple AI agent with OpenAI Responses as the backend, that uses a Hosted MCP Tool. | ||
| // In this case the OpenAI responses service will invoke any MCP tools as required. MCP tools are not invoked by the Agent Framework. | ||
| // The sample demonstrates how to use MCP tools with human-in-the-loop approval by setting ApprovalMode to AlwaysRequire. | ||
| // When a tool call is requested, the caller must explicitly approve or deny it before the agent proceeds. | ||
|
|
||
| using Azure.AI.AgentServer.AgentFramework.Extensions; | ||
| using Azure.AI.OpenAI; | ||
| using Azure.Identity; | ||
| using Microsoft.Agents.AI; | ||
| using Microsoft.Extensions.AI; | ||
|
|
||
| var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set."); | ||
| var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini"; | ||
|
|
||
| // Create an MCP tool that requires human approval before each invocation. | ||
| // Unlike the AgentWithHostedMCP sample (which uses NeverRequire), this sample gates | ||
| // every tool call behind explicit user approval — a human-in-the-loop pattern. | ||
| AITool mcpTool = new HostedMcpServerTool(serverName: "microsoft_learn", serverAddress: "https://learn.microsoft.com/api/mcp") | ||
| { | ||
| AllowedTools = ["microsoft_docs_search"], | ||
| ApprovalMode = HostedMcpServerToolApprovalMode.AlwaysRequire | ||
| }; | ||
|
|
||
| // Create an agent with the MCP tool using Azure OpenAI Responses. | ||
| // WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production. | ||
| // In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid | ||
| // latency issues, unintended credential probing, and potential security risks from fallback mechanisms. | ||
| AIAgent agent = new AzureOpenAIClient( | ||
| new Uri(endpoint), | ||
| new DefaultAzureCredential()) | ||
| .GetResponsesClient(deploymentName) | ||
| .AsIChatClient() | ||
| .CreateAIAgent( | ||
rogerbarreto marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| instructions: "You answer questions by searching the Microsoft Learn content only.", | ||
| name: "MicrosoftLearnAgentWithApproval", | ||
| tools: [mcpTool]); | ||
|
|
||
| await agent.RunAIAgentAsync(); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| # What this sample demonstrates | ||
|
|
||
| This sample demonstrates how to use a Hosted Model Context Protocol (MCP) server with an AI agent | ||
| that requires **human-in-the-loop approval** before executing any tool call. | ||
|
|
||
| The agent connects to the Microsoft Learn MCP server to search documentation, but unlike the | ||
| [AgentWithHostedMCP](../AgentWithHostedMCP) sample (which auto-approves tool calls), this sample | ||
| requires explicit approval for every MCP tool invocation. | ||
|
|
||
| Key features: | ||
| - Configuring MCP tools with required approval (`AlwaysRequire` mode) | ||
| - Human-in-the-loop pattern for tool call gating | ||
| - Using Azure OpenAI Responses with approval-gated MCP tools | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before running this sample, ensure you have: | ||
|
|
||
| 1. An Azure OpenAI endpoint configured | ||
| 2. A deployment of a chat model (e.g., gpt-4o-mini) | ||
| 3. Azure CLI installed and authenticated | ||
|
|
||
| **Note**: This sample uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure OpenAI resource. | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| Set the following environment variables: | ||
|
|
||
| ```powershell | ||
| # Replace with your Azure OpenAI endpoint | ||
| $env:AZURE_OPENAI_ENDPOINT="https://your-openai-resource.openai.azure.com/" | ||
|
|
||
| # Optional, defaults to gpt-4o-mini | ||
| $env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini" | ||
| ``` | ||
|
|
||
| ## How It Works | ||
|
|
||
| The sample connects to the Microsoft Learn MCP server with approval-gated tool calls: | ||
|
|
||
| 1. The agent is configured with a `HostedMcpServerTool` pointing to `https://learn.microsoft.com/api/mcp` | ||
| 2. Only the `microsoft_docs_search` tool is enabled from the available MCP tools | ||
| 3. Approval mode is set to `AlwaysRequire`, meaning every tool call must be explicitly approved | ||
| 4. When you ask a question, Azure OpenAI Responses requests the MCP tool call and the agent surfaces an approval request to the caller | ||
| 5. The caller must approve or deny the tool call before the agent proceeds | ||
| 6. Once approved, the MCP tool executes and the agent returns the answer | ||
|
|
||
| In this configuration, the OpenAI Responses service manages tool invocation directly — the Agent Framework does not handle MCP tool calls. The approval gate ensures that no tool call executes without explicit human consent. | ||
|
|
||
| ## Comparison with AgentWithHostedMCP | ||
|
|
||
| | Feature | AgentWithHostedMCP | AgentWithApproval | | ||
| |---------|-------------------|-------------------| | ||
| | MCP Server | Microsoft Learn | Microsoft Learn | | ||
| | Approval Mode | `NeverRequire` | `AlwaysRequire` | | ||
| | Tool Execution | Automatic | Requires human approval | | ||
| | Use Case | Trusted, automated search | Gated access with oversight | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| name: AgentWithApproval | ||
| displayName: "Microsoft Learn Response Agent with MCP Approval" | ||
| description: > | ||
| An AI agent that uses Azure OpenAI Responses with a Hosted Model Context Protocol (MCP) server, | ||
| requiring explicit human approval before each tool invocation. The agent answers questions by | ||
| searching Microsoft Learn documentation using MCP tools, but every tool call must be approved | ||
| by the caller first — demonstrating a human-in-the-loop pattern. | ||
| metadata: | ||
| authors: | ||
| - Microsoft Agent Framework Team | ||
| tags: | ||
| - Azure AI AgentServer | ||
| - Microsoft Agent Framework | ||
| - Model Context Protocol | ||
| - MCP | ||
| - Human-in-the-Loop | ||
| - Tool Call Approval | ||
| template: | ||
| kind: hosted | ||
| name: AgentWithApproval | ||
| protocols: | ||
| - protocol: responses | ||
| version: v1 | ||
| environment_variables: | ||
| - name: AZURE_OPENAI_ENDPOINT | ||
| value: ${AZURE_OPENAI_ENDPOINT} | ||
| - name: AZURE_OPENAI_DEPLOYMENT_NAME | ||
| value: gpt-4o-mini | ||
| resources: | ||
| - name: "gpt-4o-mini" | ||
| kind: model | ||
| id: gpt-4o-mini |
30 changes: 30 additions & 0 deletions
30
dotnet/samples/HostedAgents/AgentWithApproval/run-requests.http
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| @host = http://localhost:8088 | ||
| @endpoint = {{host}}/responses | ||
|
|
||
| ### Health Check | ||
| GET {{host}}/readiness | ||
|
|
||
| ### Ask a question - This will trigger an MCP tool call that requires approval | ||
| POST {{endpoint}} | ||
| Content-Type: application/json | ||
| { | ||
| "input": "Please summarize the Azure AI Agent documentation related to MCP Tool calling?" | ||
| } | ||
|
|
||
| ### Explicit input - Ask about Agent Framework (also triggers approval) | ||
| POST {{endpoint}} | ||
| Content-Type: application/json | ||
| { | ||
| "input": [ | ||
| { | ||
| "type": "message", | ||
| "role": "user", | ||
| "content": [ | ||
| { | ||
| "type": "input_text", | ||
| "text": "What is the Microsoft Agent Framework?" | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.