Update CHANGELOG for version 2.6.0 release, add new samples, and impr…

[Howie Leung (howieleung)]

Make more samples
Make public-methods.md gen by ps1
Make it called by post emit script.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

A new sample fails without preview enabled, cleanup can leak resources, and release documentation is incomplete and inconsistent.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Prepares azure-ai-projects 2.6.0 with additional samples and automated API documentation generation.

Changes:

• Adds Model Router and Toolbox/Skill samples.
• Generates public-method and API review artifacts after emission.
• Finalizes the 2.6.0 changelog and reformats an evaluation sample.

File summaries

File	Description
tests/samples/test_samples.py	Adds Responses sample discovery.
samples/responses/sample_responses_model_router.py	Demonstrates Model Router responses.
samples/evaluations/sample_synthetic_multiturn_evaluation.py	Reformats existing sample.
samples/agents/tools/sample_toolbox_with_skill.py	Adds Toolbox Skill sample.
samples/agents/tools/sample_toolbox_with_shell.py	Updates required package version.
samples/agents/tools/sample_toolbox_with_shell_and_skill.py	Adds combined Shell and Skill sample.
PostEmitter.ps1	Runs artifact generators after emission.
GeneratePublicMethods.ps1	Generates the public-method inventory.
docs/public-methods.md	Updates generated method documentation.
CHANGELOG.md	Finalizes 2.6.0 release notes.
api.metadata.yml	Updates API snapshot metadata.
api.md	Refreshes the public API snapshot.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 4
• Review effort level: Balanced

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The API snapshot introduces an undocumented breaking removal, and new samples contain configuration and cleanup defects.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

sdk/ai/azure-ai-projects/samples/agents/tools/sample_toolbox_with_shell_and_skill.py:98

• The cleanup guard starts only after the skill download, archive parsing, and shell-tool construction. If any of these steps fails after skills.create succeeds, the newly created cloud skill is left behind. Start the try/finally immediately after skill creation so every subsequent failure deletes it.

    skill_archive = b"".join(
        project_client.beta.skills.download_version(name=skill_version.name, version=skill_version.version)
    )
    with zipfile.ZipFile(io.BytesIO(skill_archive)) as archive:
        skill_instructions = archive.read("SKILL.md").decode("utf-8")

sdk/ai/azure-ai-projects/CHANGELOG.md:20

• The PR also adds sample_toolbox_with_skill.py, but the 2.6.0 sample updates omit it while listing the other new samples. Add a release-note entry so users can discover the new Prompt Agent skill-only example.

* Added `sample_toolbox_with_shell_and_skill.py` under `samples/agents/tools/`, demonstrating a Prompt Agent using an inline Skill with a `ShellToolboxTool` through a Toolbox MCP endpoint.

• Files reviewed: 11/11 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

The release artifacts contain an unexplained API removal, and two new workflows have incomplete resource cleanup.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

sdk/ai/azure-ai-projects/GeneratePublicMethods.ps1:244

• async_client is created alongside sync_client, but the finally block closes only the synchronous client. Close the async client as well (for example with asyncio.run(async_client.close())) so generator failures and repeated runs do not leave its transport lifecycle unfinished.

sdk/ai/azure-ai-projects/samples/agents/tools/sample_toolbox_with_shell_and_skill.py:98

• Cleanup starts only at the later try, so a failure while downloading or reading SKILL.md after the skill is created leaves the persisted skill behind. Start the cleanup-protected scope immediately after pre-cleanup (before skill creation/download), so every failure path deletes the created skill.

    skill_archive = b"".join(
        project_client.beta.skills.download_version(name=skill_version.name, version=skill_version.version)
    )
    with zipfile.ZipFile(io.BytesIO(skill_archive)) as archive:
        skill_instructions = archive.read("SKILL.md").decode("utf-8")

sdk/ai/azure-ai-projects/api.md:8618

• The regenerated 2.6.0 surface removes RaiConfig.invocations_moderation and all supporting RaiInvocation*/RaiSseTextSelector types, while this release's changelog still advertises content-safety moderation for custom request, response, and streaming formats. This is both an unannounced public API break and a contradiction in the release artifacts. Restore the generated API or reconcile the source and changelog before releasing.

    class azure.ai.projects.models.RaiConfig(_Model):

• Files reviewed: 11/11 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

The API artifact contradicts the release notes, and failure paths can mask errors or leak created resources.

Review details

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

sdk/ai/azure-ai-projects/PostEmitter.ps1:99

• Check the apistub result before starting public-method generation. As written, an apistub failure still runs the second generator; if that also fails, its exception masks the original failure and leaves the post-emitter output partially regenerated.
  sdk/ai/azure-ai-projects/CHANGELOG.md:23
• This release note says the sample inspects models for “each inference turn,” but the sample sends one request and prints one selected model. Describe a single request/model so the release note matches the sample.

sdk/ai/azure-ai-projects/samples/agents/tools/sample_toolbox_with_shell_and_skill.py:98

• The skill is created before this download, but the cleanup try/finally does not begin until line 107. If the download, ZIP parsing, or SKILL.md read fails, the newly created cloud skill is left behind. Start the cleanup scope before downloading the archive, as the sibling sample_toolbox_with_skill.py does.

    skill_archive = b"".join(
        project_client.beta.skills.download_version(name=skill_version.name, version=skill_version.version)
    )
    with zipfile.ZipFile(io.BytesIO(skill_archive)) as archive:
        skill_instructions = archive.read("SKILL.md").decode("utf-8")

sdk/ai/azure-ai-projects/api.md:8625

• The regenerated API removes RaiConfig.invocations_moderation plus all four supporting invocation-format types, while the 2.6.0 release notes still advertise newly added moderation support for custom request, response, and streaming body formats. The released surface therefore does not expose the configuration described by the changelog; restore the generated models/property or revise the release note to match the actual supported API.

    class azure.ai.projects.models.RaiConfig(_Model):
        rai_policy_name: str

        @overload
        def __init__(
                self, 
                *, 
                rai_policy_name: str

• Files reviewed: 12/12 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[github-actions]

[Pilot] PR Pipeline Failure Analysis

What failed

Pipeline build 6787897 failed on the python - pullrequest check (job Build Analyze), with two independent issues:

1. CSpell lint failure — sdk/ai/azure-ai-projects/GeneratePublicMethods.ps1

Relevant pipeline output

##[error]sdk/ai/azure-ai-projects/GeneratePublicMethods.ps1:87:14 - Unknown word (dunders)
##[error]sdk/ai/azure-ai-projects/GeneratePublicMethods.ps1:90:58 - Unknown word (dunders)
##[error]Spelling errors detected. To correct false positives or learn about spell checking see: https://aka.ms/azsdk/engsys/spellcheck
##[error]PowerShell exited with code '1'.

2. Mypy failure (sample code) — duplicate module name across two sample files

Relevant pipeline output

[azure-ai-projects :: mypy] samples/hosted_agents/sample_toolbox_with_skill.py: error: Duplicate module named "sample_toolbox_with_skill" (also at "/mnt/vss/_work/1/s/sdk/ai/azure-ai-projects/samples/agents/tools/sample_toolbox_with_skill.py")
[azure-ai-projects :: mypy] samples/hosted_agents/sample_toolbox_with_skill.py: note: See (mypy.readthedocs.io/redacted) for more info
[azure-ai-projects :: mypy] samples/hosted_agents/sample_toolbox_with_skill.py: note: Common resolutions include: a) using `--exclude` to avoid checking one of them, b) adding `__init__.py` somewhere, c) using `--explicit-package-bases` or adjusting MYPYPATH
[azure-ai-projects :: mypy] Found 1 error in 1 file (errors prevented further checking)
[azure-ai-projects :: mypy] mypy check completed with exit code 2

This PR added two new sample files with the identical basename sample_toolbox_with_skill.py under samples/hosted_agents/ and samples/agents/tools/, which collide as mypy modules since neither directory has an __init__.py.

Recommended next steps

• CSpell: Add dunders to the repo's cspell dictionary (e.g. .vscode/cspell.json custom words list), or reword the two lines in GeneratePublicMethods.ps1 (lines 87 and 90) to avoid the term if it isn't intentional.
• Mypy duplicate module: Rename one of the two identically-named sample files (e.g. samples/hosted_agents/sample_toolbox_with_skill.py → samples/hosted_agents/sample_hosted_agents_toolbox_with_skill.py) so the basenames no longer collide, and update any references (README, docs/public-methods.md) accordingly.
• See https://aka.ms/ci-fix

Automated fix: Requested

Generated by Pipeline Analysis Next Steps · auto · 40.2 AIC · ⌖ 2.57 AIC · ⊞ 9.6K · ◷

[Copilot]

🟡 Changes recommended

The release surface conflicts with the advertised moderation feature, and one sample can leak a created cloud resource on failure.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

sdk/ai/azure-ai-projects/api.md:8625

• The regenerated public surface drops RaiConfig.invocations_moderation and all four RaiInvocation*/RaiSseTextSelector types, while the 2.6.0 changelog still advertises newly added custom request/response/streaming moderation support. As finalized, consumers cannot use the documented feature and these removals also appear as breaking API changes in a minor release. Please either restore the moderation surface and regenerate the artifacts or correct the release scope/changelog before publishing.

    class azure.ai.projects.models.RaiConfig(_Model):
        rai_policy_name: str

        @overload
        def __init__(
                self, 
                *, 
                rai_policy_name: str

sdk/ai/azure-ai-projects/samples/agents/tools/sample_toolbox_with_shell_and_skill.py:98

• The skill has already been created before this download, but the cleanup try/finally does not begin until line 107. If the download, ZIP parsing, or local shell-tool construction fails, the remote skill is left behind. Start the cleanup-protected region immediately after skill creation, as the neighboring shipping-skill sample does.

    skill_archive = b"".join(
        project_client.beta.skills.download_version(name=skill_version.name, version=skill_version.version)
    )
    with zipfile.ZipFile(io.BytesIO(skill_archive)) as archive:
        skill_instructions = archive.read("SKILL.md").decode("utf-8")

• Files reviewed: 12/12 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

One sample fails during setup, cleanup can leak a persisted resource, and the API snapshot contradicts an advertised 2.6.0 feature.

Review details

Suppressed comments (3)

sdk/ai/azure-ai-projects/samples/agents/tools/sample_toolbox_with_shipping_skill.py:64

• get_openai_client(agent_name=...) raises ValueError unless the project client was created with allow_preview=True, so this sample fails during context-manager setup before reaching any skill or toolbox calls. Enable preview on this client, as the neighboring shell sample does.

    AIProjectClient(endpoint=endpoint, credential=credential) as project_client,

sdk/ai/azure-ai-projects/samples/agents/tools/sample_toolbox_with_shell_and_skill.py:98

• The cleanup try starts only after the skill archive has been downloaded and decoded. If either operation fails after the skill is created, the finally at line 155 is never entered and the persisted skill is leaked; move the existing try boundary above all post-creation work.

    skill_archive = b"".join(
        project_client.beta.skills.download_version(name=skill_version.name, version=skill_version.version)
    )
    with zipfile.ZipFile(io.BytesIO(skill_archive)) as archive:
        skill_instructions = archive.read("SKILL.md").decode("utf-8")

sdk/ai/azure-ai-projects/api.md:8619

• This regenerated surface removes RaiConfig.invocations_moderation and the RaiInvocationModeration, RaiInvocationContentType, RaiInvocationMode, and RaiSseTextSelector public types, while the 2.6.0 changelog still promises content-safety moderation for custom request, response, and streaming formats. As committed, that advertised feature cannot be configured through the SDK; restore the generated models/property (or withdraw the release claim if the feature was intentionally removed) and regenerate the API artifacts.

    class azure.ai.projects.models.RaiConfig(_Model):
        rai_policy_name: str

• Files reviewed: 12/12 changed files
• Comments generated: 0 new
• Review effort level: Balanced