Python: [BREAKING] Align FileAccess tools with .NET — directory discovery and recursive search

[westey-m]

Motivation and Context

This brings the Python file-access harness in line with the .NET changes in #6474. Two gaps existed in the Python harness:

1. No directory discovery — file_access_list_files only returns direct-child files, so an agent could not discover that subdirectories exist.
2. Search was not recursive — file_access_search_files only scanned the direct children of a single directory, with no way to search an entire subtree.

#6398

Description

Mirrors the .NET surface (with one intentional, documented divergence on glob semantics).

Store layer (AgentFileStore + both implementations)

• Added list_directories(directory="") — direct child subdirectories only.
• Added keyword-only recursive: bool = False to search_files; the glob and each result's file_name are relative to directory, and recursive=True walks all descendants.
• InMemoryAgentFileStore: recursive relative-path glob/file_name; list_directories collects the distinct first path segment of nested keys (case-preserving, case-insensitive de-dup).
• FileSystemAgentFileStore: recursive walk that prunes symlinked directories and skips symlinked files; list_directories excludes symlinked dirs and returns [] for a missing directory.

Provider (FileAccessProvider)

• file_access_search_files: removed the directory parameter; now searches recursively from the store root and returns store-root-relative file_name paths. Scoping is via the glob.
• Added the file_access_list_subdirectories tool.
• Updated docstrings and DEFAULT_FILE_ACCESS_INSTRUCTIONS to mention subdirectory exploration and recursive search.

Intentional divergence from .NET: Python keeps fnmatch, where * already crosses / (e.g. *.md matches markdown at any depth, reports/* scopes to a subtree). This differs from .NET's ** matcher and is documented to the model via the tool description.

Tests & docs

• Added coverage for recursive search, glob subtree/extension scoping, list_directories (in-memory + filesystem), filesystem symlinked-directory skipping, and path-traversal rejection for list_directories on both stores.
• Updated python/packages/core/AGENTS.md and the file_access_data_processing sample.

The Python _harness/_memory.py provider does not call the store search/list APIs, and search_files defaults to recursive=False, so existing behavior is preserved.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

[Copilot]

Pull request overview

Aligns the Python file-access harness/tooling with the .NET surface by adding directory discovery and making store-level search optionally recursive, while updating the provider to expose a dedicated subdirectory listing tool and a root-recursive search tool.

Changes:

• Added AgentFileStore.list_directories() and a keyword-only recursive flag to search_files() (implemented for both in-memory and filesystem stores).
• Updated FileAccessProvider to add file_access_list_subdirectories and to make file_access_search_files search recursively from the store root (scoped via glob).
• Updated tests and sample/docs to cover recursive search, directory discovery, and filesystem symlink behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
python/samples/02-agents/harness/console/observers/planning_models.py	Expands structured-output schema guidance for clarification choices.
python/samples/02-agents/context_providers/file_access_data_processing/data_processing.py	Updates sample instructions/comments to mention subdirectory exploration and the new tool count.
python/packages/core/tests/core/test_harness_file_access.py	Adds/updates unit tests for recursive search, directory listing, symlink handling, and provider tool surface.
python/packages/core/AGENTS.md	Updates harness documentation to reflect new store/provider capabilities and semantics.
python/packages/core/agent_framework/_harness/_file_access.py	Implements list_directories, recursive search behavior, and adds the new file_access_list_subdirectories tool + updated search tool semantics.

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/core/agent_framework/_harness
_file_access.py	491	36	92%	179, 311, 313, 325, 329, 624, 634, 650, 691, 718–719, 745, 749, 803–805, 827–830, 858, 951, 1061–1062, 1073–1074, 1087–1088, 1099–1100, 1114–1117, 1143–1144
TOTAL	39154	4479	88%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
7779	34 💤	0 ❌	0 🔥	2m 1s ⏱️

[github-actions]

Automated Code Review

Reviewers: 5 | Confidence: 91%

✓ Correctness

The core file access changes (recursive search, list_directories, symlink skipping) are logically correct and well-tested. The only correctness issue is in planning_models.py where implicit Python string concatenation produces sentences joined without spaces, creating malformed description text sent to the LLM.

✓ Security Reliability

The file-access changes are well-implemented from a security/reliability standpoint: path traversal is rejected by _normalize_relative_path, symlinks are correctly skipped in both _enumerate_search_files and _list_directories_sync, the recursive search is bounded by _run_search_with_timeout, and errors are caught at the tool boundary. The one issue found is in planning_models.py where adjacent Python string literals are missing separators, producing garbled prompt text.

✓ Test Coverage

The test coverage for this PR is thorough. New tests cover recursive search (both stores), list_directories (both stores), path traversal rejection, symlinked directory exclusion, provider-level tool registration, and round-trip integration. The tests validate meaningful assertions (exact file sets, security boundaries, correct relative paths). One minor gap: symlinked files are skipped by the implementation but only symlinked directories are explicitly tested; however, the same is_symlink() check handles both. No blocking issues found.

✓ Failure Modes

The core file-access changes (recursive search, list_directories) are well-implemented with consistent error handling and proper security boundaries. However, the planning_models.py change introduces garbled description text due to missing whitespace between implicitly concatenated strings. The recursive _enumerate_search_files helper inherits the existing lack of per-directory PermissionError handling, which means a single unreadable subdirectory now aborts an entire recursive search (a wider failure surface than before, though the provider's OSError catch prevents a hard crash).

✓ Design Approach

I found one design-parity issue in the new store surface: InMemoryAgentFileStore now advertises directory discovery via list_directories, but it still treats create_directory as a no-op, so empty directories are never discoverable there even though the filesystem implementation does surface them. I did not find a stronger blocking issue in the provider-level recursive search change itself.

Suggestions

• Make InMemoryAgentFileStore persist explicit directories so create_directory() and list_directories() model the same contract as FileSystemAgentFileStore.

---

Automated review by westey-m's agents