Caching v0.3: robust trajectory caching, precise parameters & observability

[philipph-askui]

Caching v0.3

A single PR covering the caching v0.3 work: it makes trajectory caching correct, precise, observable, and hard to misconfigure, and bumps the cache format to 0.3.

Reliability & mechanism

• Auto-detect the trajectory by name. In execute/auto mode the SDK looks up <cache_dir>/<filename> and, if a usable trajectory exists, injects its details (path + required parameters) into the first user message so the agent hands off to the CacheExecutor immediately. The old "list available trajectories" tool and prompt workflow are removed.
• CachingSettings.filename now actually works and may include subdirectories relative to cache_dir (e.g. filename="mytests_1/test_something" → .askui_cache/mytests_1/test_something.json) for both lookup and recording — with a guard so a filename can't escape cache_dir. Previously it was silently ignored, and nested recording crashed.
• Resuming past the last (non-cacheable) step no longer crashes. start_from_step_index == len(trajectory) means "already complete"; the pause message tells the agent the exact resume index or that it was the final step.
• execute-only mode no longer crashes (a CacheManager is always created when executing).
• verify_cache_execution now persists outcomes: success records the attempt; failure invalidates the cache.
• No cross-run leaks: the CACHE_USE prompt, the CacheExecutor speaker, and per-call caching tools no longer bleed into later act() calls.
• Empty / no-cacheable-step recordings are no longer written; recording errors in teardown are logged, not masked.
• Cache format version bumped 0.2 → 0.3.

Precise parameter identification

• Only user-entered free text is eligible; structural inputs (coordinates, key names, action/enum values, counts, tool names) are never offered to the model.
• The system prompt is conservative/precision-first — parameterize only when replaying the literal would clearly be wrong (dates relative to "now", generated IDs/tokens/OTPs, per-run identities); recording zero parameters is normal. The LLM isn't called when there are no candidates.
• Identified values are validated against the offered candidates (invalid names, empty values, and hallucinated/reformatted values are dropped).

Observability (logs and reporter)

• Caching explains what and why via the logger and the attached reporter (role Cache, so it also shows up in the HTML report): cache hit/miss, replay start, pause on a non-cacheable step (with tool name), completion, verification outcome with the agent's reason and the cache file, invalidation, and recording written/skipped.
• Invalidation logs a concise WARNING headline with the full reason at INFO.

Safer configuration

• CachingSettings, CacheExecutionSettings, and CacheWritingSettings reject unknown fields (extra="forbid"), so a misplaced option — e.g. CachingSettings(delay_time_between_actions=3.0) instead of nesting under execution_settings — raises a clear error instead of being silently ignored.

Testing & docs

• pdm run qa:fix clean (typecheck + format + lint); full unit suite green (784 passed), with dedicated tests for auto-detection, resume-at-end, parameter precision/validation, nested filenames + escape guard, extra="forbid", and the observability messages.
• docs/06_caching.md rewritten for the filename-based flow with Dynamic Parameters and Observability sections.

Reviews

Developed with three adversarial code-review passes (recording path, replay/orchestration path, and a final review of the parameter/observability changes); all substantiated findings were fixed.

Known follow-ups (intentionally out of scope)

• Whole-field, type-preserving parameter substitution (the substring/str() limitation).
• Recording success-gating beyond the empty-trajectory guard.
• Tool-level input coercion so a model/replay passing a numeric as a string doesn't crash a tool (a tools change, tracked separately).

🤖 Generated with Claude Code

[philipph-askui]

Added nested cache filename support (commit 9f71337): CachingSettings.filename may include subdirectories relative to cache_dir, so e.g. filename="mytests_1/test_something" maps to .askui_cache/mytests_1/test_something.json for both lookup and recording. Lookup already preserved subdirectories; recording previously crashed with FileNotFoundError because the nested parent dir wasn't created — now created on record and on metadata writes. Docs + tests included.