fix: enforce search_web's cutoff date from the harness, not the LLM

[ethancjackson]

Summary

Follow-up to #161. That PR added an independent leakage verifier to search_web, but it only activated when the calling LLM supplied a cutoff_date tool argument. A production trace showed this bypassed outright: 11 of 14 search_web calls in a real backtest simply omitted cutoff_date, so both the soft prompt instruction and the verifier were silently skipped, and live googleSearch results (dated the actual current date) flowed straight into a historical forecast whose origin was six weeks earlier.

This PR closes that bypass by moving the cutoff from something the model supplies to something the harness enforces: AgentPredictor.predict() now seeds the forecast's as_of into ADK session state before each run, and search_web reads it via an ADK-injected ToolContext — a parameter excluded from the LLM-visible tool schema, so the model can neither see it, omit it, nor spoof it. This value is authoritative whenever present; the model's own cutoff_date argument becomes a redundant, logged-on-disagreement fallback for callers outside AgentPredictor (e.g. interactive adk web use).

Because AgentPredictor/ForecastContext.as_of are shared across every forecasting domain, this closes the bypass everywhere in one change — no per-domain edits needed this time.

Clickup Ticket(s): N/A

Why this needed a second agent, not a better prompt

Both leaks in this saga (this one and #161's) trace back to the same root cause: a single LLM cannot reliably introspect on where its own words came from — whether a claim was retrieved just now or recalled from training. No amount of prompt engineering fixes that, because the model isn't lying about its confidence, it genuinely can't always tell the difference. The fix that actually works is architectural: a second, independent LLM call whose only job is to audit the first one's output against an external fact (the cutoff date) it wasn't in the loop to negotiate. That's a real agentic decomposition — specialized sub-agents cross-checking each other — not just a bigger prompt on a single model. This mirrors the published TimeSPEC pattern (claim-level extraction + independent verification beats prompt-only cutoff instructions by 75-99% in their evaluations).

The caveat that matters just as much: this is harm reduction, not a substitute for live evaluation. A verifier can only catch what it's told to look for and reason correctly about; it cannot conjure certainty from a fundamentally leaky retrieval channel (the live web index doesn't stop existing at the backtest origin). The only setup where future-leakage is structurally impossible, not just guarded against, is genuine live evaluation — predicting into a future that hasn't been indexed anywhere yet. Backtesting an agent with live web search, however well-guarded, is always a step down from that in terms of what it can prove.

Changes Made

• aieng-forecasting/aieng/forecasting/methods/agentic/agent_factory.py: search_web now accepts an ADK-injected tool_context: ToolContext; reads the harness-seeded as_of (new AS_OF_STATE_KEY) as the authoritative cutoff, falling back to the LLM's cutoff_date only when no harness value exists. Logs a warning if the two disagree.
• aieng-forecasting/aieng/forecasting/methods/agentic/adk_runner.py: run_text_async/_resolve_session_id gained an optional initial_state parameter, threaded into ADK's create_session(..., state=...) — purely additive, every existing call site unaffected.
• aieng-forecasting/aieng/forecasting/methods/agentic/predictor.py: AgentPredictor.predict() seeds {AS_OF_STATE_KEY: context.as_of} into the session before each run.
• New tests across all three files, including the specific case that was silently broken before (harness as_of present, LLM omits cutoff_date entirely) and the disagreement-logging path.
• Verified 2026 eval prediction files for both WTI news-agent variants, regenerated with the fix in place (only these two predictors' cache was cleared; everything else stayed cached).

Testing

• Tests pass locally (uv run pytest tests/)
• Type checking passes (uv run mypy <src_dir>)
• Linting passes (actual pre-commit hook suite, not just ad-hoc ruff — learned that lesson on fix: independent temporal-leakage verifier for search_web #161)
• Manual testing performed (describe below)

Manual testing details:

• uv run pytest aieng-forecasting/tests/aieng/forecasting/methods/agentic -q — 95 passed, no regressions.
• uv run pytest tests/ (full aieng-forecasting suite) — 387 passed, 7 skipped.
• python -m pytest implementations/tests -q — 82 passed.
• Re-ran the 2026 eval spec for both news-agent models after clearing only their stale eval cache, then audited the resulting Langfuse traces:
  • Across 169 sampled search_web calls (36 traces, both models): 0 occurrences of [SEARCH_VERIFICATION_FAILED], 0 calls with retry-range latency — everything passed on the first attempt.
  • Traced one case end to end: origin as_of=2026-05-18, raw search result dated "As of July 2, 2026" (~6 weeks post-cutoff, the same leak class as before) — the verifier flagged 6 claims, stripped them, and the analyst's final set_model_response rationale contains no trace of the July content. This is the exact scenario that slipped through silently before this fix.
• Known cosmetic side effect (not a safety issue): the Langfuse trace's top-level search_web span now displays its input/output mirroring the raw first child call rather than the tool's actual arguments/return value — likely because ADK's OTEL instrumentation can't cleanly serialize the new tool_context parameter. Confirmed via source code and the final rationale that actual runtime behavior is correct; only the trace display is affected.

Related Issues

Follow-up to #161.

Deployment Notes

The 2025 backtest prediction files for these two predictors were not re-run in this PR (only the smaller 2026 eval spec, per the new "verify before opening" process) — a full backtest rerun can follow separately if wanted, but isn't required for correctness since the fix is unconditional at the code level.

Checklist

• Code follows the project's style guidelines
• Self-review of code completed
• Documentation updated (if applicable)
• No sensitive information (API keys, credentials) exposed