[RFC] Evidence-grade pre-edit and post-edit workflows for coding agents

[nilamoromete]

What problem does this solve?

Coding agents can query useful graph primitives today, but they still have to assemble project resolution, freshness, path coverage, callers, related files, tests, and change risk across multiple calls. The agent must also decide whether an empty result means “no evidence” or “the wrong/stale/partial graph”.

After five months using a downstream workflow layer on a large private polyglot monorepo, I observed three recurring failure classes:

1. Multiple stored names can refer to the same source root, including an empty/shadow graph and a populated graph. An omitted or loosely resolved project can therefore produce a plausible empty answer from the wrong identity.
2. Cross-tool answers can contradict one another: a target can be reported outside coverage while related test files are still ranked, or tests can appear as related evidence while the recommended-test section is empty.
3. Agents spend several calls and substantial context reconciling primitives before editing. A compact composition can be cheaper overall even if its single response is larger than one primitive response.

This proposal is not meant to replace the existing tools. It asks whether the server should provide a small, deterministic workflow layer with explicit evidence quality.

The identity/freshness prerequisites overlap existing work rather than replacing it:

• Artifact bootstrap is a no-op across clone paths (project name derived from absolute repo path) #885 — path-dependent artifact/project identity
• [Windows] Repeated incremental re-index leaves an oversized/stuck SQLite WAL — node count frozen while HEAD pointer advances; delete_project returns "Permission denied" #1174 — Windows index state can become stale while metadata advances
• auto_watch is session-scoped and keyed by path-derived name — override-named indexes never refresh, and a watching session silently forks a duplicate index #1660 — watchers bind to path-derived names and can fork duplicate identities
• 📌 Epics / Roadmap — umbrella task index #595 — roadmap umbrella for stability, integration, architecture, and benchmarks

I searched the existing issues for get_edit_plan, edit planning, and MCP change-risk workflows and did not find a duplicate proposal.

Proposed solution

Semantics first

Introduce one internal evidence contract shared by any accepted workflow:

• exact project resolution; ambiguity fails closed with candidates;
• one pinned index generation per response;
• requested-path coverage checked before aggregating evidence;
• distinct outcomes: complete, partial, empty_verified, ambiguous, stale, unsupported, error;
• risk and confidence kept separate;
• stable evidence IDs, provenance, relationship/ranking reasons, and deterministic tie-breaking;
• explicit truncation counts/cursors;
• compact output by default, no snippets by default, detailed expansion opt-in;
• read-only/idempotent behavior, no re-index side effect, no LLM dependency.

An empty list is authoritative only as empty_verified when the relevant paths are covered by a fresh generation. Store or lookup errors are never rewritten as empty results.

Preferred public surface: two tools

1. get_edit_plan: bounded pre-edit context for one or more paths — symbols/callers, related files, recommended tests, optional routes, risks, confidence, and evidence-backed next steps.
2. get_change_risks: bounded post-edit review for explicit paths or the working-tree diff — blast radius, callers/routes, regression tests, risk factors, and missing evidence.

Internally, these compose six independently testable views: file context, related files, tests, callers, change risks, and edit-plan composition.

I prefer two public tools because every advertised MCP schema consumes model context. The downstream experiment currently exposes all six view names, but making all six permanent upstream tools may spend more schema tokens and create more API surface than necessary.

I am explicitly asking for maintainer direction before implementation:

• two public tools backed by internal views;
• six atomic public tools;
• MCP prompts/agent profiles that compose existing tools, with no new tool;
• evidence/consistency improvements only, with no workflow surface.

Compatibility constraints

• preserve all current tool names, inputs, defaults, and text output;
• do not add outputSchema as part of this work; current tests correctly protect clients that treat it as authoritative;
• keep stdout protocol-clean and diagnostics on stderr;
• use existing graph/store APIs and compact rendering;
• no hosted service, source upload, or model call;
• keep each accepted issue/PR narrow and under the project’s review-size guidance.

Evaluation before default rollout

I propose freezing a paired benchmark before candidate implementation:

• A: repository navigation without Codebase Memory;
• B: current upstream primitive tools;
• C: candidate workflow.

The same model, reasoning level, prompt, base commit, timeout, and tool budget are used, and resulting patches are scored blind.

Promotion targets:

• zero wrong-project answers, unsafe empty answers, and cross-section contradictions;
• at least +10 percentage points in hidden/targeted test pass rate versus B;
• at least 20% fewer unnecessary touched files and regression escapes;
• at least 30% fewer total task tokens, including advertised schemas, calls, responses, and completion;
• compact response p95 no more than 15% over the comparable primitive response budget;
• warm p95 at least 40% faster than the equivalent primitive-call chain.

Private-repository tasks and patches remain private. Only methodology, aggregate results, and synthetic/minimized fixtures would be contributed.

Proposed contribution sequence

If the direction is accepted:

1. benchmark contract and minimized fixtures, without an API change;
2. narrow identity/freshness prerequisite fixes through their own issues;
3. internal evidence envelope and deterministic ranking;
4. one accepted workflow at a time;
5. protocol, Windows, sanitizer/leak, latency, token, and blind patch-quality gates;
6. migration/rollback documentation.

Each step would use focused tests, DCO-signed commits, and one issue per PR.

Would you prefer the two-tool surface, an existing prompt/profile mechanism, or a different decomposition? I will wait for maintainer feedback before changing MCP schemas or handlers.

Alternatives considered

1. Keep only primitives. No API growth, but every client must independently reproduce project/freshness/coverage safeguards and reconcile several responses.
2. Expose all six downstream tools. Very discoverable, but increases schema tokens and long-term public surface.
3. One generic query_context tool with an operation enum. Lowest schema count, but a polymorphic contract is harder to validate and less clear to agents.
4. Client-only prompt instructions. Useful and possibly the preferred upstream direction, but they cannot by themselves make an unsafe empty result distinguishable from verified empty evidence.

Confirmations

• I searched existing issues and this is not a duplicate.