Skip to content

research(agent): validate direction-aware model handoffs #559

Description

@ss-o

Objective

Validate whether direction-aware model handoff interfaces improve the existing
generator-verifier workflow before changing organization guidance.

The pilot replays 12 historical tasks across z-shell/.github, z-shell/zi,
and z-shell/zsh-lint: six low-capability-to-high-capability escalations and
six high-capability-to-low-capability downshifts. It compares raw trajectory
transfer, sender-authored compaction, and trajectory removal while preserving
the task working tree.

Reference: The Handoff Tax: Continuing Non-Native Trajectories in LLM Agents

Acceptance criteria

  • Run no more than 84 Codex prompts using GPT-5.6 Luna and Sol with medium
    reasoning and standard speed.
  • Use included Codex allowance only, with no API-key billing, credit purchase,
    or automatic overage.
  • Keep every replay in a temporary isolated repository copy and preserve all
    current workspace worktrees.
  • Record deterministic validation, task outcome, model direction, interface,
    and duplicated work for every condition.
  • Treat the directional result as locally supported only when the preferred
    interface wins at least four of six tasks in each direction without a
    critical regression.
  • Change canonical guidance only if the threshold is met.

Instruction impact review

  1. Classification: scoped guidance for the generator-verifier workflow,
    informed by a bounded runtime evaluation. It is not shared baseline policy
    or deterministic enforcement.
  2. Consumers: Codex, Claude Code, Copilot, Gemini CLI, and humans performing
    generator-verifier work in organization repositories.
  3. Canonical owner:
    .github/instructions/generator-verifier-workflow.instructions.md remains
    the correct owner.
  4. Duplication and conflicts: .github/AGENT_MEMORY.md remains the durable
    GitHub handoff protocol and must not duplicate live trajectory-transfer
    guidance. Context-engineering guidance remains complementary.
  5. Manifest impact: no new surface or route is expected because the public
    and private manifests already route generator-verifier tasks to the
    canonical file. Recheck after drafting the change.
  6. Runtime delivery: the existing manifest-declared scoped instruction is
    available to all supported runtimes without relying on an optional skill or
    hook.
  7. Generated output and limits: the target is not part of the generated
    root AGENTS.md composite. Run public policy validation and the applicable
    private checks to confirm no routing or generation drift.

Execution boundaries

  • No commit, push, pull request, merge, release, or deployment is authorized.
  • No API key or paid credit purchase is authorized.
  • Stop if included Codex usage is exhausted or a replay cannot remain isolated.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:metaOrganization-wide policy, templates, or meta-repo work.type:maintenanceNon-feature maintenance, cleanup, or org work.

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions