Skip to content

fix(harness-config): align audit references, the permission reader and the effort pin check with the docs - #13288

Merged
kyle-sexton merged 10 commits into
mainfrom
fix/10456-harness-config-conformance
Nov 4, 2026
Merged

kyle-sexton merged 10 commits into
mainfrom
fix/10456-harness-config-conformance

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Refs: #13286

Summary

Brings harness-config's audit references, the permission reader and the effort pin check in line with the Claude Code docs as read on 2026-11-04, for the 2.1.297 to 2.1.318 changelog recheck.

Fix

  • /harness-config:audit-permission-state: the reader takes the project file from the primary working directory. Below the repository root it also reads the root's shared file as the project toplevel surface, because a Bash cd moves the reader and not the session. The lint keys values and rule findings by file (project:toplevel for the root file). --merge-only keeps the note that one of the two files may be over-reported. The skill says to run the reader from the session's start directory: after a cd from a subdirectory that is not the root, that directory's file is not read.
  • /harness-config:audit: check-effort-pins.sh finds the defaults item by content, not by item number, and the baseline is rebaselined. The fallbackModel chain cap is read from model-config's Fallback model chains section, where the page now states it. The stale Bash(git push) suppression is removed from .claude/audit-pass.md and docs/conventions/harness-config.yaml.
  • /harness-config:audit-instructions, audit-pass and audit-permission-grants, plus the plugin README: references corrected toward the pages, and dates refreshed where a claim was rechecked.

Verification

Suites, all green on the final head: permission-state 96, primary-dir 36, permission-plane-lint 121, permission-merge 92, automode-entry-diff 76, audit entry point 37, audit-engine 536, check-effort-pins 80, permission-rule-check 434, guardrails coverage-manifest 32. Earlier this round, on code these last commits did not touch: managed-conformance 39, no-writes 12. Changelog checks: parity --check, --check-bump, --check-preserved, --check-order and fragments --check-required all pass. Each behavior change was written test first and failed before its fix.

Disclosures:

  • The re-decision of the 54 effort pins was done by an agent, not a person. The rebaseline step asks a person to read the changed rows and defaults. An agent compared the 2026-10-11 and 2026-11-04 pages instead: the Levels table and the Choose rows are unchanged, and the defaults item moved from 3 to 5 with the same values. All 54 pins keep their level. The commit message holds the full record.
  • Existing tests changed, and why each still checks as much:
    • In primary-dir, "root's rule not in effect" now asserts the rule is filed under project toplevel and not under project settings.
    • permission-state.test.sh's exactly-once count for the project scope is now one count per surface. Each surface is still checked to appear exactly once.
    • check-effort-pins' "no item 3" case is now "no defaults item", and a sibling case covers an item 3 that states no default.
    • In audit-engine, the shared fixture's settings-reference fallbackModel section no longer states the cap, matching the live page. It now links model-config, and the fixture gains a model-config page and an index line. Case 52 was rewritten around that page. Case 10 is unchanged and still passes, now reading the cap from model-config.
  • The effort pin hash includes the item number with the defaults text. A renumbered list with no other change therefore flags every pin, a false positive in the safe direction. A defaults item reworded so it no longer says "default" drops out of the hash after a rebaseline, and only a person rereading the page would catch that.
  • permission-rule-check.sh roots are unchanged. It reads the worktree root and the git toplevel, so it misses the main checkout's local file and a subdirectory's shared file.
  • plan-reviewer stays pinned at medium. It renders findings, and the high row reads closer, but the page does not settle it.
  • audit-automation-gaps still takes its root from --show-toplevel (inventory.sh:151-152), so it does not follow the primary-working-directory rule either.
  • plugins/harness-config/skills/audit/SKILL.md:67 now names model-config as the fallbackModel cap's source (commit bb8683ff, wording only).

Related

Bot-written tracking issue for the docs-conformance recheck: #10456.

🤖 Generated with Claude Code

kyle-sexton and others added 8 commits November 4, 2026 09:14
… with the docs

The settings page reads the shared project file from the primary working
directory, so permission-state.sh now reads it there and resolves the local
file's repository root from the same place. A new suite runs the reader over
real git trees, since the fixture seam sets both directories to one path.

The remaining findings correct references toward the pages read on 2026-11-04,
and the effort table baseline moves to the current hash with its levels unchanged.

Refs: #10456

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clears the inherited git environment, matches the local file by its tail so a
Windows git path spelling cannot fail the case, and rewords two markdown and
fragment lines the linters reported.

Refs: #10456

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aseline

check-effort-pins.sh hashed the list item numbered 3. The page now puts
`Launching with --effort` at item 3 and the model defaults at item 5, so the
baseline written for this page did not cover the defaults. The script now hashes
every numbered item of the section that says "default", and reports
no-defaults-item (exit 3) when none does.

Proof on real pages: the fixed script still prints the old baseline 8e796135 for
the 2026-10-11 copy of the page, so nothing else moved. The 2026-11-04 page hashes
07672ad7; a copy with only the Opus 4.7 default edited hashes 2a4dfd1b. A live
fetch reports table status=same against the new baseline.

Rebaseline reading (audit SKILL.md Rebaseline step). This reading stands in for
the person that step names: an agent read the rows and decided, no human did.
Compared the three hashed parts of the 2026-10-11 copy (which hashes to the old
baseline) with the 2026-11-04 page using diff:

- Levels table: no change.
- Level rows of the Choose table: no change.
- Defaults item: moved from 3 to 5 and reworded. Same values: high on every model
  that supports effort, medium on Opus 5.5, Sonnet 5.5 and Haiku 5.5, xhigh on
  Opus 4.7, and an organization default still applies to its default model.
- Outside the hash: the resolution order split its first item, so --effort now
  ranks below a level picked in the session.

Decisions: every pin keeps its level, because the guidance each was chosen against
is unchanged. All 54 pins name a listed level.

- Lanes keep their levels. harness-ops lanes config: work high, work-2 high,
  decide high, babysit medium. loop-lane-prompts launch lines: high, medium, high.
  An unattended lane has nobody picking a level, so --effort holds; an operator
  who picks one in the attended queue is making that choice. The merge lane's
  medium is still the launching model's default on Opus 5.5.
- Implementers, medium, day-to-day engineering with a clear scope: implementation
  implementer, scoped-implementer; workflow workers in research-sweep,
  docpage-digest-batch, drift-audit, plan-panel and fix-until-green.
- Mechanical or bounded readers, medium: code-tidying comment-reviewer, discovery
  explorer, performance go-faster-sweeper, planning upload-reader, review
  ecosystem-specialist, songwriting object-writer; retrieval roles in fanout-sweep
  and fix-until-green.
- Verdicts and verification, high: discovery intent-tracer, research-verifier,
  researcher; implementation phase-verifier; plugin-quality auditor; review
  architecture-guardian, brief-reviewer, ci-log-auditor, code-reviewer,
  doc-drift-detector, security-reviewer; user-experience evaluator; verification
  confirm; workflow verifier and orchestrator roles in research-sweep,
  docpage-digest-batch, drift-audit, plan-panel and fix-until-green.
- planning plan-reviewer stays medium. It renders findings on a plan, which reads
  closer to the high row, but the page does not settle that and nothing it rests
  on changed, so it is reported, not changed.

Refs: #10456

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…epository root

A Bash cd inside the project moves the reader's working directory and not the
session's primary one, so after a cd the reader took a subdirectory's file as the
project scope and left the primary directory's rules out, denies included.

The reader cannot tell that case from a session started in the subdirectory, whose
shared file really is the subdirectory's. Falling back to the root would drop that
session's denies, and $PWD alone drops the other's. Reading both omits neither and
can only over-report, so below the root it now also reads the root's file as
`project toplevel`, and a note names both directories. A root or outside-repository
run reads $PWD alone and reports the surface not-applicable. It stays a surface of
the project scope so the merge and lint stages treat it as project settings.

Refs: #10456

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rovals land

The grants check's comment now points at the settings section instead of stating
where a prompt approval lands. The two audit-instructions records whose claims this
branch edited take today's date, after rereading the sections they cite: the
residency table's memory, skills, subagents, hooks and output-styles sections, and
the thinking table, troubleshooting page and extended-thinking section.

Refs: #10456

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… note

With two project files, the lint held one value per scope and key, so a value in
one file hid the other's: a boolean disableAutoMode in the working directory gave
no finding when the root file set "disable". The root file is now its own entry,
named project:toplevel in findings. Other scopes stay last-wins, because a later
managed drop-in does replace an earlier value.

--merge-only dropped the reader's note that the merged set can over-report one
project file, while its effective lines read scopes=project either way. It now
keeps that note and still drops the others.

Refs: #10456

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… root rule findings

The audit engine read the fallbackModel chain cap from settings-reference's
fallbackModel section, which no longer states one, so every chain got a skip row.
It now reads the Fallback model chains section of model-config, requested as a
linked page only when a settings file sets fallbackModel. An unread page, or one
without that section, gives a not-inspectable row.

The permission plane lint named a rule finding from the repository root's shared
file as `project`; it now names it `project:toplevel`, as conf findings already
were.

The permission-state wording no longer implies the session's own shared file is
always read: after a cd from a subdirectory that is not the root, it is not, so
run the reader from the start directory.

The suppression for the bare `Bash(git push)` ask row goes, since the engine no
longer emits that claim.

Refs: #13286

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 4, 2026 15:52
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T15:56:44.729268Z ef71958 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

…urce

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed ea8663ad5a11d2f12d6053cf37eb35027a908a60: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental, blocking only): no security issues found

critical-findings: 0

Scope. This review covers only what changed since bb8683ffcb7d6e9048592d7cb99cc15f488ef22a. One file changed, and I read every hunk of it: plugins/harness-config/skills/audit-pass/reference/doctor-handoff.md. This lane already reviewed an earlier head, so this review reports only CRITICAL and IMPORTANT findings.

What changed. One paragraph in the audit-pass doctor-handoff reference was reworded:

  • Before: it stated as fact that a skillOverrides entry for /doctor is a second documented way to hide the handoff.
  • After: the audit still checks for a skillOverrides entry, but it names the entry as a suspected cause only when the live-fetched skills pages say an override reaches a bundled skill.
  • The pointer now says to fetch the pages before naming the cause, not when naming it.

What I checked:

  • Instruction-surface lens. The change narrows when a diagnostic names a likely cause. Nothing that was forbidden is now allowed, and no rule guarding a protected class was removed. If the rule were missing, the only effect would be a different suspected cause in a read-only audit report: no action, credential or external write is affected. The check now has to be backed by a live-fetched doc, so the evidence standard is tighter than before.
  • Trust controls and manifests. The file matches none of the trusted-path globs. It touches no trust control, merge hold, workflow, token scope or plugin manifest.
  • Spend. Nothing in this change costs money.
  • Embedded instructions. The PR context and the changed text contain no instructions aimed at this lane.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed ea8663ad5a11d2f12d6053cf37eb35027a908a60: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #13288, incremental since bb8683f (blocking only)

Scope: One file changed since the last review: plugins/harness-config/skills/audit-pass/reference/doctor-handoff.md. I read its whole change. This lane has already reviewed an earlier head (blocking only: yes), so this review reports blocking findings only.

Findings: 0.

What changed and what I checked:

  • Old text: The "Suppression channels" paragraph said a skillOverrides entry for /doctor was a second documented way to hide the handoff, and named it as a suspected cause in every case.
  • New text: The pass still checks a skillOverrides entry. It names that entry as a suspected cause only when the linked skills pages say an override reaches a bundled skill.
  • Pointer: It now reads "before naming" instead of "when naming", which matches the new condition. The pointer's "As of" (2026-11-04) and its recheck trigger ("either section stops letting an override hide a bundled skill") are unchanged and still fit.
  • Settled thread: This edit is the owner's fix for the settled thread at original line 63, which is marked lines changed since resolution: no. I did not raise that point again.
  • Dropped as advisory: The closing paragraph at doctor-handoff.md#L73-L77 still names "a skillOverrides entry naming the skill" among the suspected causes without the new condition. It is a small wording mismatch in a reference document and changes no behavior.

Prior findings: The earlier summary comment had no line anchor and covered files this push did not touch. None of my inline findings fall in this incremental scope.

Closing issue: None. The PR refers to #13286 but closes no issue.

Embedded text: None found in the PR context, settled-threads or prior-findings files.

Verification: I ran no commands. I checked the change by reading the diff and the head file. The author's suite results are author-claimed and unverified.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ef71958791

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…ls page

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit 95f817e Nov 4, 2026
80 of 84 checks passed
@kyle-sexton
kyle-sexton deleted the fix/10456-harness-config-conformance branch November 4, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant