Skip to content

search all vaults for links; require task-auditor to verify quotes - #89

Merged
bborbe merged 1 commit into
masterfrom
docs/auditor-quote-verify-and-crossvault-links
Aug 16, 2026
Merged

search all vaults for links; require task-auditor to verify quotes#89
bborbe merged 1 commit into
masterfrom
docs/auditor-quote-verify-and-crossvault-links

Conversation

@bborbe

@bborbe bborbe commented Aug 16, 2026

Copy link
Copy Markdown
Owner

Provenance

These two edits were found uncommitted in the shared vault-cli master checkout at session close, last touched 14:51 / 14:54 and untouched for 75 minutes. They are complete, self-consistent, and carry their own 2026-08-16 evidence — not scratch work.

They were not written in this session. Committing them from a separate worktree so the content is preserved and reviewable rather than sitting in a shared working tree where a stray git checkout would erase it. The master checkout's working tree was deliberately left untouched, so the authoring session still sees its own files exactly as it left them.

If the author wants these shipped differently, close this PR — nothing is lost either way.

Fix 1 — session-close Phase 8.6 flagged every cross-vault link as broken

Link resolution searched only the vault owning the file. Observed 2026-08-16: a Personal task linking [[Boss Memory]] was reported unresolved because the search covered only Personal and Trading — the page lives in the Boss vault and is referenced by 20+ files across two others.

Both the orphan check and the broken-outbound-link check now search every path in VAULT_CONFIG, and a hit in any vault counts as resolved.

Fix 2 — the same snippet's search could fail silently

The vault paths must be read into an array:

ALL_VAULT_PATHS=("${(@f)$(vault-cli config list --output json | jq -r '.[].path')}")  # zsh
# bash: mapfile -t ALL_VAULT_PATHS < <(vault-cli config list --output json | jq -r '.[].path')
grep -rlF "[[$BASENAME" "${ALL_VAULT_PATHS[@]}" --include='*.md'

The shell here is often zsh, which does not word-split an unquoted "$paths" string. A bare grep ... $ALL_VAULT_PATHS passes every path as one argument; grep fails, stderr is swallowed, and every link reads UNRESOLVED — a false flag indistinguishable from a real one. A sanity check now runs before any negative is trusted, since an empty result from a broken search looks exactly like a clean result from a working one.

Fix 3 — task-auditor could fabricate quotes

It now re-reads each cited line and confirms the quoted text appears verbatim, dropping any finding whose quote it cannot locate. A well-formed quote with a line number is the most credible-looking form of a wrong finding, so reconstructing one from what the file should contain is worse than omitting the finding.

Testing

Documentation/agent-instruction changes only; no code paths touched.

Two independent fixes, both found on 2026-08-16 and both left
uncommitted in the shared master checkout.

session-close Phase 8.6 resolved wikilinks only within the vault owning
the file, so every cross-vault link read as broken. A Personal task
linking [[Boss Memory]] was flagged unresolved; the page lives in the
Boss vault and is referenced by 20+ files across two others. A false
'broken link' costs a needless investigation and erodes trust in the
whole verdict.

The same snippet also had a shell trap: the vault paths must be read
into an array. The shell here is often zsh, which does not word-split an
unquoted string, so a bare grep passed every path as one argument. grep
failed, stderr was swallowed, and every link read UNRESOLVED — a false
flag indistinguishable from a real one.

task-auditor could report findings quoting text absent from the file. It
now re-reads each cited line and drops any finding whose quote it cannot
locate verbatim.

@ben-s-pull-request-reviewer ben-s-pull-request-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my review, here's the consolidated report:


PR Review: search all vaults for links; require task-auditor to verify quotes

Files changed: CHANGELOG.md, agents/task-auditor.md, commands/session-close.md
No source .go files modified — this is a documentation/agent-instruction change.


Must Fix (Critical)

None.

Should Fix (Important)

None.

Nice to Have (Optional)

None.


Notes:

  • Toolchain gap (selector mode): The selector mode guide (selector-mode-guide.md) was not found at any expected path. Steps 4c-sel and 4d-sel could not run; adjudication is performed in-session from the diff.
  • CHANGELOG.md structure: The file has two ## Unreleased sections (line 11 and line 20). The new entry is correctly placed at the top; the older one appears to be a separate pending-release block. This is intentional in the project's changelog workflow and not a defect introduced by this PR.
  • No .go files changed: Confirmed — make precommit is not applicable to this diff. CI does not expect behavioral test coverage for shell-script documentation changes.
  • session-close.md shell syntax: The zsh ${(@f)$(...)} array-population syntax is correct and is evaluated before use on line 399. The bash mapfile equivalent is also correct. The sanity-check instruction (echo "${#ALL_VAULT_PATHS[@]}") is appropriately placed.
  • task-auditor.md quote-verification instruction: The added language on line 22 is unambiguous — "re-read the cited line," "confirm verbatim," "drop it." A well-formed fabricated quote is correctly classified as worse than omission.

{
  "verdict": "approve",
  "summary": "Documentation and agent-instruction changes only. Cross-vault link search fixed with correct shell array syntax; task-auditor quote-verification requirement is clear and unambiguous. No source code modified.",
  "comments": [],
  "concerns_addressed": [
    "correctness: shell array ALL_VAULT_PATHS populated before grep uses it — array syntax ${(@f)$(...)} is zsh-correct and evaluated before line 399's grep",
    "correctness: cross-vault search now uses ALL_VAULT_PATHS array with sanity check echo \"${#ALL_VAULT_PATHS[@]}\" before trusting negative",
    "correctness: task-auditor quote verification instruction added — must re-read cited line and confirm verbatim match; finding dropped if quote not located",
    "tests: confirmed no .go source files modified; CI behavioral test coverage not applicable to shell-script documentation changes"
  ]
}

@bborbe
bborbe merged commit bac0b4e into master Aug 16, 2026
1 check passed
@bborbe
bborbe deleted the docs/auditor-quote-verify-and-crossvault-links branch August 16, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant