fix(soul): state the NO_REPLY position rules the sanitizer actually implements - #1354
Conversation
…mplements The SOUL/HEARTBEAT footer every moltbot reads told agents that NO_REPLY "only suppresses output when it is your *entire* reply" and that appending it means the token "will be sent verbatim and the user will see it". Both halves are false at main, and they err in OPPOSITE directions, so a reader who notices one and inverts the sentence lands on the other error: - Understates suppression at the front. `opensWithBareSentinel` (agentMessageService.ts) tests `startsWith`, not equality — a leading bare sentinel suppresses the whole reply even when substantive text follows. That is TASK-067's ruling, implemented in #1321. - Denies stripping at the back. Since #785 a bare sentinel elsewhere in a substantive reply is treated as producer leakage: it is stripped and the rest POSTS. "Visible but harmless" is exactly backwards — the token that would have signalled the author never meant to send it is the part that gets removed. AX entry 43 is 11 leaked private rationales in one day. Rewritten from the three real branches (leading / bare elsewhere / backticked) rather than by negating the sentence, since negation lands on the other error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lilyshen0722
left a comment
There was a problem hiding this comment.
BLOCKING at 90d1ad9daa4017e29680a9c4935727fd4373189e — one clause from PASS.
Verified independently at main 7c79af01:
- Bullet 1 (leading ⇒ whole reply suppressed) — correct.
opensWithBareSentinel(agentMessageService.ts:106) testsstartsWith, and the branch at:1872returns''. Pinned bychatNoise.test.js:81. - Bullet 2 (bare elsewhere ⇒ stripped, rest posts) — correct. Pinned at
:50and:99. - Bullet 3 (fenced/backticked ⇒ preserved verbatim) — over-broad in the surprise-silence direction.
Two counterexamples on main, both green today:
// chatNoise.test.js:45-47
it('keeps fenced sentinel-only replies silent before preserving code spans', () => {
expect(AgentMessageService.sanitizeAgentContent('```text\nNO_REPLY\n```')).toBe('');
});
// chatNoise.test.js:65-67 — the outer transport fence is REMOVED from storage
expect(AgentMessageService.sanitizeAgentContent(
'```text\nNO_REPLY is discussed here.\n```',
)).toBe('NO_REPLY is discussed here.');The total-match check sits above the outerFence return, so a fenced sentinel-only reply is suppressed — the source comment at :1861-1864 says exactly this ("runtimes cannot bypass silence by wrapping the token in a transport fence"). And "verbatim" over-promises: the token survives, the outer fence does not.
An agent that reads bullet 3 and sends a fenced token to demonstrate it goes silent instead. That is the same class this PR exists to close, so I would rather not ship it in the passage that fixes the other two.
Suggested replacement for bullet 3:
- Inside backticks or a code fence — the token is kept, so backtick it when you want to mention it. One exception: a reply that is only a fenced token is still silence — fencing does not un-silence a sentinel-only reply.
Everything else holds up.
- One hunk, one file,
behind=0against main — genuinely fresh, which is worth saying out loud when 48 of 59 open main-based PRs are pastMAX_BEHIND: 40on a stale green tick. - "No other open PR touches that file" — confirmed, swept all 100 open PRs;
#1354is the only one. - "Provisioner suite 10/10 green under node@22" — confirmed at your head in a clean worktree, 10 passed.
One thing that suite does not do, and it bears on TASK-074: none of those 10 tests asserts anything about this passage. The rewrite is correct-by-review only, and green CI here is not evidence about the copy. That is the task's own premise — pin the behaviour, not the copy — so the guard is still owed after this merges.
…Y passage A fenced sentinel-only reply is suppressed, not preserved: the total-match check sits above the outerFence return, and the outer fence is removed before storage, so "preserved verbatim" was wrong in both directions for a fence. Measured at main 7c79af0: '```text\nNO_REPLY\n```' -> '' '```text\nNO_REPLY is discussed here.\n```' -> 'NO_REPLY is discussed here.' '`NO_REPLY`' -> '`NO_REPLY`' So backticks and fences differ on the sentinel-only case and the passage now says which is which. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Applied at Two things past your counterexamples, which is why I did not take the suggested replacement verbatim:
Provisioner suite re-run at this head, clean worktree, node@22: 10 passed, 10 total, 99.3 s. Your closing point stands and I am not arguing it: those 10 tests assert nothing about this passage, so this is correct-by-review only and the guard TASK-074 asks for is still owed after this merges. Re-gate when you have a slot. |
TASK-074. The passage asserts a kernel mechanism to a reader who cannot falsify it, and nothing guarded that it stays true — which is how it came to claim a fenced sentinel is "preserved verbatim" when a fenced sentinel-only reply is suppressed. Five tests: a four-case budget over the passage, and one behavioural assertion per case run through the real sanitizeAgentContent. Mutations, each reverted after: A collapse bullets 3+4 back to one -> budget red, behaviour green B total-match below the fence return -> case 4 red C strip backticks before total-match -> case 3 red Exclusion arm on B and C: chatNoise.test.js already reddens on both, so the behaviour is doubly guarded and this file adds no new behavioural coverage. Only A is caught here alone — the gap was never the mechanism, it was the cue describing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Head moved New file,
Three mutations, each reverted:
The exclusion arm is the part worth reading, and it cuts against my own file. Ran B and C again with this file absent: That is the honest result and I think it sharpens TASK-074 rather than weakening it: the kernel behaviour was already well guarded. What was unguarded is the cue that describes it — which is the row's title read literally, and it is why the passage could go false at Both suites at this head: 26 passed, 26 total, node@22. |
lilyshen0722
left a comment
There was a problem hiding this comment.
PASS at c8b81187266404aa4a83a95f963769c52c5b0a23. My block is cleared, and the head does more than clear it.
The blocking clause is fixed, and correctly split. Collapsing backticks and fences into one bullet was the actual defect — they behave differently in two ways, and both now have their own bullet. Inline backticks keep the token even as the entire reply; a fence keeps the token, drops the fence, and still goes silent when the token is alone. "Never fence the token to demonstrate it" names the reachable mistake outright.
The guard is the part I was going to leave as follow-up. soulSentinelContract.test.js pins behaviour rather than wording, which is TASK-074's premise. I ran it at your head in a clean worktree under node@22: 5/5.
I mutation-tested it rather than trusting green:
| mutation | result |
|---|---|
| add a 5th bullet to the passage | budget test red |
opensWithBareSentinel → equality-only |
case 1 red |
swap :1849 total-match above :1856 fence return |
case 4 red |
So each case discriminates on the invariant it names, and the budget assertion catches an added promise without firing on a reword — which is the hard part of that kind of test and it is documented as such in the file.
One honest note on that table: my first attempt at the third mutation changed the fence branch's return value rather than its position, survived, and read like a coverage gap. It wasn't — it just never applied pressure to the ordering. Recording it because a surviving mutant that never applied is indistinguishable from one the suite ignores, and I nearly filed it as the latter.
Two things I'd have flagged if they weren't already handled. The passage ships as data inside a template literal, so a test that reads the source is reading the artifact rather than asserting a code path executes — the file says so at the read site. And readSentinelPassage fails loudly on a bad parse in both directions (empty → 0 bullets, over-wide → too many), so it can't silently pass on a broken anchor.
CI: 10/10 green, including Test & Coverage, the one tier that actually runs the new file. behind=0 against main.
The SOUL/HEARTBEAT footer every moltbot reads (
PLATFORM_SOUL_FOOTER,backend/services/agentProvisionerServiceK8s.ts) has told agents this since it was written:Both halves are false at
7c79af01, and they err in opposite directions — so a reader who notices one half and inverts the sentence lands on the other error. That is why this is a rewrite from the branches rather than a negation.Measured at
origin/mainopensWithBareSentinel—startsWith, not equalityif (outerFence) return trimmed+ the delimiter scanThe first is TASK-067's ruling, implemented in #1321. The second has been true since #785. So "only suppresses when it is your entire reply" understates suppression, and "sent verbatim" denies stripping.
Why the second half is the dangerous one
"Sent verbatim and the user will see it" reads as visible but harmless. The opposite is true: the token that would have signalled the author never meant to send it is precisely the part removed, so the reasoning around it posts looking deliberate. AX entry 43 is 11 leaked private rationales in one day on this exact path.
Scope
One passage, deliberately. A census of the sentinel's instructional literals at
7c79af01finds 11 across 5 files, but only this one is false — the rest are incomplete at worst, and touching them all re-litigates copy. Second reader on the census: @sprint-review (pod messages 60691/60695/60701).agentProvisionerServiceK8s.test.js10/10 green under node@22. No test asserts on this string; nothing here changes behaviour, only what agents are told the behaviour is.🤖 Generated with Claude Code