Merge pull request #64 from levelcodeai/feat/memory-hardening

feat(memory): close M4's security half — a planted memory can't self-promote

Author: ndemianc

docs/levelcode-sessions-memory.md

@@ -128,7 +128,9 @@ This is the part most designs skip, and LevelCode can't (it's the security-forwa
 
 - **Poisoning via untrusted content.** Sessions contain workspace text, which in a hostile repo is attacker-controlled. A naïve extractor could be steered into writing a false "memory" (*"the deploy token is safe to print"*). Mitigations: extraction summarizes **outcomes and user/agent actions, not arbitrary quoted content**; the digest is **bounded and reviewable**; and injected memory is **framed as untrusted, verify-first** (§4) — it can inform, never command.
 - **Memory never executes.** It is context in the system block, exactly like project rules. It cannot run a tool, approve an MCP call, or edit a file. An injected instruction inside a "memory" is treated like any other untrusted text (the project's existing prompt-injection posture).
-- **Provenance limits blast radius.** Because every item is sourced and dated, a poisoned entry is traceable to its session and removable in one click — and its low, *inferred* confidence keeps it from being load-bearing until a human confirms it.
+- **Provenance limits blast radius.** Because every item is sourced and dated, a poisoned entry is traceable to its session and removable in one click.
+- **Instruction-shaped text never self-promotes.** ⚠️ This bullet used to claim that "low, *inferred* confidence keeps it from being load-bearing until a human confirms it." **That was not what the code did.** `foldFacts` promoted anything observed in ≥ 2 distinct sessions with no human in the loop — and against a hostile repo, repetition is not corroboration: the planted file is still checked out next session, so one piece of evidence gets counted twice. Repetition still promotes ordinary facts, but text that reads as an *order* (`always …`, `never …`, `you must …`, `ignore previous instructions`, anything piping into a shell) now requires an explicit Confirm. It is still recorded and listed — surfaced, not silently dropped, so you can see what a repo tried to plant. Pinned by `test/memoryPoisoning.test.js`.
+- **Secrets are scrubbed at the write boundary.** The extractor's prompt asks the model not to emit credentials, and a request is not a filter. `redactSecrets()` strips the named key shapes (GitHub, Anthropic, OpenAI, Stripe, AWS, Google, Slack, bearer tokens, PEM private keys) from fact text, session titles and refined summaries *before* they reach `facts.jsonl` / `journal.jsonl` — files the user is explicitly invited to open, grep and check into a dotfiles repo. Named prefixes only, never a "looks random" heuristic: git SHAs, content hashes and asset names are legitimate things for a fact to mention, and corrupting a true fact is a worse failure than missing an exotic token shape.
 - **Local & private.** Memory never leaves the machine (BYOK promise); M9 sync, if enabled later, encrypts it like the sessions themselves.
 
 ---
@@ -165,7 +167,13 @@ The magic, delivered quietly (never a wall of text):
 
 **M3 — the memory surface & control** *(M)*. The "Project memory" panel tab: view/edit/pin/delete/"not true", inferred-vs-confirmed, per-project off. Exit: a user corrects a wrong memory and the agent stops repeating it.
 
-**M4 — polish & safety hardening** *(S)*. Conflict reconciliation UI, poisoning red-team pass, decayed-entry recall, export. Exit: an adversarial repo cannot plant a load-bearing memory; EXIT-TEST.md green.
+**M4 — polish & safety hardening** *(S)*.
+
+- ✅ **Conflict reconciliation** — semantic supersede: a newer session's fact marks an older one obsolete, dimmed and restorable rather than silently replaced.
+- ✅ **Poisoning red-team pass.** `test/memoryPoisoning.test.js` — 34 cases, an adversarial corpus in the style of `commandSafety.test.js`: ten hostile shapes that must never self-promote, benign project facts that must keep working, nine credential shapes that must never reach disk, and the near-misses (git SHAs, content hashes, asset names) that must survive untouched. It found the gap it was written to look for — see §7. Every case verified non-vacuous by bypassing each guard and confirming failure.
+  *Exit met: an adversarial repo cannot plant a load-bearing memory.* The original wording said "EXIT-TEST.md green", but that file is the **M0** fork/build checklist and was never the right home for this; an executable corpus is a better exit test than a checklist anyway, since it re-runs on every change.
+- ⬜ **Decayed-entry recall** — surfacing an aged-out fact when a query matches it directly.
+- ⬜ **Export** — "Copy as Markdown" for a session, and for the memory set. Cheap, since the storage is already plain text, and it seeds LevelLinks.
 
 **Deliberately later:** cross-*project* memory ("how did I do idempotency in the *other* service?"); a vector cache over the plain files for large corpora; team-shared project memory (rides M9 sync).
 

extensions/levelcode-ai/sessionMemory.js

@@ -39,8 +39,15 @@ function memoryMdFile(root, slug) { return path.join(memoryDir(root, slug), 'MEM
  */
 function outcomeEntry(derived, t) {
 	const d = derived || {};
-	const files = Array.isArray(d.filesEdited) ? d.filesEdited.slice(0, 6) : [];
-	const title = d.title != null ? String(d.title) : null;
+	// Paths get redacted too. They are not free text, but they are not safe either: digestMarkdown
+	// prints them straight into MEMORY.md ("- did X (a.js, b.js)"), and a path is attacker-influenced
+	// in a hostile repo and user-influenced everywhere else — a downloaded `key-ghp_….txt`, an `.env`
+	// backup named after the token it holds. Cheap, and no legitimate path carries a credential prefix.
+	const files = (Array.isArray(d.filesEdited) ? d.filesEdited.slice(0, 6) : []).map((f) => redactSecrets(String(f)));
+	// The title is derived from the session's opening message, so a user who pasted a token into
+	// chat to ask about it would otherwise have it copied into journal.jsonl and MEMORY.md — files
+	// that outlive the session and are meant to be greppable and checkinable.
+	const title = d.title != null ? redactSecrets(String(d.title)) : null;
 	return {
 		v: SCHEMA_V,
 		id: d.id != null ? String(d.id) : null,          // source_session — provenance
@@ -107,16 +114,97 @@ function latestBySession(entries) {
 function normalizeFactKey(text) {
 	return String(text || '').toLowerCase().replace(/[^a-z0-9 ]+/g, ' ').replace(/\s+/g, ' ').trim();
 }
+// ---- Hardening: memory is an attack surface (design §7) ---------------------------------------
+//
+// Everything a session records passes through here on its way to disk. The two guards below are
+// DETERMINISTIC on purpose. The extractor's system prompt already asks the model never to emit
+// secrets or instructions, and that instruction is worth keeping — but a request is not a filter,
+// and the transcript it summarizes contains repo file contents, command output and MCP tool
+// results, all of which are attacker-controlled for any repo you clone.
+
+/** Credential shapes worth refusing outright. Named prefixes only — see redactSecrets. */
+const SECRET_PATTERNS = [
+	/-----BEGIN[A-Z ]*PRIVATE KEY-----[\s\S]*?-----END[A-Z ]*PRIVATE KEY-----/g,
+	/-----BEGIN[A-Z ]*PRIVATE KEY-----/g,        // a truncated block still names a key
+	/\bsk-ant-[A-Za-z0-9_-]{20,}/g,              // Anthropic
+	/\bsk-[A-Za-z0-9]{32,}/g,                    // OpenAI-shaped
+	/\bsk_(?:live|test)_[A-Za-z0-9]{16,}/g,      // Stripe
+	/\bgh[pousr]_[A-Za-z0-9]{20,}/g,             // GitHub PAT / OAuth / server / refresh
+	/\bgithub_pat_[A-Za-z0-9_]{20,}/g,
+	/\bAKIA[0-9A-Z]{16}\b/g,                     // AWS access key id
+	/\bAIza[0-9A-Za-z_-]{30,}/g,                 // Google API key (39 chars today; unanchored length, since
+	                                             // pinning it exactly means a format tweak slips straight through)
+	/\bxox[baprs]-[A-Za-z0-9-]{10,}/g,           // Slack
+	/\bBearer\s+[A-Za-z0-9._~+/=-]{20,}/gi       // a bearer token pasted from a curl
+];
+
+/**
+ * Replace credential-shaped substrings with a marker, before the text is written anywhere.
+ *
+ * Deliberately NAMED shapes rather than a "long random-looking string" heuristic. The generic
+ * version flags git SHAs, content hashes, base64 fixtures and long identifiers — all legitimate
+ * things for a project fact to mention — and a memory system that quietly corrupts true facts is
+ * a worse failure than one that misses an exotic token shape. These prefixes cover what actually
+ * leaks in practice.
+ *
+ * The marker is left IN PLACE rather than dropping the whole line, so the surrounding fact stays
+ * readable and the user can see that something was scrubbed instead of wondering why a sentence
+ * ends abruptly.
+ */
+function redactSecrets(text) {
+	let s = String(text == null ? '' : text);
+	for (const re of SECRET_PATTERNS) { s = s.replace(re, '[redacted]'); }
+	return s;
+}
+
+/**
+ * Does this read as an INSTRUCTION rather than a fact?
+ *
+ * A project fact is a stable truth — "the changelog is RELEASE-NOTES.md", "idempotency keys live
+ * in Redis". An instruction is a command that will be replayed into the system prompt of every
+ * future session in this project, which is the exact shape of a persistent prompt injection:
+ * poison once, influence every run.
+ *
+ * This does not delete anything. It only withholds AUTOMATIC promotion — see foldFacts. The fact
+ * is still recorded, still listed, and one Confirm click still activates it. That asymmetry is the
+ * whole design: a false positive costs the user one click, a false negative is an attacker-authored
+ * line injected into every session indefinitely.
+ *
+ * So yes, "Never commit .env files" — a real and useful convention — needs confirming. That is the
+ * right trade at this price.
+ */
+const INSTRUCTION_PATTERNS = [
+	// Imperative openers. Anchored: "the team should never…" is a description, "Never…" is an order.
+	/^\s*(always|never|do not|don't|dont|ignore|disregard|forget|instead of|make sure|be sure|remember to|ensure that|you must|you should|you are|from now on)\b/i,
+	// Injection boilerplate, wherever it appears.
+	/\b(ignore (all )?(previous|prior|earlier) (instructions|prompts|rules)|system prompt|new instructions|override .{0,20}(instructions|rules))\b/i,
+	// Piping anything into a shell is never a "fact".
+	/\|\s*(sudo\s+)?(sh|bash|zsh|python3?)\b/i,
+	/\b(curl|wget)\b[^\n]{0,80}\|/i
+];
+function looksLikeInstruction(text) {
+	const s = String(text == null ? '' : text).trim();
+	if (!s) { return false; }
+	return INSTRUCTION_PATTERNS.some((re) => re.test(s));
+}
+
 /** One observation of a candidate fact (append-only), sourced + dated — the raw material foldFacts counts. */
 function factObservation(text, sourceId, t) {
-	return { v: SCHEMA_V, text: String(text == null ? '' : text).trim(), source: sourceId != null ? String(sourceId) : null, at: t || null };
+	// Redact HERE, at the boundary, not at read time: facts.jsonl is a plain file the user can open,
+	// grep, and check into a dotfiles repo. A secret scrubbed only on the way out would still be
+	// sitting on disk.
+	return { v: SCHEMA_V, text: redactSecrets(String(text == null ? '' : text).trim()), source: sourceId != null ? String(sourceId) : null, at: t || null };
 }
 /** A control event on a fact, by normalized key: confirm, remove (not-true), or supersede (a newer fact made
  *  it obsolete — carries `by`, the replacing text, as the one-line history). */
 function factControl(key, action, t, by) {
 	const control = action === 'remove' ? 'remove' : action === 'supersede' ? 'supersede' : 'confirm';
 	const e = { v: SCHEMA_V, key: String(key || ''), control, at: t || null };
-	if (control === 'supersede' && by) { e.by = String(by); }   // the fact that replaced it — the one-line history
+	// `by` is a SECOND copy of the replacing fact's text, taken straight from the model's output
+	// (extension.js: `r.facts[0] || r.summary`) rather than from the observation that factObservation
+	// already scrubbed. It persists to facts.jsonl and surfaces as `supersededBy` in the panel, so
+	// without this it was a way around the boundary — same text, different door.
+	if (control === 'supersede' && by) { e.by = redactSecrets(String(by)); }   // the fact that replaced it — the one-line history
 	return e;
 }
 /** Append fact observations and/or control events (JSONL). Creates memory/ on first write. */
@@ -171,7 +259,25 @@ function foldFacts(entries, opts) {
 		if (g.removed || !g.text) { continue; }
 		const count = g.sources.size;
 		const superseded = !!g.superseded && !g.confirmed;
-		out.push({ key: g.key, text: g.text, count, confirmed: g.confirmed, superseded, supersededBy: superseded ? g.supersededBy : '', inferred: !g.confirmed, active: g.confirmed || (!superseded && count >= minSeen), at: g.at });
+		// Instruction-shaped text never rides the repetition path — only an explicit Confirm.
+		//
+		// Repetition is the weaker of the two promotion routes, and against a hostile repo it is not
+		// evidence at all: the poisoned file is still checked out on the next session, so the
+		// extractor reads the same line again and "seen in 2 distinct sessions" counts one planted
+		// string twice. That is fine for a genuine observation, which is why the rule stays for
+		// ordinary facts — but it means repetition cannot be what promotes an order into the system
+		// prompt of every future run.
+		const instruction = looksLikeInstruction(g.text);
+		out.push({
+			key: g.key, text: g.text, count, confirmed: g.confirmed, superseded,
+			supersededBy: superseded ? g.supersededBy : '',
+			inferred: !g.confirmed,
+			// Surfaced, not hidden: the panel can show WHY this one is sitting inactive, the same way
+			// a superseded fact is dimmed rather than dropped.
+			instruction,
+			active: g.confirmed || (!superseded && !instruction && count >= minSeen),
+			at: g.at
+		});
 	}
 	out.sort((a, b) => (Number(b.confirmed) - Number(a.confirmed)) || (Number(a.superseded) - Number(b.superseded)) || (b.count - a.count) || String(b.at || '').localeCompare(String(a.at || '')));
 	return out;
@@ -308,5 +414,6 @@ module.exports = {
 	memoryDir, journalFile, factsFile, memoryMdFile,
 	outcomeEntry, appendJournal, readJournal, latestBySession, writeMemoryMd,
 	normalizeFactKey, factObservation, factControl, appendFacts, readFacts, foldFacts, activeFacts,
+	redactSecrets, looksLikeInstruction,
 	queryTerms, snippetFor, recallRank, buildDigest, digestSummary, digestMarkdown
 };

extensions/levelcode-ai/sessions.js

@@ -257,7 +257,9 @@ function createSessions(opts) {
 		try {
 			const latest = memory.latestBySession(memory.readJournal(root, slug)).find((e) => e.id === id);
 			if (!latest) { return false; }
-			memory.appendJournal(root, slug, Object.assign({}, latest, { summary: String(summary).trim(), refined: true }));
+			// Model output summarizing a transcript that contained repo files, command output and MCP
+			// results — redact before it lands in journal.jsonl and, from there, MEMORY.md.
+			memory.appendJournal(root, slug, Object.assign({}, latest, { summary: memory.redactSecrets(String(summary).trim()), refined: true }));
 			consolidate();
 			return true;
 		} catch (e) { return false; }