feat(sessions): search a project's earlier agent sessions — codegraph sessions + codegraph_sessions - #1702
feat(sessions): search a project's earlier agent sessions — codegraph sessions + codegraph_sessions#1702bompus wants to merge 3 commits into
Conversation
… sessions + codegraph_sessions A code graph answers "how does X work"; it cannot answer "why is X like this" or "what did the last session decide about Y". That history lives in the transcripts the agent already wrote — hundreds of megabytes of JSONL nobody greps. This indexes the prose of a project's Claude Code sessions (~/.claude/projects/<slug>/: prompts, replies, compaction summaries; tool traffic and thinking stay out) into an FTS5 table with porter stemming and BM25 rank, in its own .codegraph/sessions.db beside the graph so the graph's schema, migrations and bulk-load FTS rebuild stay untouched. Refresh happens on query and re-reads only files whose size or mtime moved: 237 transcripts (238 MB) index in ~6 s the first time and ~140 ms after. Hits name session id, title, role, time and the matching passage; role, sinceDays, session (id prefix) and any (OR the words) narrow or widen. The tool joins codegraph_explore in the default MCP surface — a different question over a different corpus, so it cannot steer a mis-pick against explore — and `codegraph sessions` prints the same text for subagents without MCP. "sessions": false in codegraph.json opts a project out; CODEGRAPH_SESSIONS_DIR points at another transcript directory. Readers are one module per agent host, Claude Code first.
…ts are indexed Two findings from the first live verification of codegraph_sessions. Parallel tool calls run on the daemon's worker threads, one connection to sessions.db each, and every one of them sees the same changed transcript. node:sqlite's busy timeout is zero, so all but the first failed with "database is locked". The connection now waits (busy_timeout 5 s), and a file is re-indexed under BEGIN IMMEDIATE after re-reading its row, so the threads that lost the race skip the file instead of indexing it twice. A prompt the user sends while a turn is running is stored by Claude Code as an attachment entry (attachment.type "queued_command"), not a user message, so the reader never saw it. It is now indexed as the user. PRAGMA user_version marks the reader version; an index written by an older reader is re-read once in full (about 2 s for 239 transcripts).
|
Pushed a second commit (b1ab174) after the first live run of Parallel calls failed with "database is locked". In daemon mode read tools run on the worker-thread pool, so several Prompts sent mid-turn were never indexed. Claude Code stores a message the user sends while a turn is running as an Tests: |
… which race the same way
|
Third commit (5397d49): busy_timeout is now set before the constructor's CREATE TABLE and user_version writes, which race between worker threads the same way the per-file writes did. Verified from a live Claude Code session: six parallel codegraph_sessions calls in one response, all succeed. |
A code graph answers "how does X work"; it cannot answer "why is X like this" or "what did the last session decide about Y". That history lives in the transcripts the agent already wrote — for one active project here, 237 JSONL files and 238 MB that nobody greps. This adds a second tool over that corpus.
What it does.
codegraph_sessions(MCP) andcodegraph sessions <words>(CLI) run a full-text search over the prose of a project's Claude Code sessions: user prompts, assistant text and compaction summaries. Tool calls, tool results, thinking blocks, meta entries and anything under 20 characters stay out. FTS5 with porter stemming and BM25 rank, sotrimmedfindstrimmingand the passage that says most about the words comes first. A hit names its session id, title, role, timestamp and the matching passage with the words marked;role,sinceDays,session(an id prefix) andany(OR the words) narrow or widen. The output is text an agent can act on: the session id is whatclaude --resumeand the transcript file take.Where the index lives.
.codegraph/sessions.db, its own file besidecodegraph.db, through the samecreateDatabaseadapter. Kept separate on purpose: putting the tables in the graph database touchesschema.sql,migrations.tsand the bulk-load FTS rebuild for no query gain, and the graph'snodes_ftsuses unicode61 without stemming, which is right for identifiers and wrong for prose. The index refreshes on each call for files whose size or mtime moved and forgets files that are gone. Measured on the 237-file project: 6.5 s for the first index on the bundled Node, ~140 ms for a query afterwards with one live session changed.Surface. The tool joins
codegraph_exploreinDEFAULT_MCP_TOOLSand in the tiny-repo core set. It is a different question over a different corpus, so it cannot steer a mis-pick against explore, and the measured argument for one tool — fewer mis-picks between look-alike code tools — does not reach it. The server instructions gain one routing bullet ("why is this like this / what did the last session decide" → sessions) and say "for code there is one tool" instead of "a single tool"."sessions": falseincodegraph.jsonopts a project out (the tool then answers with the switch's name);CODEGRAPH_SESSIONS_DIRpoints at another transcript directory, which the tests use. The Claude Code slug is the project path with non-alphanumerics as-; on Windows the drive letter is lowercased in practice, so the lowercased slug is tried second.Shape.
src/sessions/claude-code.tsis the reader (one module per agent host, so a Cursor or Copilot reader is a second file with the sameSessionDocoutput);src/sessions/index.tsis the index and query (SessionsIndex,querySessions,formatSessionHits). Every query word is quoted before it reaches FTS5, so flags, paths and punctuation cannot break the match syntax. Nothing in the graph's schema, extraction or explore path changes.Tests.
__tests__/sessions-index.test.ts(doc extraction, quoting, stemming, filters, incremental refresh, a replaced and a deleted file) and__tests__/cli-sessions-command.test.ts(end to end against the built CLI withCODEGRAPH_SESSIONS_DIR);mcp-tool-allowlistupdated for the two-tool default;mcp-tool-annotationsandmcp-unindexedpass unchanged.tscclean. The full suite on my Windows host shows no new failure against the same host's baseline (the failures there are EPERM temp-dir cleanup and unbuilt viewer assets, the same set with and without this change).Coverage. Claude Code only, in this PR. Of the nine agents the README supports, the others keep their conversations in stores this reader does not open: Codex CLI, Gemini CLI and the Copilot CLI write per-session JSON or JSONL files under their own home directories (
~/.codex/sessions/,~/.gemini/tmp/<hash>/chats/,~/.copilot/session-state/), each a reader module of the same shape asclaude-code.tsonce the project can be matched to a session; Cursor, Antigravity and Kiro are VS Code descendants that keep chat inside the IDE's SQLite state, and OpenCode and Hermes Agent keep theirs in their own SQLite databases. A project with none of these indexed answers with the guidance text, not an error. TheSessionDoccontract (ts,role,textper turn, one file per session) is what a second reader has to produce; the FTS index, the CLI and the tool do not change per host.If the shape is acceptable, the other hosts are follow-on PRs of one reader module each — Codex CLI, Gemini CLI and the Copilot CLI first, since their stores are plain files — and I am glad to take them, or to review someone's who has the store to test against.
Not in this PR: those readers, embeddings, cross-project search, watcher-driven refresh, a viewer tab.