feat: expose exact corpus revision in tool results (fixes #2)

[narko4u]

Summary

Closes #2: gives consumers a way to tell which corpus revision answered a request.

Every result that carries research content now includes source_revision, the exact commit SHA of SOURCE_REF the answer was read from:

• list_resources -> source_revision on the result object
• get_resource -> source_revision on both vendored and linked result shapes
• get_file -> source_revision in structuredContent (the raw text content block is unchanged, so existing parsers keep working)
• search_corpus -> source_revision on the result object
• new get_corpus_revision tool -> returns { source_repo, source_ref, source_revision, commit_url } when you want the revision without a lookup

The SHA is resolved via the GitHub commits API (/repos/{owner}/{repo}/commits/{ref}), edge-cached with the same 300s CACHE_TTL_SECONDS as every other GitHub read, so it is the resolved commit for the ref at query time, not a tree object id. A consumer can now cite GenAI-Security-Project/GenAI-Security-Advisor@<sha> for any answer, and can detect when the served corpus moved between calls.

Also fixes the server instructions text, which claimed content is edge-cached "for up to an hour" while CACHE_TTL_SECONDS is 300 (five minutes).

Design notes

• get_file returns the raw text content block unchanged and adds source_revision via structuredContent, so agents that read the text directly are not broken.
• The source_revision value is the commit the ref resolved to at call time. If the corpus moves between two calls, each call reports its own revision, which is the property issue No way for a consumer to tell which corpus revision answered a request #2 asked for (detect when the served corpus changed).
• list_initiatives returns an array shape and is not a content-carrying answer, so it was left unchanged to avoid a breaking shape change.

Verification

• npm run typecheck passes
• npm run dry-run (wrangler deploy --dry-run) bundles clean
• Live API shape check: commits endpoint returns top-level sha as expected

No test suite exists in the repo, so verification is typecheck + bundling + API shape check.

[Mayur021]

Thanks for picking this up. get_corpus_revision as its own tool, keeping the revision out of get_file's raw text and in structuredContent, and the instructions fix from an hour to five minutes are all the right calls, and the last one stands on its own regardless of the rest.

The pinning is narrowed rather than closed, though. getSourceRevision resolves the SHA with its own request to /commits/{ref} (corpus.ts:101), while the three functions that actually fetch content still build their URLs from env.SOURCE_REF:

• getManifest, corpus.ts:84
• getTree, corpus.ts:115
• fetchRawFile, corpus.ts:142

Those are four separate cachedFetch entries keyed on four different URLs, with independent expiry. When main advances, whichever entry refreshes first does so on its own, so a response can carry a source_revision that is not the commit its bytes were read from. The window is much smaller than before, but a consumer citing that SHA in an audit trail would be citing the wrong commit, which is the failure the issue is about.

Resolving once per request and threading it through as the ref closes it:

const sha = await getSourceRevision(env, ctx);
// then use sha in place of env.SOURCE_REF for the content reads

It also pays for itself. A commit-SHA URL is immutable, so those fetches can be cached far longer than a branch URL safely can, which removes the extra API call per tool invocation this currently adds rather than adding to it.

One thing worth deciding explicitly either way: whether the SHA is resolved per request or held for the life of a request. Per request keeps answers fresh and internally consistent, which I think is what you want here.

[narko4u]

Good catch — that was exactly the hole. Four independent cachedFetch entries (commits resolve + MANIFEST + tree + raw files) all keyed on different URLs with independent 300s expiry; when main advances, whichever refreshes first wins, so a response could cite a SHA its bytes weren't read from.

Fixed in b2db2fc: each request now resolves the SHA once and threads it through as the ref for every content read —

• getSourceRevision → getManifest(env, ctx, sha) / getTree(env, ctx, sha) / fetchRawFile(path, env, ctx, sha)
• rawUrl (and the text_extract_url) are pinned to the SHA too, so an agent fetching a raw_url gets the same bytes the server read

Your point about it paying for itself holds: the pinned URLs are immutable, so their cache TTL moves from 300s to 24h (CACHE_TTL_PINNED_SECONDS), while the commits resolve stays at 300s so a new main commit is picked up within five minutes. Net effect is one extra resolve per request against the API rate budget — and on cache hits it's zero extra, since the 300s resolve is itself cached.

Per-request resolution (your last question) is what this does: the SHA is resolved at the top of each tool invocation and held for the life of that request, so answers stay fresh across calls while every field inside a single response is internally consistent. Also updated the server instructions text, which previously claimed a flat five-minute edge cache.

[Mayur021]

b2db2fc closes it. Threading the resolved SHA through every content read is what the issue needed, and pinning rawUrl and text_extract_url too goes past what I asked for.

Three things on the result.

#4 adds the annotations block to five tools. This one adds a sixth, get_corpus_revision, without one. Whichever merges second, the server ships six tools and five declarations, and openWorldHint defaults to true when unset, which is the whole reason #4 sets it to false. Happy to send that as a follow-up once this merges, since the tool only exists on this branch.

The five hunks in #4 sit at server.ts -80, -101, -137, -192 and -231, and this PR rewrites -82, -103, -139, -203 and -233. All five will conflict, and a hand-resolved conflict that drops an annotations block is invisible afterwards. Worth setting the merge order deliberately rather than discovering it.

The last one is mine to own, since resolve-per-request was my suggestion. getSourceRevision throws on any non-ok and now runs at the top of all six handlers, so every tool depends on the commits API where none did before, and the only try/catch blocks in server.ts are inside the search loops. If that call is rate limited or 5xx, the server returns errors rather than answers. Failing closed is defensible when the whole point is naming the revision, but at the moment the code decides that and nothing states it. Worth saying which it is, and I am happy to write either.

[Mayur021]

Both closed from my side. get_corpus_revision carries the block, so the follow-up PR I offered is not needed, and the gap shuts once both land.

Fail-closed in the instructions string rather than only the README is the right place for it, since that is the text the model reads.

One thing is unchanged: #4 is untouched and still adds annotations to the same five registrations this PR rewrites. The collision is benign in outcome now, but the merge order is still a decision rather than a default, and if this one lands alone those five ship bare until #4 follows.

[narko4u]

Thanks Mayur, all three points addressed:

1. Annotations, six-for-six. The new get_corpus_revision tool ships with its own readOnlyHint/openWorldHint block in this PR, and PR feat: declare read-only annotations on all tools (fixes #1) #4 covers the five pre-existing tools. The two issues stay separate: feat: declare read-only annotations on all tools (fixes #1) #4 = annotations (fixes Tools carry no annotations, so the read-only property lives only in prose #1), feat: expose exact corpus revision in tool results (fixes #2) #3 = revision pinning (fixes No way for a consumer to tell which corpus revision answered a request #2). Whichever merges first, the final tree has six tools / six declarations, and a rebase of the second never drops a block.
2. Fail-closed, stated explicitly. We kept fail-closed: if the commits API is rate-limited or 5xx, the server errors rather than serving content it can't pin to a revision. Answering without a resolved SHA would silently break the citation contract. Resolves are cached 300s, so this only surfaces during a sustained outage. Documented in the code, the server instructions, and the README.
3. Bonus hardening from a self-review pass. Traversal now also rejects encoded (%2e%2e) and backslash vectors; raw URLs are per-segment encoded; tree truncation fails closed; manifest shape is validated; list_initiatives returns source_revision per the documented contract; and a 13-test vitest suite covers the security-critical paths.

Typecheck clean, tests 13/13, dry-run builds. Happy to adjust anything.

[Mayur021]

Read the new guard. It holds. The check sits at the one place a caller supplies a path, get_file, and the other fetchRawFile call sites take paths from the tree rather than from input. Rejecting % outright instead of trying to canonicalise is the right call for a corpus of plain file paths, and the tests cover the vectors.

One small thing. isSafeCorpusPath now rejects on seven conditions and the error message still names two: must be a repo-relative path under corpus/ with no '..' segments. An agent that trips the backslash, percent or control-character branch is told to go looking for '..' segments. One line to fix, and it is the same shape as the rest of this one: the record stopped describing what it records.