Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
| `since` | | Only advisories created after this date (`2026-10-22` or RFC 3339) are triaged. Required unless `advisory` is set. |
| `advisory` | | Triage this GHSA only, whatever its state and creation date. |
| `engine` | `claude` | `claude` (Claude Code and the Anthropic API) or `openai` (Codex and the OpenAI API). |
| `model` | `claude-opus-5-5` / `gpt-5.5` | Model used by both the agent and the duplicate search. `gpt-6.1-sol` with a Codex access token. |
| `model` | `claude-opus-5-5` / `gpt-5.5` | Model used by the reader, duplicate search and exact confirmation. `gpt-6.1-sol` with a Codex access token. |
| `daybreak` | `false` | Turn on OpenAI's Daybreak cyber access program in Codex. `openai` engine only, and the API key's organisation must be approved for it. |
| `runner-image` | tamis agent image, pinned by digest | Image the agent runs in. It must contain the agent CLI, git, curl and `harness-proxy`. A custom image skips the attestation check. |
| `references` | | Whitespace-separated local git references, one for each supported branch. The first is analysed, the code the report cites is located on all of them. Required, at most ten. A missing reference fails the analysis. |
Expand All @@ -27,11 +27,11 @@ Fetch the configured references with `fetch-depth: 0`. For php-src, include the

## Private reports

The private artifact contains a full `.md` report and a `.comment.md` copy of the reporter-visible text. Both list the same duplicates, and the full report also notes a duplicate search that could not complete. Each copy names the source commit and input revision it covers. When an agent run fails, an `.agent-error.txt` file keeps the agent's error output and the error events of its JSON output, which may quote the report.
The private artifact contains a full `.md` report with every duplicate proposed by the global search and a `.comment.md` copy of the reporter-visible text, limited to published duplicates and separately confirmed exact unpublished duplicates. The full report also notes a duplicate search or confirmation that could not complete. Each copy names the source commit and input revision it covers. When an agent run fails, an `.agent-error.txt` file keeps the agent's error output and the error events of its JSON output, which may quote the report.

## Resource budgets

An execution permits at most 800 provider requests and 256 MiB of cumulative request bodies. Each advisory shares 200 requests and 64 MiB across its reader run and comparisons; each agent token additionally permits 150 requests, not counting the refused routes Codex calls on every run. A request body cannot exceed 1 MiB. API responses request at most 16,000 output tokens; the ChatGPT backend has its own output limits. The thirty-minute advisory deadline also cancels host-side git commands. At most ten reports and one report per known reporter are attempted per execution, and a report is only started while the execution still has a whole advisory budget, otherwise it waits for the next execution. SDK retries are disabled.
An execution permits at most 800 provider requests and 256 MiB of cumulative request bodies. Each advisory shares 200 requests and 64 MiB across its reader run and comparisons; each agent token additionally permits 150 requests, not counting the refused routes Codex calls on every run. A request body cannot exceed 1 MiB. API responses request at most 16,000 output tokens; the ChatGPT backend has its own output limits. The thirty-minute advisory deadline also cancels host-side git commands. At most ten unpublished exact candidates are separately confirmed. At most ten reports and one report per known reporter are attempted per execution, and a report is only started while the execution still has a whole advisory budget, otherwise it waits for the next execution. SDK retries are disabled.

These bounds limit each execution, not spending across scheduled runs. Use a dedicated provider account/project with an enforced spending or credit ceiling, restrict models, disable hosted tools and network allowlists there, and configure billing alerts. Provider pricing, tokenisation and ChatGPT credit policies differ, so byte/request limits are not a monetary guarantee. Keep a workflow `timeout-minutes` and a single concurrency group. Budget exhaustion leaves a report retriable rather than marking it permanently complete.

Expand All @@ -50,7 +50,7 @@ The comment lists the files it was checked against. Without any of them, the sec

## Engines

| Engine | Agent CLI | Duplicate search | Model API |
| Engine | Reader and global search | Exact confirmation | Model API |
|---|---|---|---|
| `claude` | Claude Code | Anthropic API | `api.anthropic.com` |
| `openai` | Codex | OpenAI Chat Completions | `api.openai.com` |
Expand Down
4 changes: 2 additions & 2 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ The comment is posted for real. An advisory with a completed comment from the co
| `internal/triage` | Advisory selection and the triage pipeline |
| `internal/sandbox` | The agent container, through harness |
| `internal/modelproxy` | The proxy keeping the model key on the host |
| `internal/engine` | The in-process model loop of the duplicate search |
| `internal/tools` | The duplicate search's read-only tools |
| `internal/engine` | The in-process model loop of exact duplicate confirmation |
| `internal/tools` | The exact duplicate confirmation's read-only tools |
| `internal/report` | Report schema, verification and rendering |
| `internal/repo` | Read-only access to tracked files |
| `internal/github` | GitHub REST client |
Expand Down
12 changes: 9 additions & 3 deletions docs/how-it-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,20 +23,26 @@ The JSON is decoded strictly: an unknown or mistyped field sends the error back

## 3. Duplicate search

One offline agent then searches every other advisory of the repository for the same bug, whatever their state: published, in triage, draft or closed, since a closed duplicate tells the maintainers the issue was already handled. It gets the report, the clues whose location exists in the checkout, and one file per advisory, which it greps and reads the way a maintainer would, and the code read-only for the cases the texts leave open. The report, the clues and the advisories remain hostile data. It answers with the identifiers of the duplicates it found, each with a confidence and an exact flag, and an identifier outside the advisories it was given is refused. All models share the advisory and run budgets.
One offline agent then searches every other advisory of the repository for the same bug, whatever their state: published, in triage, draft or closed, since a closed duplicate tells the maintainers the issue was already handled. It gets the report, the clues whose location exists in the checkout, and one file per advisory, which it greps and reads the way a maintainer would, and the code read-only for the cases the texts leave open. The report, the clues and the advisories remain hostile data. It answers with the identifiers of the duplicates it found, each with a confidence and an exact flag, and an identifier outside the advisories it was given is refused.

The full search result is preserved in the private report. Each unpublished candidate called exact with high confidence is checked again in its own model context, which receives the report, that candidate without its identifier, the checked clues and the code. It returns one `exact` boolean. Only the host knows which candidate that answer belongs to and copies a confirmed identifier into the public comment. All models share the advisory and run budgets. The log records the confirmation duration, usage and number of confirmed candidates.

Both stages compare concrete defects, including secondary findings in a report. A different PoC, version, configuration or severity estimate does not turn the same defective operation into a different bug. An exact match requires positive evidence of a shared defect; a similar symptom or a broad patch fixing several independent defects is insufficient. An exact match does not settle any additional findings in either report, which maintainers must still review before closing it.

Matching is separate from judging validity: an issue already fixed or rejected under the security policy can still be a duplicate. A finding explicitly split into a separate advisory, however, no longer belongs to the current report's scope.

## 4. Verification and comment

Before posting, tamis checks every claim against GitHub and the checkout:

| Claim | Check |
|---|---|
| Duplicates | Must be one of the advisories the search was given. An unpublished advisory appears, by identifier only, when the search calls it an exact duplicate with high confidence. Partial matches on unpublished advisories are dropped. |
| Duplicates | Must be one of the advisories the search was given. Its complete result is preserved privately. An unpublished advisory appears publicly, by identifier only, after its exact, high-confidence result passes a separate one-candidate confirmation. |
| Issues and pull requests | Must exist. Titles and links come from GitHub, not from the model. |
| Cited code | Only references named in the report are kept, each marked as found in the repository or not. A cited line and the code the report quotes there are kept only when the report contains them. The quote is looked up in the file at the scanned commit, nearest the cited line, and when it is gone, `git log -S` names the commit that last changed it, since reports often target another version. |
| CWE and CVSS | Must be well-formed. CWE names come from the MITRE catalogue. |

Claims that fail a check are dropped, and the comment counts them. Just before publication, the current report and non-bot comments are read again; if they changed, no result is posted. The public comment is saved before publication. A failed duplicate search leaves the duplicates out of the comment, is marked in the private report, and does not invalidate the published triage. A failure before publication leaves the advisory retriable.
Claims that fail a check are dropped, and the comment counts them. Just before publication, the current report and non-bot comments are read again; if they changed, no result is posted. The public comment is saved before publication. A failed duplicate search leaves the duplicates out of the comment; a failed exact confirmation leaves that unpublished candidate out. Both are marked in the private report and do not invalidate the published triage. A failure before publication leaves the advisory retriable.

## The comment

Expand Down
2 changes: 1 addition & 1 deletion docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ tamis reads text written by an outside reporter, with access to embargoed adviso

| Threat | Mitigation |
|---|---|
| A prompt injection makes the agent leak other advisories | The duplicate search is one offline agent that reads the report next to every other advisory, embargoed and closed ones included, so it can search them the way a maintainer would. It only returns identifiers from those advisories, each with a confidence and an exact flag, and tamis writes the comment from templates. An embargoed advisory reaches the comment only as the identifier of an exact, high-confidence duplicate, without its title or text. A hostile report or a hostile advisory can still push the search to that answer, which tells the reporter that another private report exists on the same bug, or sway its verdict on another advisory. A failed search leaves the duplicates out of the comment rather than blocking it. |
| A prompt injection makes the agent leak other advisories | The duplicate search is one offline agent that reads the report next to every other advisory, embargoed and closed ones included, so it can search them the way a maintainer would. Its complete result is preserved privately; published matches remain reporter-visible. Each unpublished exact, high-confidence candidate is confirmed in a separate model context holding that candidate only; the host owns its identifier and publishes it only when this second model returns `exact: true`. A hostile report can still push confirmations to disclose candidate identifiers, but an instruction hidden in one candidate only reaches that candidate's confirmation. A failed search or confirmation leaves unconfirmed unpublished identifiers out of the comment. |
| The agent leaks secrets | The container holds no secret. The model key stays on the runner, the container receives a clean single-branch clone with an empty `.git/config`, and the GitHub token never enters it. |
| The agent exfiltrates data over the network | The container sits on an isolated internal network and only knows a private egress sidecar. The model proxy pins the model and rejects remote URLs, remote MCP servers, hosted tools, stored conversations and background calls. |
| The comment loads a remote image carrying data, or pings people | All model text is escaped: no links, images, HTML, entities or mentions. |
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ require (
github.com/git-pkgs/cwe v0.1.1
github.com/openai/openai-go/v3 v3.85.1
github.com/stretchr/testify v1.12.1
golang.org/x/sync v0.23.0
)

require (
Expand Down Expand Up @@ -37,7 +38,6 @@ require (
go.yaml.in/yaml/v3 v3.0.5 // indirect
go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect
golang.org/x/mod v0.41.0 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/term v0.35.0 // indirect
golang.org/x/text v0.42.0 // indirect
Expand Down
16 changes: 8 additions & 8 deletions internal/report/report.go
Original file line number Diff line number Diff line change
Expand Up @@ -101,13 +101,15 @@ type Duplicate struct {
URL string `json:"-"`
Published bool `json:"-"`
CreatedAt time.Time `json:"-"`
// Confirmed records a separate, single-candidate check owned by the host.
Confirmed bool `json:"-"`
}

// visible says whether the reporter may see d.
func (d Duplicate) visible() bool {
// A reporter learns nothing from an exact duplicate of their own bug, a partial match on an
// unpublished advisory would reveal another report.
return d.Published || d.Exact && d.Confidence == "high"
return d.Published || d.Confirmed && d.Exact && d.Confidence == "high"
}

type Related struct {
Expand Down Expand Up @@ -335,8 +337,7 @@ type Checker struct {

// Verify drops claims pointing at things that do not exist and fills the facts from GitHub.
func (c Checker) Verify(ctx context.Context, r *Report) error {
// Dropped duplicates are not counted in Removed: the count would tell the reporter that an
// unpublished advisory came close.
// Duplicates stay available to RenderPrivate. Render applies the public disclosure boundary.
dups := r.Duplicates[:0]
seen := map[string]bool{}
for _, d := range r.Duplicates {
Expand All @@ -347,9 +348,6 @@ func (c Checker) Verify(ctx context.Context, r *Report) error {
}
seen[key] = true
d.URL, d.Published, d.CreatedAt = a.HTMLURL, a.State == "published", a.CreatedAt
if !d.visible() {
continue
}
dups = append(dups, d)
}
r.Duplicates = dups
Expand Down Expand Up @@ -504,8 +502,10 @@ func RenderPrivate(r Report) string {
advisory += " (oldest report)"
}
confidence := label(d.Confidence)
if d.Exact {
if d.Exact && (d.Published || d.Confirmed) {
confidence += " • Exact duplicate"
} else if d.Exact {
confidence += " • Proposed exact duplicate"
}
status := "Unpublished"
if d.Published {
Expand All @@ -514,7 +514,7 @@ func RenderPrivate(r Report) string {
// No title: the reporter reads this comment, and other advisories may still be under embargo.
fmt.Fprintf(&b, "| %s | %s | %s |\n", advisory, status, confidence)
}
b.WriteString("\n_Unpublished advisories are only listed as exact duplicates, without their title._\n")
b.WriteString("\n_The reporter-visible comment only lists unpublished advisories after a separate exact-duplicate confirmation, without their title. A match covers the shared vulnerability; review any additional findings before closing the report._\n")

b.WriteString("\n#### Related issues and pull requests\n\n")
if len(r.Related) == 0 {
Expand Down
14 changes: 8 additions & 6 deletions internal/report/report_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ func TestVerifyDropsWhatDoesNotExist(t *testing.T) {
}
}

func TestVerifyOnlyKeepsExactUnpublishedDuplicates(t *testing.T) {
func TestVerifyKeepsUnpublishedDuplicatesForThePrivateReport(t *testing.T) {
c := checker(t)
c.Advisories = map[string]github.Advisory{
"GHSA-pub": {State: "published", HTMLURL: "https://x/pub"},
Expand All @@ -116,7 +116,7 @@ func TestVerifyOnlyKeepsExactUnpublishedDuplicates(t *testing.T) {
}
r := Report{Duplicates: []Duplicate{
{GHSAID: "GHSA-pub", Confidence: "low"},
{GHSAID: "GHSA-exact", Confidence: "high", Exact: true},
{GHSAID: "GHSA-exact", Confidence: "high", Exact: true, Confirmed: true},
{GHSAID: "GHSA-near", Confidence: "high"},
{GHSAID: "GHSA-draft", Confidence: "medium"},
{GHSAID: "GHSA-closed", Confidence: "high", Exact: true},
Expand All @@ -129,11 +129,13 @@ func TestVerifyOnlyKeepsExactUnpublishedDuplicates(t *testing.T) {
for _, d := range r.Duplicates {
kept = append(kept, d.GHSAID)
}
require.Equal(t, []string{"GHSA-pub", "GHSA-exact", "GHSA-closed"}, kept)
require.Equal(t, []string{"GHSA-pub", "GHSA-exact", "GHSA-near", "GHSA-draft", "GHSA-closed", "GHSA-mixed"}, kept)
require.True(t, r.Duplicates[0].Published)
require.False(t, r.Duplicates[1].Published)
require.Equal(t, "https://x/exact", r.Duplicates[1].URL)
require.Zero(t, r.Removed)
require.Contains(t, RenderPrivate(r), "GHSA-near")
require.NotContains(t, Render(r), "GHSA-near")
}

func TestVerifyFailsOnGitHubErrors(t *testing.T) {
Expand Down Expand Up @@ -187,9 +189,9 @@ func TestRender(t *testing.T) {
"> Suggestions generated with an LLM. They help sort the report and do not replace a maintainer's review.",
"Heap overflow claimed, cc @\u200bsomeone\n",
"| Advisory | Status | Confidence |",
"| [GHSA-real](https://x/GHSA-real) | Unpublished | High • Exact duplicate |",
"| [GHSA-real](https://x/GHSA-real) | Unpublished | High • Proposed exact duplicate |",
"| [GHSA-near](https://x/GHSA-near) | Published | Low |",
"_Unpublished advisories are only listed as exact duplicates, without their title._",
"_The reporter-visible comment only lists unpublished advisories after a separate exact-duplicate confirmation, without their title. A match covers the shared vulnerability; review any additional findings before closing the report._",
"| Issue or PR | Title | Confidence |",
"| [PR #12](https://x/12) | Harden | Medium |",
"| [Issue #13](https://x/13) | Crash | Low |",
Expand All @@ -208,7 +210,7 @@ func TestRender(t *testing.T) {

func TestRenderShowsOnlyExactUnpublishedMatches(t *testing.T) {
r := Report{Duplicates: []Duplicate{
{GHSAID: "EXACT-ID", Confidence: "high", Exact: true},
{GHSAID: "EXACT-ID", Confidence: "high", Exact: true, Confirmed: true},
{GHSAID: "UNSURE-ID", Confidence: "medium", Exact: true},
{GHSAID: "PARTIAL-ID", Confidence: "high"},
{GHSAID: "PUBLIC-ID", Confidence: "low", Published: true},
Expand Down
Loading
Loading