Skip to content

Rewrite the documentation for readability #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alexandre-daubois merged 1 commit into from
Oct 28, 2026
Merged

Rewrite the documentation for readability #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 21 additions & 19 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,28 @@

[![CI](https://github.com/php/tamis/actions/workflows/ci.yml/badge.svg)](https://github.com/php/tamis/actions/workflows/ci.yml)

First-pass triage of private vulnerability reports on GitHub. When a new security advisory comes in, tamis has an LLM agent investigate it against the repository and posts its findings as a comment on the advisory.
tamis does a first-pass triage of private vulnerability reports on GitHub. When a new security advisory arrives, an LLM agent investigates it against the repository and tamis posts the findings as a comment on the advisory.

For each report, tamis:

- lists public duplicates in the comment, and embargoed advisories only when they are exact duplicates, by identifier,
- finds public issues and pull requests already fixing or hardening the same code,
- lists public duplicates, and names an embargoed advisory only when it is an exact duplicate,
- finds public issues and pull requests that already fix or harden the same code,
- checks that the files and functions the report cites exist,
- reads the security policy and flags what may exclude the report,
- suggests a CWE and a CVSS 4.0 base vector.

It informs the maintainers' decision and never makes it.
The maintainers keep the decision. tamis only prepares it.

The agent runs in an isolated container driven by [harness](https://github.com/alpha-omega-security/harness), the Alpha-Omega library behind [scrutineer](https://github.com/alpha-omega-security/scrutineer). [Security](docs/security.md) details what that sandbox allows.
The agent runs in an isolated container driven by [harness](https://github.com/alpha-omega-security/harness), the Alpha-Omega library behind [scrutineer](https://github.com/alpha-omega-security/scrutineer). [Security](docs/security.md) describes what the sandbox allows.

## Before installing

GitHub cannot post confidential advisory comments yet, so the reporter reads the comment. tamis also sends the report to the model provider you choose. Check that your disclosure policy allows both.

## Quick start

1. Create a GitHub App for your organisation with the "Repository security advisories" permission in read and write, and install it on the project repository only.
2. In a private repository, store the App's client ID and private key, and an Anthropic or OpenAI API key, or a [Codex access token](docs/configuration.md#codex-access-token).
1. Create a GitHub App for your organisation with the "Repository security advisories" permission in read and write. Install it on the project repository only.
2. In a private repository, store the App's client ID and private key, and an Anthropic or OpenAI API key or a [Codex access token](docs/configuration.md#codex-access-token).
3. Add this workflow to the private repository:

```yaml
Expand Down Expand Up @@ -57,7 +61,7 @@ jobs:
repository: your-org/your-project
fetch-depth: 0
persist-credentials: false
# Replace this placeholder with the full SHA of a reviewed Tamis commit.
# Replace this placeholder with the full SHA of a reviewed tamis commit.
- uses: php/tamis@REPLACE_WITH_REVIEWED_FULL_COMMIT_SHA
with:
repository: your-org/your-project
Expand All @@ -67,23 +71,21 @@ jobs:
api-key: ${{ secrets.ANTHROPIC_API_KEY }}
github-token: ${{ steps.app.outputs.token }}
bot-login: your-app-slug[bot]
# List every branch your project supports; these refs must exist in the checkout.
# List every branch your project supports. Each ref must exist in the checkout.
references: master origin/release-branch
```

Set `since` to the installation date so existing advisories are left alone. Keep the workflow repository private: the action uploads reports there as artifacts, retained for seven days. The [setup guide](docs/setup.md) covers source branches and provider spending limits.
Set `since` to the installation date so tamis leaves existing advisories alone. Keep the workflow repository private, since the action uploads the reports there as artifacts kept for seven days. The [setup guide](docs/setup.md) covers source branches and provider spending limits.

## Documentation

- [Setup](docs/setup.md): GitHub App, private repository, secrets and first run.
- [Configuration](docs/configuration.md): inputs, project guidance and engines.
- [How it works](docs/how-it-works.md): the triage pipeline and the comment it posts.
- [Security](docs/security.md): threat model, isolation and known limits.
- [Development](docs/development.md): building, testing and running locally.

## Before installing

The reporter can read the comment, since the GitHub API cannot post confidential comments yet, and the report is sent to the model provider you choose. Make sure your disclosure policy allows both.
| Page | Contents |
|---|---|
| [Setup](docs/setup.md) | GitHub App, private repository, secrets and first run |
| [Configuration](docs/configuration.md) | Inputs, budgets, project guidance and engines |
| [How it works](docs/how-it-works.md) | The triage pipeline and the comment it posts |
| [Security](docs/security.md) | Threat model, isolation and known limits |
| [Development](docs/development.md) | Building, testing and running locally |

## License

Expand Down
73 changes: 49 additions & 24 deletions docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,49 +4,74 @@

| Input | Default | Description |
|---|---|---|
| `api-key` | | API key of the model provider, or a Codex access token with `openai`. Required. |
| `github-token` | | Token able to read and comment the repository advisories. Required. |
| `bot-login` | owner of the token | Login the comments are posted as, e.g. `my-app[bot]`. Required with a GitHub App token. |
| `repository` | current repository | Repository owning the advisories. |
| `path` | `.` | Path of the repository checkout, relative to the workspace. |
| `since` | | Only advisories created after this date (`2026-10-22` or RFC 3339) are triaged. Required unless `advisory` is set. |
| `api-key` | | Required. API key of the model provider, or a Codex access token with `engine: openai`. |
| `github-token` | | Required. Token able to read and comment the repository advisories. |
| `references` | | Required, at most ten. Whitespace-separated local git references, one per supported branch. tamis analyses the first and locates the cited code on all of them. A missing reference fails the analysis. |
| `since` | | Triage only advisories created after this date, `2026-10-22` or RFC 3339. Required unless `advisory` is set. |
| `advisory` | | Triage this GHSA only, whatever its state and creation date. |
| `engine` | `claude` | `claude` (Claude Code and the Anthropic API) or `openai` (Codex and the OpenAI API). |
| `model` | `claude-opus-5-5` / `gpt-5.5` | Model used by the reader, duplicate search and exact confirmation. `gpt-6.1-sol` with a Codex access token. |
| `daybreak` | `false` | Turn on OpenAI's Daybreak cyber access program in Codex. `openai` engine only, and the API key's organisation must be approved for it. |
| `runner-image` | tamis agent image, pinned by digest | Image the agent runs in. It must contain the agent CLI, git, curl and `harness-proxy`. A custom image skips the attestation check. |
| `references` | | Whitespace-separated local git references, one for each supported branch. The first is analysed, the code the report cites is located on all of them. Required, at most ten. A missing reference fails the analysis. |
| `bot-login` | Owner of the token | Login tamis posts as, such as `my-app[bot]`. Required with a GitHub App token. |
| `repository` | Current repository | Repository owning the advisories. |
| `path` | `.` | Path of the repository checkout, relative to the workspace. |
| `engine` | `claude` | `claude` for Claude Code and the Anthropic API, `openai` for Codex and the OpenAI API. |
| `model` | `claude-opus-5-5` or `gpt-5.5` | Model of the reader, the duplicate search and the exact confirmation. `gpt-6.1-sol` with a Codex access token. |
| `daybreak` | `false` | Turns on OpenAI's Daybreak cyber access program in Codex. Needs `engine: openai` and an API key whose organisation OpenAI approved for it. |
| `runner-image` | tamis agent image, pinned by digest | Image the agent runs in. It must contain the agent CLI, git, curl and `harness-proxy`. The action skips the attestation check for a custom image. |

Outside GitHub Actions, the same settings are read from environment variables: `LLM_API_KEY`, `GH_TOKEN`, `BOT_LOGIN`, `REPOSITORY`, `REPO_PATH`, `SINCE`, `ADVISORY`, `ENGINE`, `MODEL`, `DAYBREAK`, `RUNNER_IMAGE` and `REFERENCES`. `OUTPUT_DIR` must name a private directory for the reports. The action creates that directory in the runner's temporary storage and uploads it to the private workflow repository with seven-day retention.
The action refuses to run from a public workflow repository or from a `uses:` that is not a full commit SHA. Update that SHA through a reviewed pull request.

The action requires a private workflow repository and a full commit SHA in `uses:`. Update that SHA through a reviewed pull request.
Outside GitHub Actions, tamis reads the same settings from environment variables: `LLM_API_KEY`, `GH_TOKEN`, `BOT_LOGIN`, `REPOSITORY`, `REPO_PATH`, `SINCE`, `ADVISORY`, `ENGINE`, `MODEL`, `DAYBREAK`, `RUNNER_IMAGE` and `REFERENCES`. `OUTPUT_DIR` must name a private directory for the reports. The action creates it in the runner's temporary storage and uploads it to the workflow repository with a seven-day retention.

## Source branches

Fetch the configured references with `fetch-depth: 0`. For php-src, include the default branch and every PHP branch covered by your disclosure policy, for example `master origin/PHP-8.4` when those are the branches you support. Keep the list current as releases enter or leave support. Tamis analyses each reference separately and includes its exact commit in both copies of the result; it does not infer the maintained branches from a reporter's prose. No source code is executed.
Fetch the references with `fetch-depth: 0`. For php-src, list the default branch and each PHP branch your disclosure policy covers, such as `master origin/PHP-8.4`, and update the list as releases enter or leave support.

tamis analyses each reference on its own and records its exact commit in both copies of the result. It does not guess the maintained branches from the reporter's prose, and it never executes the source code.

## Private reports

The private artifact contains a full `.md` report with every duplicate proposed by the global search and a `.comment.md` copy of the reporter-visible text, limited to published duplicates and separately confirmed exact unpublished duplicates. The full report also notes a duplicate search or confirmation that could not complete. Each copy names the source commit and input revision it covers. When an agent run fails, an `.agent-error.txt` file keeps the agent's error output and the error events of its JSON output, which may quote the report.
The private artifact holds, per advisory:

| File | Contents |
|---|---|
| `.md` | The full report, with every duplicate the global search proposed and any duplicate search or confirmation that could not complete |
| `.comment.md` | The text the reporter sees, limited to published duplicates and to the unpublished exact duplicates that passed the [one-candidate confirmation](how-it-works.md#3-duplicate-search) |
| `.agent-error.txt` | Written when an agent run fails. It keeps the agent's error output and the error events of its JSON output, which may quote the report. |

Each report names the source commit and the input revision it covers.

## Resource budgets

An execution permits at most 800 provider requests and 256 MiB of cumulative request bodies. Each advisory shares 200 requests and 64 MiB across its reader run and comparisons; each agent token additionally permits 150 requests, not counting the refused routes Codex calls on every run. A request body cannot exceed 1 MiB. API responses request at most 16,000 output tokens; the ChatGPT backend has its own output limits. The thirty-minute advisory deadline also cancels host-side git commands. At most ten unpublished exact candidates are separately confirmed. At most one report per known reporter is attempted per execution, and a report is only started while the execution still has a whole advisory budget and thirty minutes left of its 3 hours 50 minutes, otherwise it waits for the next execution. SDK retries are disabled, and an agent run that fails without a valid report is retried once after a minute.
| Scope | Limit |
|---|---|
| Execution | 800 provider requests and 256 MiB of request bodies in total, over at most 3 hours 50 minutes |
| Advisory | 200 requests and 64 MiB, shared by its reader run and its comparisons, and a 30-minute deadline that also cancels host-side git commands |
| Agent token | 150 requests, not counting the refused routes Codex calls on each run |
| Request | 1 MiB of body. API responses ask for at most 16,000 output tokens, and the ChatGPT backend applies its own output limits. |
| Exact confirmations | 10 unpublished candidates per advisory |
| Reports | One per known reporter and per execution |

tamis starts a report only while the execution still holds a whole advisory budget and thirty minutes of its time. Otherwise the report waits for the next execution. SDK retries are off, but tamis retries an agent run that fails without a valid report once, after a minute. When a budget runs out, the report stays retriable and tamis does not mark it complete.

These bounds cap one execution, not the spending across scheduled runs. Provider pricing, tokenisation and ChatGPT credit policies differ, so request and byte limits give no monetary guarantee. On the provider side:

- use a dedicated account or project with an enforced spending or credit ceiling,
- restrict the models, and disable hosted tools and network allowlists,
- configure billing alerts.

These bounds limit each execution, not spending across scheduled runs. Use a dedicated provider account/project with an enforced spending or credit ceiling, restrict models, disable hosted tools and network allowlists there, and configure billing alerts. Provider pricing, tokenisation and ChatGPT credit policies differ, so byte/request limits are not a monetary guarantee. Keep a workflow `timeout-minutes` of at least 240 and a single concurrency group. Budget exhaustion leaves a report retriable rather than marking it permanently complete.
In the workflow, keep a `timeout-minutes` of at least 240 and a single concurrency group.

## Project guidance

A `.github/tamis.md` file in the project is appended to the agent's instructions. Use it for what the model cannot guess: what is in scope, what the project's public API is, how the project scores CVSS, which configurations are realistic.
tamis appends a `.github/tamis.md` file from the project to the agent's instructions. Use it for what the model cannot guess, such as the scope, the public API, how the project scores CVSS and which configurations are realistic.

## Security policy

The agent checks the report against the files that describe what the project treats as a security issue. tamis looks for them among the tracked files of the repository root, `.github/` and `docs/`:
The agent checks the report against the files that define what the project treats as a security issue. tamis looks for them among the tracked files of the repository root, `.github/` and `docs/`:

- `SECURITY` with a `.md`, `.markdown`, `.rst`, `.txt` or `.adoc` extension, in any case,
- then `AGENTS.md` and `CLAUDE.md`.
1. `SECURITY` with a `.md`, `.markdown`, `.rst`, `.txt` or `.adoc` extension, in any case,
2. then `AGENTS.md` and `CLAUDE.md`.

The comment lists the files it was checked against. Without any of them, the section is left out.
The comment lists the files the agent checked. Without any of them, tamis leaves the section out.

## Engines

Expand All @@ -56,8 +81,8 @@ The comment lists the files it was checked against. Without any of them, the sec
| `openai` | Codex | OpenAI Chat Completions | `api.openai.com` |
| `openai` with a Codex access token | Codex | Codex, in a container without GitHub access | `chatgpt.com` |

The agent CLI versions are pinned in [`docker/agent/Dockerfile`](../docker/agent/Dockerfile).
[`docker/agent/Dockerfile`](../docker/agent/Dockerfile) pins the agent CLI versions.

## Codex access token

A ChatGPT workspace without OpenAI API access can create a [Codex access token](https://learn.chatgpt.com/docs/enterprise/access-tokens) in its admin console and pass it as `api-key` with `engine: openai`. It is recognised by its `at-` prefix. The ChatGPT backend does not serve the API models, so the default model becomes `gpt-6.1-sol`. Prefer a token of a service account, with the shortest expiration you can rotate.
A ChatGPT workspace without OpenAI API access can create a [Codex access token](https://learn.chatgpt.com/docs/enterprise/access-tokens) in its admin console and pass it as `api-key` with `engine: openai`. tamis recognises it by its `at-` prefix. The ChatGPT backend does not serve the API models, so the default model becomes `gpt-6.1-sol`. Prefer the token of a service account, with the shortest expiration you can rotate.
Loading
Loading