Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,20 @@
- The 1.0.2 documentation overhaul is reverted: `docs/` and `README.md` go back byte-for-byte to the commit #756 merged onto, along with the 14 generated locales and 14 translated READMEs #759 regenerated from those English sources. The release half of #756 stays — `package.json` and the Cargo workspace are untouched, since they have moved on to 1.0.4-beta.0 and the release tag the CLI builds its daemon download URL from is that npm version. Under `## 1.0.2` the heading and its release narrative stay, because 1.0.2 did ship; the `### Docs` entries underneath described the overhaul and go with it. Leaving the locales in place was the alternative considered and rejected: the nightly translate job is content-hash cached, so pages whose pre-overhaul English hashes it had already seen would have been skipped rather than repaired, stranding every non-English reader on a translation of text that no longer exists (#773)
- The landing page no longer opens with the harness paragraph claiming that "the same events, the same policies, and the same session history apply to every one" of the twelve. Removed from `docs/index.mdx` and all 14 locales (#773)

- The commands the reverted docs teach are checked against the shipped CLI again, and three of them did not run at all: `failproofai pack add core` is refused (`"core" is no longer a pack name` — ours is a pack like anyone else's now), `failproofai pack add --bundled` is an unknown flag, and `pack build` retired into `publish`. `packs.mdx`, `publish-a-pack.mdx`, `failure-behavior.mdx` and `reference/failproof-cli.mdx` are corrected; `policy`, `pack` and `p` remain working aliases for `policies` and are documented as such rather than silently relied on (#788)
- `failproofai config --connect <url> --token <key>` is no longer taught as the way to set a machine up. `--connect` short-circuits to enrolment and **returns** — no daemon, no hooks — so anyone who followed the quickstart got a machine that showed up in Cloud and then collected and enforced nothing. Fresh machines now get plain `failproofai config`, with the key arriving through `FAILPROOFAI_CLOUD_TOKEN` rather than argv, where `ps` shows it to every user on the box. The examples read it in with `read -s`, since a key typed into any command — `export` included — still lands in shell history, and CI is told to inject it from the secret store with shell tracing off. Corrected in `start/quickstart.mdx`, `start/setup.mdx`, `reference/failproof-cli.mdx`, `reference/harnesses.mdx`, `reference/custom-agents.mdx` and `reference/overview.mdx` (#788)
- `start/setup.mdx`'s "Local enforcement" card promised guardrails without sending session data to Cloud, then documented only the Cloud flow. The page now shows the local one: `failproofai config` with no key — **Not now — stay local** in a terminal, and on its own with no terminal — then a pack (#788)
- Nothing claims that installing hooks enables the policy catalog any more. `policies --install` with no names wires hooks and touches no policy, and setup says so itself when it finishes ("Nothing is enforcing yet"), so `README.md` and `start/quickstart.mdx` now carry the `failproofai policies add FailproofAI/policies` step that actually puts guardrails on a machine — 38 policies, of which the pack's manifest switches on 10. The always-on `block-failproofai-commands` guard is documented separately, as the one thing enforcing before that command runs (#788)
- The README's harness paragraph no longer promises "same events, same policies" across all twelve. `src/hooks/enforcement-capability.ts` is the authority and disagrees: blocking a tool call before it runs is verified on all twelve, but turn-end gates on eight — OpenCode, Pi, Hermes and Goose have none — so a Stop policy deployed on the strength of that sentence enforced nothing. It now states what is shared (one policy API, one session history), what is not, and links the per-harness matrix. `docs/index.mdx` lost the same sentence in #773; the README kept it (#788)
- `--machine-label` is documented in the order it actually works. On `config` it is *always* a rename — the branch that reads it fires whenever `--connect` and `--disconnect` are absent, so `failproofai config --token <key> --machine-label <name>` never reaches the wizard and answers "this machine is not connected … so it has no name to change". Setup comes first, the label after (#788)
- `sanitize-api-keys` is out of the README's "What it stops" table. It matches `PostToolUse`, which `ENFORCEMENT_CAPABILITY` classes observe-only, so the tool has already run and its output has already reached the model: it reports a secret rather than keeping one out of the context (#669). The nine policies left in that table are all `PreToolUse` and do gate the call, on every harness (#788)
- 102 published pages across seven locales opened with two consecutive `---` lines, so Mintlify closed the frontmatter block before any key was in it and rendered `title:`, `description:` and `icon:` as body text on a page with no title — visible on `docs.befailproof.ai` for as long as it stood. The stray delimiter is stripped, and `findTranslationError` gained the check that could not have caught it: every existing test asks `YAML.parse`, which reads a leading `---` as a document-start marker and returns a clean `{title, …}`, so the defect was invisible to the validator that exists to prevent exactly this. A second, Mintlify-shaped view of the block is now compared against it, and like Mintlify it takes only an unindented `---` as a delimiter, so an indented one is not mistaken for an empty block (#788)
- `docs/start/integrations.mdx` is in the navigation. The hub page for the five framework guides was on disk and linked from `reference/overview.mdx` and `reference/custom-agents.mdx`, but listed in no sidebar, in English and all 14 locales. Nav and disk now agree exactly, with nothing orphaned and nothing dangling (#788)
- The Policies section follows how a policy is actually made instead of listing every surface side by side. There are two ways to get one — write it (`policies/editor`: from an audit via **generate policy** and **write this policy**, or by hand) or take a pack from the [policy hub](https://befailproof.ai/policy-hub/) (`policies/packs`) — then one path: test (the new `policies/test`, which leads with the editor's backtest), deploy (which now starts with adding a machine), versions and rollback, and finally publishing a pack and failure behavior. `builtin`, `builtin-catalog`, `custom` and `fleet` are folded into `packs`, `editor`, `test` and `deploy` and deleted along with their 56 translations; their URLs redirect in all 15 languages, as does the website's `/built-in-policies`. `local-configuration` moves to the Reference tab and takes the catalog's parameter table with it. What the builtin pages said that still holds moves to `packs`: the Failproof pack's 38 policies and 10 defaults, checked against its published manifest, with a table marking which common guards a plain `policies add` switches on — `block-rm-rf`, `block-force-push` and `block-secrets-write` are not among them. The catalog's claim that the `sanitize-*` policies redact secrets "before the model sees them" goes with it: they match observation-only `PostToolUse`, so they report a secret found in tool output rather than withhold one (#669). `rollback.mdx` also stops calling CLI rollback a dashboard-only workflow — `fp fleet rollback <machine-id> <generation>` exists (#788)
- Every command in the Policies pages is checked against both CLIs' source, and 13 claims did not hold. The worst: the "deploy in observe mode" example enforced, because a bare `fp fleet deploy --add <id>` keeps the machine's effect or defaults to `enforce` — it now passes `<id>:observe`. `fp policies test` takes `--file`, not `--file-path` (fixed in the Cloud CLI reference too, alongside `fp fleet rollback`'s missing `GENERATION`); `compose` is session-only and refuses API keys; a pack policy switches on machine-wide whatever `--scope` says, and a bare name resolves to the pack that declares it, not to the builtin; re-adding a pack at a terminal re-opens the picker and replaces the selection; `failproofai publish` creates the repository before it validates the build, and at a terminal commits for you rather than refusing (#788)
- Evaluations get their own section, **Evaluate agents**, at the top of the Find failures tab: the two kinds of evaluator — hosted Python written in the dashboard, and your own worker for LLM judges — then writing one (drafted by the assistant and grounded in your payload keys, or by hand), testing it against real sessions without storing anything, deploying and versioning it (a successor disables its predecessor, rolling back re-enables it, and history is scored on request, up to 90 days at a time), and reading results, which absorbs the old Online evaluations page. The Evaluator SDK reference is rewritten for Evaluator v2's outbound-only worker in `failproofai_sdk.evaluator`: the push-model service it documented — `EVALUATOR_ENDPOINT`, `/evaluate`, `JobPending` — is retired, so every example on that page described an SDK that no longer ships. `evaluations:run`, the permission a worker's key needs, joins the permissions table (#788)
- Three review findings on the Policies rebuild above. `/built-in-policies` redirected in English only, so the fourteen localized copies of that page — deleted with the English one in #699 — kept answering 404 instead of landing on each locale's `policies/packs`; all fourteen redirects are added, matching how `builtin`, `builtin-catalog`, `custom` and `fleet` already redirect. `rollback.mdx` claimed a `fp policies disable` could itself be rolled back, which the same page denies ten lines earlier: `rollback` refuses a generation that names a disabled policy, and every generation from before the disable names that one — `fp policies enable` is the way back, and the page now says so. And `reference/evaluator-sdk.mdx` documented `FAILPROOFAI_EVALUATOR_ALLOW_INSECURE_HTTP` as a plain capability with no cost attached: `EvaluatorClient` sends `FAILPROOFAI_EVALUATOR_TOKEN` as an `Authorization: Bearer` header on every request and the transcripts it fetches are whole sessions, so plain HTTP to a non-loopback host hands both to anyone on the path, and the token keeps working until it is rotated. The flag is now marked for isolated development networks only (#788)

### Dependencies

- The `sharp` override moves 0.35.0 → 0.35.4, closing GHSA-rgj7-g3m4-5g8c (CVSS 8.9 — two Critical libheif RCEs inherited through libvips, reachable when processing untrusted input). Same surface-late shape as browserslist above and the chromadb entries: the advisory published 2026-09-08 21:25 UTC, between main's green Supply Chain run at 14:18 and its red one the next morning, so it turned every open branch red without any dependency change of its own — reproduced identically by running CI's scanner image against main's untouched lockfile. It stays an override rather than becoming a direct dependency because sharp is still only an optional dependency of next; what changed since #591 pinned it is the direction of the constraint. That pin was needed because next asked for `^0.34.5`, a range that excluded the then-fixed 0.35.0 — next 16.3.4 now asks for `^0.35.4`, so the pin had inverted into the thing holding the tree *below* what its dependent wants. Verified past a green lockfile the same way #591 did: sharp 0.35.4 loads against libvips 8.18.6 (`@img/sharp-libvips-*` 1.3.0 → 1.3.3) and round-trips a PNG encode; the scanner image reports "No issues found" (exit 0) with `osv-scanner.toml` gaining no new entry (#790)
Expand Down Expand Up @@ -604,7 +618,7 @@ disappearing quietly.

- Drop the Status link from the docs sidebar. It was a `navigation.global.anchors` entry, which Mintlify pins above the page tree on every page in every tab — permanent real estate for a link that answers a question almost no reader of a docs page is asking. Support stays, since that one is reached from anywhere in the docs by someone who is already stuck. (#718)

- Drop a duplicated entry and a repeated `### Fixes` heading from this release's own section. The canary-images entry was committed twice — once carrying an unfilled `(#PR)` placeholder and once as `(#705)` — and the section then opened a second `### Fixes` block a few entries after the first. Both were invisible while the changelog was only ever read on GitHub; the release announcement renders straight from these sections, so a duplicated headline and a heading appearing twice were about to show up in a public channel. (#721)
- Drop a duplicated entry and a repeated `### Fixes` heading from this release's own section. The canary-images entry was committed twice — once carrying an unfilled `(#788)` placeholder and once as `(#705)` — and the section then opened a second `### Fixes` block a few entries after the first. Both were invisible while the changelog was only ever read on GitHub; the release announcement renders straight from these sections, so a duplicated headline and a heading appearing twice were about to show up in a public channel. (#721)
Comment thread
nk-ag marked this conversation as resolved.

## 1.0.1-beta.0 — 2026-08-14

Expand Down
42 changes: 29 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,11 @@ tool calls before they execute. 39 built-in policies. Zero latency. Runs locally
## Supported harnesses

Twelve harnesses in two classes — ten coding CLIs, and two chat and assistant
gateways (Hermes, OpenClaw). Same events, same policies, same session history,
whichever one your agent runs in.
gateways (Hermes, OpenClaw). One policy API and one session history across all
of them. What a policy can *block* is per-harness: stopping a tool call before
it runs is verified on all twelve, turn-end gates on eight. The
[per-harness matrix](https://docs.befailproof.ai/reference/harnesses#enforcement-capability)
lists the events each one honours.

Agents that run in none of them report through the [Python SDK](https://docs.befailproof.ai/reference/custom-agents),
which gives you tracing, sessions and audits. Enforcement there needs a hook in
Expand Down Expand Up @@ -136,19 +139,29 @@ your own runtime — [talk to us](mailto:support@befailproof.ai) and we'll map i

```sh
npm install -g failproofai
failproofai policies --install # or just run `failproofai` and accept the first-run prompt
failproofai
failproofai config # wire up your agents and the daemon
failproofai policies add FailproofAI/policies # choose what to enforce
failproofai # dashboard on localhost:8020
```

39 built-in policies activate immediately. Dashboard at `localhost:8020`. Disable the first-run prompt with `FAILPROOFAI_NO_FIRST_RUN=1`.
Setup wires the hooks and picks **no** policies — that second command is what
puts guardrails on the machine, and any pack is typed the same way
(`failproofai policies add <owner>/<repo>`; `policies show <owner>/<repo>` reads
one first). Run `failproofai config` with no terminal — CI, a container, an
agent driving it — and it applies rather than asking. On a machine that has
never been set up, any other command runs the same wizard first; disable that
with `FAILPROOFAI_NO_FIRST_RUN=1`.

Until a pack arrives, the only thing enforcing is `block-failproofai-commands`,
which is always on and cannot be switched off or paused: an agent that can pause
enforcement can switch off every other policy.

---

## What it stops

| Policy | What it blocks |
|---|---|
| `sanitize-api-keys` | API keys leaking into the agent's context |
| `block-env-files` | Reads of `.env` and other secret files |
| `warn-repeated-tool-calls` | The agent looping on the same call |
| `block-sudo` | Privilege escalation |
Expand All @@ -157,10 +170,13 @@ failproofai
| `block-rm-rf` | Recursive file deletion |
| `block-force-push` / `block-push-master` | `git push --force`, direct pushes to `main` |

The first five apply to any agent that can call a tool. The last three are the
developer favourites — coding CLIs are the harness class we cover deepest.
Every one of these gates the call *before* it runs, so they hold on all twelve
harnesses. The first four apply to any agent that can call a tool; the last
three are the developer favourites — coding CLIs are the harness class we cover
deepest. The `sanitize-*` family is separate: it runs after a tool returns, so
it reports a secret in tool output rather than keeping it out of the context.

→ [All 39 built-in policies](https://docs.befailproof.ai/policies/builtin)
→ [All 39 built-in policies](https://docs.befailproof.ai/policies/packs)

---

Expand Down Expand Up @@ -191,7 +207,7 @@ Three decisions available to every policy:
| `deny(message)` | Block it — message goes back to the agent |
| `instruct(message)` | Let it through, but add context to the agent's next prompt |

→ [Custom policies guide](https://docs.befailproof.ai/policies/custom)
→ [Write a policy](https://docs.befailproof.ai/policies/editor)

---

Expand Down Expand Up @@ -242,9 +258,9 @@ own cluster is available on the Enterprise plan.

| Enforce | |
|---|---|
| [Built-in policies](https://docs.befailproof.ai/policies/builtin) | All 39 policies with parameters |
| [Custom policies](https://docs.befailproof.ai/policies/custom) | Write your own |
| [Configuration](https://docs.befailproof.ai/policies/local-configuration) | Config scopes and merge rules |
| [Policy packs](https://docs.befailproof.ai/policies/packs) | The Failproof AI policies, and packs from the policy hub |
| [Write a policy](https://docs.befailproof.ai/policies/editor) | From an audit, or in code |
| [Configuration](https://docs.befailproof.ai/policies/local-configuration) | Config scopes, merge rules and policy parameters |

| Instrument your own agent | |
|---|---|
Expand Down
Loading