diff --git a/CHANGELOG.md b/CHANGELOG.md index 9282faee5..e27568bf9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,20 @@ - The 1.0.2 documentation overhaul is reverted: `docs/` and `README.md` go back byte-for-byte to the commit #756 merged onto, along with the 14 generated locales and 14 translated READMEs #759 regenerated from those English sources. The release half of #756 stays — `package.json` and the Cargo workspace are untouched, since they have moved on to 1.0.4-beta.0 and the release tag the CLI builds its daemon download URL from is that npm version. Under `## 1.0.2` the heading and its release narrative stay, because 1.0.2 did ship; the `### Docs` entries underneath described the overhaul and go with it. Leaving the locales in place was the alternative considered and rejected: the nightly translate job is content-hash cached, so pages whose pre-overhaul English hashes it had already seen would have been skipped rather than repaired, stranding every non-English reader on a translation of text that no longer exists (#773) - The landing page no longer opens with the harness paragraph claiming that "the same events, the same policies, and the same session history apply to every one" of the twelve. Removed from `docs/index.mdx` and all 14 locales (#773) +- The commands the reverted docs teach are checked against the shipped CLI again, and three of them did not run at all: `failproofai pack add core` is refused (`"core" is no longer a pack name` — ours is a pack like anyone else's now), `failproofai pack add --bundled` is an unknown flag, and `pack build` retired into `publish`. `packs.mdx`, `publish-a-pack.mdx`, `failure-behavior.mdx` and `reference/failproof-cli.mdx` are corrected; `policy`, `pack` and `p` remain working aliases for `policies` and are documented as such rather than silently relied on (#788) +- `failproofai config --connect --token ` is no longer taught as the way to set a machine up. `--connect` short-circuits to enrolment and **returns** — no daemon, no hooks — so anyone who followed the quickstart got a machine that showed up in Cloud and then collected and enforced nothing. Fresh machines now get plain `failproofai config`, with the key arriving through `FAILPROOFAI_CLOUD_TOKEN` rather than argv, where `ps` shows it to every user on the box. The examples read it in with `read -s`, since a key typed into any command — `export` included — still lands in shell history, and CI is told to inject it from the secret store with shell tracing off. Corrected in `start/quickstart.mdx`, `start/setup.mdx`, `reference/failproof-cli.mdx`, `reference/harnesses.mdx`, `reference/custom-agents.mdx` and `reference/overview.mdx` (#788) +- `start/setup.mdx`'s "Local enforcement" card promised guardrails without sending session data to Cloud, then documented only the Cloud flow. The page now shows the local one: `failproofai config` with no key — **Not now — stay local** in a terminal, and on its own with no terminal — then a pack (#788) +- Nothing claims that installing hooks enables the policy catalog any more. `policies --install` with no names wires hooks and touches no policy, and setup says so itself when it finishes ("Nothing is enforcing yet"), so `README.md` and `start/quickstart.mdx` now carry the `failproofai policies add FailproofAI/policies` step that actually puts guardrails on a machine — 38 policies, of which the pack's manifest switches on 10. The always-on `block-failproofai-commands` guard is documented separately, as the one thing enforcing before that command runs (#788) +- The README's harness paragraph no longer promises "same events, same policies" across all twelve. `src/hooks/enforcement-capability.ts` is the authority and disagrees: blocking a tool call before it runs is verified on all twelve, but turn-end gates on eight — OpenCode, Pi, Hermes and Goose have none — so a Stop policy deployed on the strength of that sentence enforced nothing. It now states what is shared (one policy API, one session history), what is not, and links the per-harness matrix. `docs/index.mdx` lost the same sentence in #773; the README kept it (#788) +- `--machine-label` is documented in the order it actually works. On `config` it is *always* a rename — the branch that reads it fires whenever `--connect` and `--disconnect` are absent, so `failproofai config --token --machine-label ` never reaches the wizard and answers "this machine is not connected … so it has no name to change". Setup comes first, the label after (#788) +- `sanitize-api-keys` is out of the README's "What it stops" table. It matches `PostToolUse`, which `ENFORCEMENT_CAPABILITY` classes observe-only, so the tool has already run and its output has already reached the model: it reports a secret rather than keeping one out of the context (#669). The nine policies left in that table are all `PreToolUse` and do gate the call, on every harness (#788) +- 102 published pages across seven locales opened with two consecutive `---` lines, so Mintlify closed the frontmatter block before any key was in it and rendered `title:`, `description:` and `icon:` as body text on a page with no title — visible on `docs.befailproof.ai` for as long as it stood. The stray delimiter is stripped, and `findTranslationError` gained the check that could not have caught it: every existing test asks `YAML.parse`, which reads a leading `---` as a document-start marker and returns a clean `{title, …}`, so the defect was invisible to the validator that exists to prevent exactly this. A second, Mintlify-shaped view of the block is now compared against it, and like Mintlify it takes only an unindented `---` as a delimiter, so an indented one is not mistaken for an empty block (#788) +- `docs/start/integrations.mdx` is in the navigation. The hub page for the five framework guides was on disk and linked from `reference/overview.mdx` and `reference/custom-agents.mdx`, but listed in no sidebar, in English and all 14 locales. Nav and disk now agree exactly, with nothing orphaned and nothing dangling (#788) +- The Policies section follows how a policy is actually made instead of listing every surface side by side. There are two ways to get one — write it (`policies/editor`: from an audit via **generate policy** and **write this policy**, or by hand) or take a pack from the [policy hub](https://befailproof.ai/policy-hub/) (`policies/packs`) — then one path: test (the new `policies/test`, which leads with the editor's backtest), deploy (which now starts with adding a machine), versions and rollback, and finally publishing a pack and failure behavior. `builtin`, `builtin-catalog`, `custom` and `fleet` are folded into `packs`, `editor`, `test` and `deploy` and deleted along with their 56 translations; their URLs redirect in all 15 languages, as does the website's `/built-in-policies`. `local-configuration` moves to the Reference tab and takes the catalog's parameter table with it. What the builtin pages said that still holds moves to `packs`: the Failproof pack's 38 policies and 10 defaults, checked against its published manifest, with a table marking which common guards a plain `policies add` switches on — `block-rm-rf`, `block-force-push` and `block-secrets-write` are not among them. The catalog's claim that the `sanitize-*` policies redact secrets "before the model sees them" goes with it: they match observation-only `PostToolUse`, so they report a secret found in tool output rather than withhold one (#669). `rollback.mdx` also stops calling CLI rollback a dashboard-only workflow — `fp fleet rollback ` exists (#788) +- Every command in the Policies pages is checked against both CLIs' source, and 13 claims did not hold. The worst: the "deploy in observe mode" example enforced, because a bare `fp fleet deploy --add ` keeps the machine's effect or defaults to `enforce` — it now passes `:observe`. `fp policies test` takes `--file`, not `--file-path` (fixed in the Cloud CLI reference too, alongside `fp fleet rollback`'s missing `GENERATION`); `compose` is session-only and refuses API keys; a pack policy switches on machine-wide whatever `--scope` says, and a bare name resolves to the pack that declares it, not to the builtin; re-adding a pack at a terminal re-opens the picker and replaces the selection; `failproofai publish` creates the repository before it validates the build, and at a terminal commits for you rather than refusing (#788) +- Evaluations get their own section, **Evaluate agents**, at the top of the Find failures tab: the two kinds of evaluator — hosted Python written in the dashboard, and your own worker for LLM judges — then writing one (drafted by the assistant and grounded in your payload keys, or by hand), testing it against real sessions without storing anything, deploying and versioning it (a successor disables its predecessor, rolling back re-enables it, and history is scored on request, up to 90 days at a time), and reading results, which absorbs the old Online evaluations page. The Evaluator SDK reference is rewritten for Evaluator v2's outbound-only worker in `failproofai_sdk.evaluator`: the push-model service it documented — `EVALUATOR_ENDPOINT`, `/evaluate`, `JobPending` — is retired, so every example on that page described an SDK that no longer ships. `evaluations:run`, the permission a worker's key needs, joins the permissions table (#788) +- Three review findings on the Policies rebuild above. `/built-in-policies` redirected in English only, so the fourteen localized copies of that page — deleted with the English one in #699 — kept answering 404 instead of landing on each locale's `policies/packs`; all fourteen redirects are added, matching how `builtin`, `builtin-catalog`, `custom` and `fleet` already redirect. `rollback.mdx` claimed a `fp policies disable` could itself be rolled back, which the same page denies ten lines earlier: `rollback` refuses a generation that names a disabled policy, and every generation from before the disable names that one — `fp policies enable` is the way back, and the page now says so. And `reference/evaluator-sdk.mdx` documented `FAILPROOFAI_EVALUATOR_ALLOW_INSECURE_HTTP` as a plain capability with no cost attached: `EvaluatorClient` sends `FAILPROOFAI_EVALUATOR_TOKEN` as an `Authorization: Bearer` header on every request and the transcripts it fetches are whole sessions, so plain HTTP to a non-loopback host hands both to anyone on the path, and the token keeps working until it is rotated. The flag is now marked for isolated development networks only (#788) + ### Dependencies - The `sharp` override moves 0.35.0 → 0.35.4, closing GHSA-rgj7-g3m4-5g8c (CVSS 8.9 — two Critical libheif RCEs inherited through libvips, reachable when processing untrusted input). Same surface-late shape as browserslist above and the chromadb entries: the advisory published 2026-09-08 21:25 UTC, between main's green Supply Chain run at 14:18 and its red one the next morning, so it turned every open branch red without any dependency change of its own — reproduced identically by running CI's scanner image against main's untouched lockfile. It stays an override rather than becoming a direct dependency because sharp is still only an optional dependency of next; what changed since #591 pinned it is the direction of the constraint. That pin was needed because next asked for `^0.34.5`, a range that excluded the then-fixed 0.35.0 — next 16.3.4 now asks for `^0.35.4`, so the pin had inverted into the thing holding the tree *below* what its dependent wants. Verified past a green lockfile the same way #591 did: sharp 0.35.4 loads against libvips 8.18.6 (`@img/sharp-libvips-*` 1.3.0 → 1.3.3) and round-trips a PNG encode; the scanner image reports "No issues found" (exit 0) with `osv-scanner.toml` gaining no new entry (#790) @@ -604,7 +618,7 @@ disappearing quietly. - Drop the Status link from the docs sidebar. It was a `navigation.global.anchors` entry, which Mintlify pins above the page tree on every page in every tab — permanent real estate for a link that answers a question almost no reader of a docs page is asking. Support stays, since that one is reached from anywhere in the docs by someone who is already stuck. (#718) -- Drop a duplicated entry and a repeated `### Fixes` heading from this release's own section. The canary-images entry was committed twice — once carrying an unfilled `(#PR)` placeholder and once as `(#705)` — and the section then opened a second `### Fixes` block a few entries after the first. Both were invisible while the changelog was only ever read on GitHub; the release announcement renders straight from these sections, so a duplicated headline and a heading appearing twice were about to show up in a public channel. (#721) +- Drop a duplicated entry and a repeated `### Fixes` heading from this release's own section. The canary-images entry was committed twice — once carrying an unfilled `(#788)` placeholder and once as `(#705)` — and the section then opened a second `### Fixes` block a few entries after the first. Both were invisible while the changelog was only ever read on GitHub; the release announcement renders straight from these sections, so a duplicated headline and a heading appearing twice were about to show up in a public channel. (#721) ## 1.0.1-beta.0 — 2026-08-14 diff --git a/README.md b/README.md index 636d2666b..63496d24d 100644 --- a/README.md +++ b/README.md @@ -31,8 +31,11 @@ tool calls before they execute. 39 built-in policies. Zero latency. Runs locally ## Supported harnesses Twelve harnesses in two classes — ten coding CLIs, and two chat and assistant -gateways (Hermes, OpenClaw). Same events, same policies, same session history, -whichever one your agent runs in. +gateways (Hermes, OpenClaw). One policy API and one session history across all +of them. What a policy can *block* is per-harness: stopping a tool call before +it runs is verified on all twelve, turn-end gates on eight. The +[per-harness matrix](https://docs.befailproof.ai/reference/harnesses#enforcement-capability) +lists the events each one honours. Agents that run in none of them report through the [Python SDK](https://docs.befailproof.ai/reference/custom-agents), which gives you tracing, sessions and audits. Enforcement there needs a hook in @@ -136,11 +139,22 @@ your own runtime — [talk to us](mailto:support@befailproof.ai) and we'll map i ```sh npm install -g failproofai -failproofai policies --install # or just run `failproofai` and accept the first-run prompt -failproofai +failproofai config # wire up your agents and the daemon +failproofai policies add FailproofAI/policies # choose what to enforce +failproofai # dashboard on localhost:8020 ``` -39 built-in policies activate immediately. Dashboard at `localhost:8020`. Disable the first-run prompt with `FAILPROOFAI_NO_FIRST_RUN=1`. +Setup wires the hooks and picks **no** policies — that second command is what +puts guardrails on the machine, and any pack is typed the same way +(`failproofai policies add /`; `policies show /` reads +one first). Run `failproofai config` with no terminal — CI, a container, an +agent driving it — and it applies rather than asking. On a machine that has +never been set up, any other command runs the same wizard first; disable that +with `FAILPROOFAI_NO_FIRST_RUN=1`. + +Until a pack arrives, the only thing enforcing is `block-failproofai-commands`, +which is always on and cannot be switched off or paused: an agent that can pause +enforcement can switch off every other policy. --- @@ -148,7 +162,6 @@ failproofai | Policy | What it blocks | |---|---| -| `sanitize-api-keys` | API keys leaking into the agent's context | | `block-env-files` | Reads of `.env` and other secret files | | `warn-repeated-tool-calls` | The agent looping on the same call | | `block-sudo` | Privilege escalation | @@ -157,10 +170,13 @@ failproofai | `block-rm-rf` | Recursive file deletion | | `block-force-push` / `block-push-master` | `git push --force`, direct pushes to `main` | -The first five apply to any agent that can call a tool. The last three are the -developer favourites — coding CLIs are the harness class we cover deepest. +Every one of these gates the call *before* it runs, so they hold on all twelve +harnesses. The first four apply to any agent that can call a tool; the last +three are the developer favourites — coding CLIs are the harness class we cover +deepest. The `sanitize-*` family is separate: it runs after a tool returns, so +it reports a secret in tool output rather than keeping it out of the context. -→ [All 39 built-in policies](https://docs.befailproof.ai/policies/builtin) +→ [All 39 built-in policies](https://docs.befailproof.ai/policies/packs) --- @@ -191,7 +207,7 @@ Three decisions available to every policy: | `deny(message)` | Block it — message goes back to the agent | | `instruct(message)` | Let it through, but add context to the agent's next prompt | -→ [Custom policies guide](https://docs.befailproof.ai/policies/custom) +→ [Write a policy](https://docs.befailproof.ai/policies/editor) --- @@ -242,9 +258,9 @@ own cluster is available on the Enterprise plan. | Enforce | | |---|---| -| [Built-in policies](https://docs.befailproof.ai/policies/builtin) | All 39 policies with parameters | -| [Custom policies](https://docs.befailproof.ai/policies/custom) | Write your own | -| [Configuration](https://docs.befailproof.ai/policies/local-configuration) | Config scopes and merge rules | +| [Policy packs](https://docs.befailproof.ai/policies/packs) | The Failproof AI policies, and packs from the policy hub | +| [Write a policy](https://docs.befailproof.ai/policies/editor) | From an audit, or in code | +| [Configuration](https://docs.befailproof.ai/policies/local-configuration) | Config scopes, merge rules and policy parameters | | Instrument your own agent | | |---|---| diff --git a/__tests__/scripts/translate-docs/validate-translation.test.ts b/__tests__/scripts/translate-docs/validate-translation.test.ts index 97d4eb43e..0b60c6818 100644 --- a/__tests__/scripts/translate-docs/validate-translation.test.ts +++ b/__tests__/scripts/translate-docs/validate-translation.test.ts @@ -23,6 +23,52 @@ describe("findTranslationError", () => { expect(error).toMatch(/does not parse/); }); + it("flags a stray second opening delimiter", async () => { + // The exact shape that shipped to 102 published pages across seven locales. + // Every check that asks YAML calls this clean — YAML reads the leading + // `---` as a document-start marker and returns `{title, description}` — but + // Mintlify closes the block on line 2, so the keys render as body text and + // the page has no title at all. Only the second, Mintlify-shaped view sees + // it, which is the whole reason that view exists. + const rendered = `---\n---\ntitle: "Fähigkeit"\ndescription: "Eine Seite"\n---\n\n# Körper\n`; + const error = await findTranslationError(rendered, SOURCE); + expect(error).not.toBeNull(); + expect(error).toContain("empty"); + // Says what to do, not just what is wrong — the message is read by a model + // that has to produce a corrected page on the retry. + expect(error).toContain("exactly one opening `---`"); + }); + + it("flags a stray second opening delimiter on a frontmatter-less source", async () => { + // Nothing about the defect depends on the source having frontmatter: the + // page is equally broken either way, so the check runs for every shape. + const rendered = `---\n---\ntitle: "Titel"\n---\n\nEtwas Prosa.\n`; + const error = await findTranslationError(rendered, README_SOURCE); + expect(error).not.toBeNull(); + expect(error).toContain("empty"); + }); + + it("still flags a stray delimiter that carries trailing whitespace", async () => { + // Mintlify closes the block on `--- ` just as it does on `---`, so the + // trailing space changes nothing about the defect. + const rendered = `---\n--- \ntitle: "Fähigkeit"\ndescription: "Eine Seite"\n---\n\n# Körper\n`; + const error = await findTranslationError(rendered, SOURCE); + expect(error).not.toBeNull(); + expect(error).toContain("empty"); + }); + + it("does not read an indented `---` as a delimiter", async () => { + // A delimiter starts its line. Comparing trimmed lines let an indented + // `---` open or close the block, so both of these were rejected for an + // empty frontmatter block they do not have — a retry burned on a defect + // that is not there. + const indentedAfterOpener = `---\n ---\n Titel\n---\n\n# Körper\n`; + expect(await findTranslationError(indentedAfterOpener, README_SOURCE)).toBeNull(); + + const indentedFirstLine = ` ---\n---\n\n# Körper\n`; + expect(await findTranslationError(indentedFirstLine, README_SOURCE)).toBeNull(); + }); + it("flags a frontmatter block the model dropped entirely", async () => { // A missing block is still valid YAML (mintlify tolerates it), so only the // key-parity check against the source catches it. diff --git a/docs/admin/keys-and-permissions.mdx b/docs/admin/keys-and-permissions.mdx index 82ca36dc2..d2a364ee0 100644 --- a/docs/admin/keys-and-permissions.mdx +++ b/docs/admin/keys-and-permissions.mdx @@ -54,7 +54,7 @@ Key secrets are shown when created or regenerated. Store them in a secret manage | Events | `events:add`, `events:read` | | Keys | `keys:create`, `keys:read`, `keys:disable`, `keys:regenerate`; `keys:update` is human-session only | | Users | `users:create`, `users:read`, `users:update`, `users:delete` | -| Evaluations | `evaluations:read`, `evaluations:trigger` | +| Evaluations | `evaluations:read`, `evaluations:trigger`, `evaluations:run` | | Dashboards | `dashboards:read`, `dashboards:write`, `dashboards:delete` | | Queries | `queries:read`, `queries:write`, `queries:delete`, `queries:run` | | Assistant | `agent:use` | diff --git a/docs/ar/admin/keys-and-permissions.mdx b/docs/ar/admin/keys-and-permissions.mdx index 0f8682ef5..fc3ef8cb9 100644 --- a/docs/ar/admin/keys-and-permissions.mdx +++ b/docs/ar/admin/keys-and-permissions.mdx @@ -1,5 +1,4 @@ --- ---- title: "المفاتيح والأذونات" description: "إنشاء مفاتيح API ذات نطاق محدد للآلات والأتمتة والمشغلين." icon: "key-round" diff --git a/docs/ar/admin/overview.mdx b/docs/ar/admin/overview.mdx index 8827906ad..23483ed1d 100644 --- a/docs/ar/admin/overview.mdx +++ b/docs/ar/admin/overview.mdx @@ -1,5 +1,4 @@ --- ---- title: "الإدارة" description: "تشغيل الوصول والاستخدام والمنظمات والأمان دون دمجها في سير عمل الموثوقية." icon: "settings-2" diff --git a/docs/ar/admin/usage.mdx b/docs/ar/admin/usage.mdx index 532df2e43..066940331 100644 --- a/docs/ar/admin/usage.mdx +++ b/docs/ar/admin/usage.mdx @@ -1,5 +1,4 @@ --- ---- title: "الاستخدام" description: "فحص استهلاك المنظمة والنافذة الفعالة للفواتير." icon: "chart-no-axes-combined" diff --git a/docs/ar/audits/agent-contracts.mdx b/docs/ar/audits/agent-contracts.mdx index 442b6c2bb..abd6d8a32 100644 --- a/docs/ar/audits/agent-contracts.mdx +++ b/docs/ar/audits/agent-contracts.mdx @@ -1,5 +1,4 @@ --- ---- title: "سياق الوكيل" description: "أخبر المراجعات بما يجب أن يفعله كل وكيل، وما يجب أن ينتجه، وما يجب أن لا يفعله أبداً." icon: "bot" diff --git a/docs/ar/audits/alerts.mdx b/docs/ar/audits/alerts.mdx index a7a8b5af1..bb9d34362 100644 --- a/docs/ar/audits/alerts.mdx +++ b/docs/ar/audits/alerts.mdx @@ -1,5 +1,4 @@ --- ---- title: "التنبيهات" description: "كتشف تكرار الحوادث وإعادة توجيهها إلى المستجيبين المناسبين." icon: "bell-ring" diff --git a/docs/ar/audits/cadence.mdx b/docs/ar/audits/cadence.mdx index 35661228f..32cf486b6 100644 --- a/docs/ar/audits/cadence.mdx +++ b/docs/ar/audits/cadence.mdx @@ -1,5 +1,4 @@ --- ---- title: "تكرار المراجعة" description: "حدد موعد تشغيل المراجعات المتكررة وكمية البيانات التي تراجعها." icon: "calendar-clock" diff --git a/docs/ar/audits/findings-and-issues.mdx b/docs/ar/audits/findings-and-issues.mdx index 8655cd01e..ff12081d9 100644 --- a/docs/ar/audits/findings-and-issues.mdx +++ b/docs/ar/audits/findings-and-issues.mdx @@ -1,5 +1,4 @@ --- ---- title: "النتائج والمشاكل" description: "تحويل أدلة التدقيق إلى عمل إعادة معالجة مملوك وقابل للتتبع." icon: "clipboard-check" diff --git a/docs/ar/audits/recipes.mdx b/docs/ar/audits/recipes.mdx index d93b47879..402c90080 100644 --- a/docs/ar/audits/recipes.mdx +++ b/docs/ar/audits/recipes.mdx @@ -1,5 +1,4 @@ --- ---- title: "وصفات التدقيق" description: "أهداف البداية للتحقيقات الشائعة لفشل الوكيل." icon: "book-open-check" diff --git a/docs/ar/audits/run.mdx b/docs/ar/audits/run.mdx index 7b857ebaa..6e221a7d1 100644 --- a/docs/ar/audits/run.mdx +++ b/docs/ar/audits/run.mdx @@ -1,5 +1,4 @@ --- ---- title: "تشغيل ومراجعة التدقيق" description: "قم بتشغيل التدقيق والتحقق من نطاقه وفحص النتائج الناتجة." icon: "play" diff --git a/docs/ar/index.mdx b/docs/ar/index.mdx index 92b389a32..dea610256 100644 --- a/docs/ar/index.mdx +++ b/docs/ar/index.mdx @@ -1,5 +1,4 @@ --- ---- title: "اجعل وكيلك failproof" description: "قابلية المراقبة والإنفاذ لكل محرك يعمل به وكلاؤك — برامج سطر الأوامر للترميز، بوابات الدردشة، المساعدات المستضافة ذاتياً، والوكلاء المزودين بآليات المراقبة الخاصة بك." icon: "shield-check" diff --git a/docs/ar/policies/builtin-catalog.mdx b/docs/ar/policies/builtin-catalog.mdx deleted file mode 100644 index ea5cc26f1..000000000 --- a/docs/ar/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "كتالوج السياسات المدمجة" -description: "راجع كل سياسة مدمجة في Failproof AI، ومحفزها، والحالة الموصى بها، والمعاملات القابلة للتكوين." -icon: "list-checks" ---- - -الحزمة المثبتة هي مصدر الحقيقة لتوفر السياسات. قم بتشغيل `failproofai policies` بعد كل تحديث لأن إدخالات الكتالوج والسلوك قد يتغيران مع إصدار الحزمة. - -## خط الأساس الموصى به - -يقوم الإعداد الموجه حالياً بتفعيل معقمات الأسرار وحماية البيئة والحماية الذاتية وحراس الأوامر الكارثية وسلامة الفرع المحمي: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` **مفعّل دائماً**. يُدرج أعلاه للكمال فقط، لكنه يسجل عند كل تقييم بغض النظر عما إذا كان يظهر في مجموعة السياسات المفعلة لديك، ولا يمكن تعطيله أو إيقافه مؤقتاً — حيث أن الحماية التي يمكن للوكيل إيقافها ليست حماية حقيقية. - -خط الأساس الموصى به أضيق عن قصد من **الكل**. قد تقاطع سياسات البنية التحتية وسير العمل العمل الصحيح ويجب تفعيلها للمستودعات والآلات التي تحتاجها. - -## الأسرار والبيئة - -| السياسة | المحفز | النتيجة | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | إخفاء JWTs من مخرجات الأداة قبل أن يراها النموذج. | -| `sanitize-api-keys` | `PostToolUse` | إخفاء مفاتيح OpenAI و Anthropic و GitHub و AWS و Stripe و Google الشائعة. | -| `sanitize-connection-strings` | `PostToolUse` | إخفاء سلاسل اتصال قواعد البيانات التي تحتوي على بيانات اعتماد. | -| `sanitize-private-key-content` | `PostToolUse` | إخفاء أجسام مفاتيح PEM الخاصة. | -| `sanitize-bearer-tokens` | `PostToolUse` | إخفاء رموز التفويض bearer. | -| `protect-env-vars` | `PreToolUse` على أدوات shell | حظر الأوامر التي تُفرغ متغيرات البيئة. | -| `block-env-files` | `PreToolUse` | حظر قراءة وكتابة ملفات `.env`. | -| `block-read-outside-cwd` | `PreToolUse` على أدوات read أو glob أو grep أو shell | الاحتفاظ بالقراءات داخل دليل عمل الجلسة. | -| `block-secrets-write` | `PreToolUse` على أدوات الكتابة | حظر الكتابة إلى أسماء ملفات المفاتيح السرية وبيانات الاعتماد الشائعة. | - -## الأوامر الخطيرة والبنية التحتية - -| السياسة | المحفز | النتيجة | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`، `PermissionRequest` | حظر `sudo` ما لم تطابق نمط السماح. | -| `block-curl-pipe-sh` | `PreToolUse` | حظر البرامج النصية المحملة الموجهة مباشرة إلى shell. | -| `block-rm-rf` | `PreToolUse` | حظر أنماط الحذف العودي الكارثية. | -| `block-failproofai-commands` | `PreToolUse`، `PermissionRequest` | **مفعّل دائماً، لا يمكن تعطيله.** حظر كل استدعاء CLI لـ Failproof AI وإيقاف ذاتي وحذف مدير الحزم. | -| `block-kubectl` | `PreToolUse` | التحكم في أوامر Kubernetes. | -| `block-terraform` | `PreToolUse` | التحكم في أوامر Terraform و OpenTofu. | -| `block-aws-cli` | `PreToolUse` | التحكم في أوامر AWS CLI. | -| `block-gcloud` | `PreToolUse` | التحكم في أوامر Google Cloud CLI. | -| `block-az-cli` | `PreToolUse` | التحكم في أوامر Azure CLI. | -| `block-helm` | `PreToolUse` | التحكم في أوامر Helm. | -| `block-gh-pipeline` | `PreToolUse` | التحكم في عمليات GitHub CLI التي تُحدث التغييرات المتعلقة بسير العمل والتشغيل والدمج والإصدار والذاكرة المؤقتة والأسرار. | - -## سلامة Git وقاعدة البيانات - -| السياسة | المحفز | النتيجة | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | حظر الدفع المباشر إلى الفروع المحمية المكونة. | -| `block-force-push` | `PreToolUse` | حظر الدفع القسري؛ يبقى `--force-with-lease` مسموحاً بالتنفيذ الحالي. | -| `block-work-on-main` | `PreToolUse` | حظر الالتزامات والدمج على الفروع المحمية. | -| `warn-git-amend` | `PreToolUse` | تحذير قبل إعادة كتابة التزام باستخدام `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | تحذير قبل حذف أو مسح المخزنات مؤقتاً بشكل دائم. | -| `warn-all-files-staged` | `PreToolUse` | تحذير على `git add -A` أو `git add .` أو `git add --all` الواسعة. | -| `warn-destructive-sql` | `PreToolUse` | تحذير على `DROP` و `TRUNCATE` و `DELETE` بدون `WHERE` عبر عملاء قواعد البيانات المعروفة. | -| `warn-schema-alteration` | `PreToolUse` | تحذير على عمليات `ALTER TABLE` المعروفة الخاصة بالأعمدة وإعادة التسمية. | - -## الحزم وسلوك النظام ودورات الوكيل - -| السياسة | المحفز | النتيجة | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | تحذير قبل النشر إلى سجلات الحزم. | -| `warn-global-package-install` | `PreToolUse` | تحذير قبل التثبيت العام للحزم. | -| `prefer-package-manager` | `PreToolUse` | توجيه الوكيل لاستخدام مدير حزم مسموح. | -| `warn-large-file-write` | `PreToolUse` على أدوات الكتابة | تحذير فوق حد حجم الملف المكون. | -| `warn-background-process` | `PreToolUse` | تحذير على أنماط العمليات الخلفية المنفصلة أو طويلة الأجل. | -| `warn-repeated-tool-calls` | `PreToolUse` | تحذير بعد ثلاث أو أكثر من استدعاءات الأداة المتطابقة. | - -## سير عمل نهاية المهمة - -تتطلب هذه السياسات جهازاً يصدر حدث `Stop` متوافقاً. - -| السياسة | النتيجة | -| --- | --- | -| `require-commit-before-stop` | رفض الإكمال بينما يبقى عمل متتبع غير ملتزم. | -| `require-push-before-stop` | رفض الإكمال بينما تبقى التزامات محلية فقط. | -| `require-pr-before-stop` | تطلب طلب دمج للفرع الحالي. | -| `require-no-conflicts-before-stop` | تطلب دمج نظيف ضد الفرع الأساسي المكون. | -| `require-ci-green-before-stop` | تطلب اكتمال فحوصات CI الحالية بنجاح. | - -## مرجع المعاملات - -كون المعاملات تحت كائن `policyParams` للنطاق المحدد. تتحقق الأنواع من قبل كل سياسة. - -| السياسة | المعامل | النوع والافتراضي | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`، `[]`؛ تحتوي الإدخالات على `regex` و `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`، `[]` | -| `block-sudo` | `allowPatterns` | `string[]`، `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`، `[]` | -| حاجبات البنية التحتية | `allowPatterns` | `string[]`، `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`، `[]` | -| `block-push-master` | `protectedBranches` | `string[]`، `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`، `["main", "master"]` | -| `prefer-package-manager` | `allowed`، `blocked` | `string[]`، `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`، `1024` | -| `require-push-before-stop` | `remote`، `baseBranch` | `string`، `"origin"`؛ `string`، `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`، `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`، `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - نمط السماح يوسع ما قد يفعله الوكيل. اختبر التقسيم الدقيق ومتغيرات الأوامر على جهاز الاستهداف قبل نشره عبر الأسطول. - \ No newline at end of file diff --git a/docs/ar/policies/builtin.mdx b/docs/ar/policies/builtin.mdx deleted file mode 100644 index ef479347f..000000000 --- a/docs/ar/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "السياسات المدمجة" -description: "تفعيل حواجز مصانة للحالات الشائعة لفشل الوكلاء." -icon: "library" ---- - -تغطي السياسات المدمجة معالجة الأسرار والملفات البيئية وأوامر shell المدمرة والفروع المحمية وأدوات السحابة والبنية التحتية ونشر الحزم والاستدعاءات المتكررة والتحقق من سير العمل عند نهاية المهمة. - -## تفعيل التحقق من السياسة المدمجة - - - - 1. ثبّت السياسة على جهاز متصل باستخدام واجهة سطر الأوامر المحلية. - 2. قم بتشغيل إجراء اختبار آمن في الوكيل المدعوم. - 3. انتقل إلى **Observe → policy** وقم بالتصفية حسب اسم السياسة أو بيئة الجهاز أو القرار. - 4. افتح الجلسة المرتبطة لتأكيد مدخلات الأداة المطابقة والسبب المُرجع. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - أزلها باستخدام `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -اعرض قائمة بالسياسات المتاحة في نسختك المثبتة: - -```bash -failproofai policies -``` - -فعّل سياسة واحدة لمشروع: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -فعّل عدة سياسات لأنظمة اختبار محددة: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -بعض السياسات تقبل معاملات أو يتم وضع علامة عليها كنسخة تجريبية. راجع الوصف ونطاق المطابقة والسلوك الافتراضي قبل النشر. قد تحمي السياسة التي تحافظ على سير عمل واحد من العمليات الصحيحة في سير عمل آخر. - - - راجع جميع السياسات الـ 40 الحالية، مشغلاتها والخط الأساسي الموصى به والمعاملات. - - - - فضّل نطاق المشروع للتوقعات الخاصة بالمستودع ونطاق المستخدم لمتطلبات السلامة الموسعة على الجهاز. - \ No newline at end of file diff --git a/docs/ar/policies/custom.mdx b/docs/ar/policies/custom.mdx deleted file mode 100644 index 5cd4348c8..000000000 --- a/docs/ar/policies/custom.mdx +++ /dev/null @@ -1,73 +0,0 @@ ---- ---- -title: "سياسات مخصصة" -description: "اكتب سياسة لحالة فشل فريدة من نوعها في سير عمل وكيلك." -icon: "shield-plus" ---- - -أنشئ ملفًا ينتهي بـ `policies.js` أو `policies.mjs` أو `policies.ts` تحت `.failproofai/policies/`. يتم تحميل ملفات الاتفاقية تلقائيًا في نطاق المشروع والمستخدم. - -## اختبر السياسة قبل النشر على Cloud - - - - 1. ثبّت السياسة المخصصة على جهاز اختبار واحد وفعّل كلاً من الإجراء المطابق وعدم التطابق الشرعي. - 2. انتقل إلى **Observe → policy** وقارن بين القرارين. - 3. افتح كل جلسة مرتبطة وتحقق من أن حمل الحدث يحتوي على أدلة كافية للقاعدة. - 4. عندما يكون السلوك صحيحًا، انقل المصدر المراجع إلى **Admin → policy editor** ونشّر نسخة. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - يتم تحميل ملفات الاتفاقية تحت `.failproofai/policies/` بدون `--custom`. احتفظ بأمر التثبيت الصريح في CI عندما يجب أن يفشل التحقق على وحدة معطوبة. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -هذا يطابق `production/config.yml` و `/srv/production/config.yml` و `/srv/production` و `C:\\production\\config.yml` لكل من `Write` و `Edit`. لا يطابق الأسماء مثل `production-backup` لأن `production` يجب أن تكون قطعة مسار كاملة. - -تحقق من صحة وثبّت ملف صريح: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -سياق السياسة يتضمن نوع الحدث، والحمل المعياري، واسم الأداة والمدخلات، وبيانات تعريف الجلسة، والمعاملات، والمصدر CLI عند توفره. - -## اختبر مسارات الفشل - -قم بتشغيل التحقق من الصحة بعد تغيير ملف الإدخال أو أي وحدة محلية يستوردها: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -مسار CLI الصارم يفشل للملفات المفقودة وأخطاء بناء الجملة والاستيرادات غير المحللة والاستثناءات على مستوى أعلى وانتهاءات مهلة تحميل الوحدة. في وقت الفرض، يتم تسجيل ملف مخصص معطوب والتخطي عنه بحيث يمكن للسياسات المدمجة أن تستمر. تعامل مع أي تحذير تحميل كفقدان للفرض المتوقع وأرسل تنبيهًا عليه في سجلات الإنتاج. - -استخدم أسماء فريدة عالميًا عبر السياسات الصريحة والاتفاقية والمدارة من Cloud. اجعل وظائف السياسة حتمية، وقيّد الاتصالات الخارجية بمهل زمنية قصيرة، وأرجع `allow` أو `instruct` أو `deny` مقصودة على كل مسار. - - - السياسة المخصصة هي كود الفرض. اختبر الحقول المفقودة وأسماء الأدوات البديلة والمدخلات غير الصحيحة—ليس فقط المطابقة المتوقعة. - \ No newline at end of file diff --git a/docs/ar/policies/deploy.mdx b/docs/ar/policies/deploy.mdx index b2aa395d4..55b293fb9 100644 --- a/docs/ar/policies/deploy.mdx +++ b/docs/ar/policies/deploy.mdx @@ -1,5 +1,4 @@ --- ---- title: "نشر السياسات" description: "طرح نسخة سياسة تمت مراجعتها على الأجهزة المخطط لها." icon: "cloud-upload" diff --git a/docs/ar/policies/editor.mdx b/docs/ar/policies/editor.mdx index ed6ed1238..9668f9c04 100644 --- a/docs/ar/policies/editor.mdx +++ b/docs/ar/policies/editor.mdx @@ -1,5 +1,4 @@ --- ---- title: "محرر السياسات" description: "إنشاء ومراجعة السياسات ذات الإصدارات من نمط فشل مؤكد." icon: "file-pen-line" diff --git a/docs/ar/policies/failure-behavior.mdx b/docs/ar/policies/failure-behavior.mdx index ba16d9ece..e666f018a 100644 --- a/docs/ar/policies/failure-behavior.mdx +++ b/docs/ar/policies/failure-behavior.mdx @@ -1,5 +1,4 @@ --- ---- title: "سلوك الفشل" description: "فهم ما يحدث عند عدم توفر تقييم السياسة أو مُحقِّق محلي." icon: "shield-alert" diff --git a/docs/ar/policies/fleet.mdx b/docs/ar/policies/fleet.mdx deleted file mode 100644 index e2e5c65d5..000000000 --- a/docs/ar/policies/fleet.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- ---- -title: "نشر السياسات على الأجهزة" -description: "تحديد الأجهزة المسجلة والمحدثة والتي تطبق إصدارات السياسات المقصودة." -icon: "network" ---- - -يجيب تغطية الأسطول عما إذا كانت سياسة موجودة حيث توجد المخاطرة. تتبع الأجهزة من خلال معرّف مستقر وتسمية قابلة للقراءة من قبل الإنسان، ثم قارن حالة النشر المخصصة والمبلغ عنها. - -## التحقق من التغطية - - - - 1. انتقل إلى **المسؤول → الإنفاذ** واستعرض إجمالي الإنفاذ والمراقبة. - 2. ابحث عن جهاز حسب المعرّف أو التسمية، أو قم بالتصفية للأجهزة التي تفتقد سياسة. - 3. وسّع الصف لمقارنة السياسات المخصصة والنشر المبلغ عنه آخر تسجيل دخول والسجل. - 4. أعد التحديث بعد فترة الاستطلاع للجهاز عندما يبقى النشر المطبق قيد الانتظار. - - ![أسطول الإنفاذ يعرض تغطية السياسة وحالة نشر الجهاز وتعيينات المراقبة والإنفاذ.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - استخدم `fp events --agent-id --since 24h` للتأكد من وصول نشاط وكيل الجهاز إلى السحابة. - - - -استخدم طرق عرض التغطية للعثور على: - -- الأجهزة التي لم تسحب النشر الأخير أبداً -- الأجهزة المسجلة التي توقفت عن الإبلاغ عن النشاط -- سياسة مخصصة للبيئة أو المجموعة الخاطئة -- انجراف الإصدار بعد تحديث متقطع - -إعادة تسمية جهاز دون إعادة الاتصال: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -التحقق من الحالة المحلية: - -```bash -failproofai config --status -``` - - - استخدم تسميات تحدد عبء العمل والبيئة. غالباً ما تكون أسماء المضيفين وحدها غير كافية بعد التوسيع التلقائي أو استبدال الجهاز. - \ No newline at end of file diff --git a/docs/ar/policies/local-configuration.mdx b/docs/ar/policies/local-configuration.mdx index 690de26c5..f8144b958 100644 --- a/docs/ar/policies/local-configuration.mdx +++ b/docs/ar/policies/local-configuration.mdx @@ -1,5 +1,4 @@ --- ---- title: "التكوين المحلي" description: "التحكم في نطاق السياسة والمعاملات والملفات المخصصة وإعدادات Failproof AI على مستوى الجهاز." icon: "file-cog" diff --git a/docs/ar/policies/overview.mdx b/docs/ar/policies/overview.mdx index 5e2a3ec67..9be3a2c13 100644 --- a/docs/ar/policies/overview.mdx +++ b/docs/ar/policies/overview.mdx @@ -1,5 +1,4 @@ --- ---- title: "السياسات" description: "راقب أو وجّه أو احجب إجراءات الوكيل قبل أن تتكرر حالة فشل معروفة." icon: "shield-check" diff --git a/docs/ar/policies/publish-a-pack.mdx b/docs/ar/policies/publish-a-pack.mdx index 602fdb5a2..048637e78 100644 --- a/docs/ar/policies/publish-a-pack.mdx +++ b/docs/ar/policies/publish-a-pack.mdx @@ -1,5 +1,4 @@ --- ---- title: "نشر حزمة" description: "شحن سياساتك الخاصة كإصدار GitHub يمكن لأي شخص تثبيته." icon: "upload" diff --git a/docs/ar/policies/rollback.mdx b/docs/ar/policies/rollback.mdx index 621137578..acecf62b1 100644 --- a/docs/ar/policies/rollback.mdx +++ b/docs/ar/policies/rollback.mdx @@ -1,5 +1,4 @@ --- ---- title: "التراجع عن النشر" description: "استعادة نشر سياسة معروف عند حدوث اضطراب في عمل الوكيل الصحيح." icon: "rotate-ccw" diff --git a/docs/ar/reference/cloud-cli.mdx b/docs/ar/reference/cloud-cli.mdx index 41ebf2238..42b2069c5 100644 --- a/docs/ar/reference/cloud-cli.mdx +++ b/docs/ar/reference/cloud-cli.mdx @@ -1,5 +1,4 @@ --- ---- title: "Failproof Cloud CLI" description: "مرجع شامل للاستعلام عن وإدارة Failproof AI Cloud باستخدام fp." icon: "cloud-cog" diff --git a/docs/ar/reference/evaluator-sdk.mdx b/docs/ar/reference/evaluator-sdk.mdx index 6f3eb4f27..2084d1107 100644 --- a/docs/ar/reference/evaluator-sdk.mdx +++ b/docs/ar/reference/evaluator-sdk.mdx @@ -1,5 +1,4 @@ --- ---- title: "Evaluator SDK" description: "أنشئ خدمة تقيّم جلسات Failproof AI بشكل متزامن أو غير متزامن." icon: "gauge" diff --git a/docs/ar/reference/failproof-cli.mdx b/docs/ar/reference/failproof-cli.mdx index 0344448d9..1339a14c2 100644 --- a/docs/ar/reference/failproof-cli.mdx +++ b/docs/ar/reference/failproof-cli.mdx @@ -1,5 +1,4 @@ --- ---- title: "واجهة أوامر Failproof AI" description: "ثبّت الخطافات، أدِر السياسات المحلية، اتصل بالسحابة، وشغّل مراقب الخادم المحلي." icon: "terminal" diff --git a/docs/ar/reference/policy-sdk.mdx b/docs/ar/reference/policy-sdk.mdx index eb0e43a4b..1a433f1a8 100644 --- a/docs/ar/reference/policy-sdk.mdx +++ b/docs/ar/reference/policy-sdk.mdx @@ -1,5 +1,4 @@ --- ---- title: "السياسات المخصصة" description: "قم بتأليف واختبار ونشر سياسات JavaScript أو TypeScript لحالات الفشل الخاصة بوكلائك." icon: "shield-plus" diff --git a/docs/ar/sessions/assistant.mdx b/docs/ar/sessions/assistant.mdx index 6fd63c8fb..bc00d786f 100644 --- a/docs/ar/sessions/assistant.mdx +++ b/docs/ar/sessions/assistant.mdx @@ -1,5 +1,4 @@ --- ---- title: "مساعد Failproof" description: "حلل وشغل Failproof AI باستخدام اللغة الطبيعية، من الأسئلة والاستفسارات إلى لوحات المعلومات والتدقيق." icon: "message-square-text" diff --git a/docs/ar/sessions/dashboards.mdx b/docs/ar/sessions/dashboards.mdx index 318610784..a1fd12604 100644 --- a/docs/ar/sessions/dashboards.mdx +++ b/docs/ar/sessions/dashboards.mdx @@ -1,5 +1,4 @@ --- ---- title: "لوحات المعلومات" description: "تتبع إشارات الموثوقية التي تهم الوكيل أو سير العمل." icon: "layout-dashboard" diff --git a/docs/ar/sessions/errors.mdx b/docs/ar/sessions/errors.mdx index 8277065b5..55f9df2cb 100644 --- a/docs/ar/sessions/errors.mdx +++ b/docs/ar/sessions/errors.mdx @@ -1,5 +1,4 @@ --- ---- title: "الأخطاء" description: "جمّع الأخطاء المتكررة وافتح الجلسات المرتبطة بها." icon: "circle-alert" diff --git a/docs/ar/sessions/evaluations.mdx b/docs/ar/sessions/evaluations.mdx index ad4b8f0b1..8d5168a1a 100644 --- a/docs/ar/sessions/evaluations.mdx +++ b/docs/ar/sessions/evaluations.mdx @@ -1,5 +1,4 @@ --- ---- title: "التقييمات المباشرة" description: "قيّم الجلسات المباشرة والمكتملة من حيث الجودة والامتثال والتكلفة والكمون." icon: "gauge" diff --git a/docs/ar/sessions/live-events.mdx b/docs/ar/sessions/live-events.mdx index 7c678d3ac..a5476ae67 100644 --- a/docs/ar/sessions/live-events.mdx +++ b/docs/ar/sessions/live-events.mdx @@ -1,5 +1,4 @@ --- ---- title: "الأحداث المباشرة" description: "شاهد نشاط الوكيل يصل أثناء تشغيل جلسة عمل." icon: "radio" diff --git a/docs/ar/sessions/models.mdx b/docs/ar/sessions/models.mdx index b47c3ccec..862d72e63 100644 --- a/docs/ar/sessions/models.mdx +++ b/docs/ar/sessions/models.mdx @@ -1,5 +1,4 @@ --- ---- title: "النماذج" description: "قارن زمن الاستجابة والرموز واستخدام السياق وتوزيع حركة المرور بين النماذج." icon: "cpu" diff --git a/docs/ar/sessions/overview.mdx b/docs/ar/sessions/overview.mdx index 5726912ee..9933bb518 100644 --- a/docs/ar/sessions/overview.mdx +++ b/docs/ar/sessions/overview.mdx @@ -1,5 +1,4 @@ --- ---- title: "الجلسات" description: "ابدأ بسجل كامل لتشغيل وكيل واحد." icon: "workflow" diff --git a/docs/ar/sessions/policy-decisions.mdx b/docs/ar/sessions/policy-decisions.mdx index 6fe89515e..c972d4a2d 100644 --- a/docs/ar/sessions/policy-decisions.mdx +++ b/docs/ar/sessions/policy-decisions.mdx @@ -1,5 +1,4 @@ --- ---- title: "قرارات السياسة" description: "اطّلع على السياسات المقيّمة والمحظورة والموجّهة والمسموح بها." icon: "shield-check" diff --git a/docs/ar/sessions/read-a-trace.mdx b/docs/ar/sessions/read-a-trace.mdx index 212dddf39..a3600aa9c 100644 --- a/docs/ar/sessions/read-a-trace.mdx +++ b/docs/ar/sessions/read-a-trace.mdx @@ -1,5 +1,4 @@ --- ---- title: "قراءة التتبع" description: "ابحث عن الحدث الذي غيّر مسار جلسة الوكيل." icon: "route" diff --git a/docs/ar/start/first-policy.mdx b/docs/ar/start/first-policy.mdx index ccc76b888..6647e7f1f 100644 --- a/docs/ar/start/first-policy.mdx +++ b/docs/ar/start/first-policy.mdx @@ -1,5 +1,4 @@ --- ---- title: "منع فشلك الأول باستخدام سياسة" description: "قم بتأليف نسخة من السياسة، وانشرها في وضع المراقبة، ثم طبقها." icon: "shield-check" diff --git a/docs/ar/start/integrations.mdx b/docs/ar/start/integrations.mdx index 047a0ff6f..a56459630 100644 --- a/docs/ar/start/integrations.mdx +++ b/docs/ar/start/integrations.mdx @@ -1,5 +1,4 @@ --- ---- title: "جهز وكيلك" sidebarTitle: "الأطر العمل" description: "اربط أي إطار عمل وكيل مدعوم إلى Failproof AI برمز واحد." diff --git a/docs/ar/start/integrations/crewai.mdx b/docs/ar/start/integrations/crewai.mdx index 16861f7bf..e580e5d34 100644 --- a/docs/ar/start/integrations/crewai.mdx +++ b/docs/ar/start/integrations/crewai.mdx @@ -1,5 +1,4 @@ --- ---- title: "CrewAI" sidebarTitle: "CrewAI" description: "تجهيز الفرق والتدفقات والوكلاء حسب الدور والأدوات والذاكرة وردود الفعل البشرية." diff --git a/docs/ar/start/integrations/langchain.mdx b/docs/ar/start/integrations/langchain.mdx index c6f9eee46..a35b43efa 100644 --- a/docs/ar/start/integrations/langchain.mdx +++ b/docs/ar/start/integrations/langchain.mdx @@ -1,5 +1,4 @@ --- ---- title: "LangChain و LangGraph" sidebarTitle: "LangChain و LangGraph" description: "قم بتتبع الرسوم البيانية والعقد والأدوات والمسترجعات واستدعاءات النموذج برمز واحد." diff --git a/docs/ar/start/quickstart.mdx b/docs/ar/start/quickstart.mdx index b59e7768f..1daebeea6 100644 --- a/docs/ar/start/quickstart.mdx +++ b/docs/ar/start/quickstart.mdx @@ -1,5 +1,4 @@ --- ---- title: "البدء السريع" description: "التقط جلسة وكيل، ابحث عن عطل، وابدأ في منعه." icon: "zap" diff --git a/docs/ar/start/quickstarts/crewai.mdx b/docs/ar/start/quickstarts/crewai.mdx index d948814b2..9ee22d1db 100644 --- a/docs/ar/start/quickstarts/crewai.mdx +++ b/docs/ar/start/quickstarts/crewai.mdx @@ -1,5 +1,4 @@ --- ---- title: "CrewAI" description: "قم بالتثبيت والمراقبة وشاهد أول تتبع لك." icon: "/images/frameworks/crewai.svg" diff --git a/docs/ar/start/quickstarts/custom-agents.mdx b/docs/ar/start/quickstarts/custom-agents.mdx index e5b6d2347..a25d12073 100644 --- a/docs/ar/start/quickstarts/custom-agents.mdx +++ b/docs/ar/start/quickstarts/custom-agents.mdx @@ -1,5 +1,4 @@ --- ---- title: "وكلاء مخصصون" description: "غلّف وكيلك في ثلاث كتل `with` وسيبدأ التسجيل." icon: "code" diff --git a/docs/ar/start/quickstarts/langchain.mdx b/docs/ar/start/quickstarts/langchain.mdx index b16ccee4b..53c3f78f1 100644 --- a/docs/ar/start/quickstarts/langchain.mdx +++ b/docs/ar/start/quickstarts/langchain.mdx @@ -1,5 +1,4 @@ --- ---- title: "LangChain و LangGraph" description: "التثبيت والأداة والحصول على أول تتبع لديك." icon: "/images/frameworks/langchain.svg" diff --git a/docs/de/policies/builtin-catalog.mdx b/docs/de/policies/builtin-catalog.mdx deleted file mode 100644 index 96dfd40e3..000000000 --- a/docs/de/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "Integrierter Richtlinienkatalog" -description: "Übersicht aller integrierten Failproof AI-Richtlinien mit Auslöser, empfohlenem Status und konfigurierbaren Parametern." -icon: "list-checks" ---- - -Das installierte Paket ist die maßgebliche Quelle für die Verfügbarkeit von Richtlinien. Führen Sie nach jedem Upgrade `failproofai policies` aus, da sich Katalogeinträge und Verhalten mit der Paketversion ändern können. - -## Empfohlene Grundkonfiguration - -Die empfohlene Auswahl des geführten Setups aktiviert derzeit Secret-Sanitizer, Umgebungsschutz, Selbstschutz, Schutz vor kritischen Befehlen und Sicherheit für geschützte Branches: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` ist **immer aktiv**. Die Richtlinie ist oben der Vollständigkeit halber aufgeführt, wird jedoch bei jeder Auswertung registriert, unabhängig davon, ob sie in Ihrer aktivierten Gruppe erscheint. Sie kann weder deaktiviert noch pausiert werden – ein Schutz, den der Agent selbst abschalten kann, ist kein Schutz. - -Die empfohlene Auswahl ist bewusst schmaler als **Alles**. Infrastruktur- und Workflow-Richtlinien können valide Arbeit unterbrechen und sollten nur für die Repositories und Maschinen aktiviert werden, die sie benötigen. - -## Secrets und Umgebung - -| Richtlinie | Auslöser | Ergebnis | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | JWTs aus der Tool-Ausgabe entfernen, bevor das Modell sie sieht. | -| `sanitize-api-keys` | `PostToolUse` | Gängige OpenAI-, Anthropic-, GitHub-, AWS-, Stripe- und Google-Schlüssel entfernen. | -| `sanitize-connection-strings` | `PostToolUse` | Datenbankverbindungsstrings mit Zugangsdaten entfernen. | -| `sanitize-private-key-content` | `PostToolUse` | PEM-Private-Key-Inhalte entfernen. | -| `sanitize-bearer-tokens` | `PostToolUse` | Authorization-Bearer-Tokens entfernen. | -| `protect-env-vars` | `PreToolUse` bei Shell-Tools | Befehle blockieren, die Umgebungsvariablen ausgeben. | -| `block-env-files` | `PreToolUse` | Lese- und Schreibzugriffe auf `.env`-Dateien blockieren. | -| `block-read-outside-cwd` | `PreToolUse` bei Lese-, Glob-, Grep- oder Shell-Tools | Lesezugriffe auf das Arbeitsverzeichnis der Sitzung beschränken. | -| `block-secrets-write` | `PreToolUse` bei Schreib-Tools | Schreibzugriffe auf gängige Secret-Key- und Credential-Dateinamen blockieren. | - -## Gefährliche Befehle und Infrastruktur - -| Richtlinie | Auslöser | Ergebnis | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | `sudo` blockieren, sofern kein Allow-Muster übereinstimmt. | -| `block-curl-pipe-sh` | `PreToolUse` | Heruntergeladene Skripte blockieren, die direkt an eine Shell weitergeleitet werden. | -| `block-rm-rf` | `PreToolUse` | Kritische rekursive Löschmuster blockieren. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Immer aktiv, kann nicht deaktiviert werden.** Jeden Failproof AI CLI-Aufruf, Self-Pause und Paketmanager-Deinstallation blockieren. | -| `block-kubectl` | `PreToolUse` | Kubernetes-Befehle kontrollieren. | -| `block-terraform` | `PreToolUse` | Terraform- und OpenTofu-Befehle kontrollieren. | -| `block-aws-cli` | `PreToolUse` | AWS CLI-Befehle kontrollieren. | -| `block-gcloud` | `PreToolUse` | Google Cloud CLI-Befehle kontrollieren. | -| `block-az-cli` | `PreToolUse` | Azure CLI-Befehle kontrollieren. | -| `block-helm` | `PreToolUse` | Helm-Befehle kontrollieren. | -| `block-gh-pipeline` | `PreToolUse` | Mutierende GitHub CLI-Operationen für Workflows, Runs, Merges, Releases, Caches und Secrets kontrollieren. | - -## Git- und Datenbanksicherheit - -| Richtlinie | Auslöser | Ergebnis | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Direkte Pushes auf konfigurierte geschützte Branches blockieren. | -| `block-force-push` | `PreToolUse` | Force-Pushes blockieren; `--force-with-lease` bleibt durch die aktuelle Implementierung erlaubt. | -| `block-work-on-main` | `PreToolUse` | Commits und Merges auf geschützten Branches blockieren. | -| `warn-git-amend` | `PreToolUse` | Warnung vor dem Umschreiben eines Commits mit `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | Warnung vor dem dauerhaften Löschen oder Leeren von Stashes. | -| `warn-all-files-staged` | `PreToolUse` | Warnung bei breit gefasstem `git add -A`, `git add .` oder `git add --all`. | -| `warn-destructive-sql` | `PreToolUse` | Warnung bei `DROP`, `TRUNCATE` und `DELETE` ohne `WHERE` über erkannte Datenbankclients. | -| `warn-schema-alteration` | `PreToolUse` | Warnung bei erkannten `ALTER TABLE`-Spalten- und Umbenennungsoperationen. | - -## Pakete, Systemverhalten und Agent-Schleifen - -| Richtlinie | Auslöser | Ergebnis | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Warnung vor der Veröffentlichung in Paketregistries. | -| `warn-global-package-install` | `PreToolUse` | Warnung vor der globalen Paketinstallation. | -| `prefer-package-manager` | `PreToolUse` | Den Agenten anweisen, einen erlaubten Paketmanager zu verwenden. | -| `warn-large-file-write` | `PreToolUse` bei Schreib-Tools | Warnung bei Überschreiten des konfigurierten Dateigrößen-Schwellenwerts. | -| `warn-background-process` | `PreToolUse` | Warnung bei getrennten oder langlebigen Hintergrundprozess-Mustern. | -| `warn-repeated-tool-calls` | `PreToolUse` | Warnung nach drei oder mehr identischen Tool-Aufrufen. | - -## Aufgabenabschluss-Workflow - -Diese Richtlinien erfordern ein Harness, das ein kompatibles `Stop`-Ereignis ausgibt. - -| Richtlinie | Ergebnis | -| --- | --- | -| `require-commit-before-stop` | Abschluss verweigern, solange verfolgte Änderungen uncommittet sind. | -| `require-push-before-stop` | Abschluss verweigern, solange Commits nur lokal vorhanden sind. | -| `require-pr-before-stop` | Einen Pull Request für den aktuellen Branch verlangen. | -| `require-no-conflicts-before-stop` | Einen konfliktfreien Merge mit dem konfigurierten Basis-Branch verlangen. | -| `require-ci-green-before-stop` | Erfolgreichen Abschluss der CI-Prüfungen für den aktuellen HEAD verlangen. | - -## Parameterreferenz - -Parameter werden unter dem `policyParams`-Objekt des gewählten Scopes konfiguriert. Die Typen werden von jeder Richtlinie validiert. - -| Richtlinie | Parameter | Typ und Standard | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; Einträge enthalten `regex` und `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Infrastruktur-Blocker | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - Ein Allow-Muster erweitert den Handlungsspielraum eines Agenten. Testen Sie die genaue Tokenisierung und Befehlsvarianten auf dem Ziel-Harness, bevor Sie es in einer gesamten Flotte einsetzen. - \ No newline at end of file diff --git a/docs/de/policies/builtin.mdx b/docs/de/policies/builtin.mdx deleted file mode 100644 index 6f9c9367c..000000000 --- a/docs/de/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Eingebaute Richtlinien" -description: "Aktiviere gepflegte Schutzmaßnahmen für häufige Agent-Fehlerszenarien." -icon: "library" ---- - -Eingebaute Richtlinien decken die Behandlung von Geheimnissen, Umgebungsdateien, destruktive Shell-Befehle, geschützte Branches, Cloud- und Infrastruktur-Tools, Paketveröffentlichungen, wiederholte Aufrufe sowie Workflow-Prüfungen am Aufgabenende ab. - -## Eine eingebaute Richtlinie aktivieren und überprüfen - - - - 1. Installiere die Richtlinie auf einem verbundenen Rechner mit der lokalen CLI. - 2. Führe eine sichere Testaktion im instrumentierten Agenten aus. - 3. Gehe zu **Observe → policy** und filtere nach dem Richtliniennamen, der Maschinenumgebung oder der Entscheidung. - 4. Öffne die verknüpfte Sitzung, um die übereinstimmende Tool-Eingabe und den zurückgegebenen Grund zu bestätigen. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - Entferne sie mit `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -Alle in der installierten Version verfügbaren Richtlinien auflisten: - -```bash -failproofai policies -``` - -Eine Richtlinie für ein Projekt aktivieren: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Mehrere Richtlinien für ausgewählte Harnesses aktivieren: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Einige Richtlinien akzeptieren Parameter oder sind als Beta markiert. Überprüfe Beschreibung, Geltungsbereich und Standardverhalten vor dem Rollout. Eine Richtlinie, die einen Workflow schützt, kann in einem anderen gültige Operationen blockieren. - - - Alle 40 aktuellen Richtlinien, ihre Auslöser, die empfohlene Grundkonfiguration und Parameter einsehen. - - - - Bevorzuge den Projektbereich für repository-spezifische Anforderungen und den Benutzerbereich für maschinenweite Sicherheitsanforderungen. - \ No newline at end of file diff --git a/docs/de/policies/custom.mdx b/docs/de/policies/custom.mdx deleted file mode 100644 index 1fabbf193..000000000 --- a/docs/de/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Benutzerdefinierte Richtlinien" -description: "Schreibe eine Richtlinie für einen Fehlerfall, der einzigartig für deinen Agenten-Workflow ist." -icon: "shield-plus" ---- - -Erstelle eine Datei mit der Endung `policies.js`, `policies.mjs` oder `policies.ts` unter `.failproofai/policies/`. Konventionsdateien werden automatisch auf Projekt- und Benutzerebene geladen. - -## Richtlinie vor der Cloud-Veröffentlichung testen - - - - 1. Installiere die benutzerdefinierte Richtlinie auf einem Testrechner und löse sowohl eine übereinstimmende Aktion als auch eine legitime Nicht-Übereinstimmung aus. - 2. Gehe zu **Observe → policy** und vergleiche die beiden Entscheidungen. - 3. Öffne jede verknüpfte Sitzung und überprüfe, ob das Ereignis-Payload genügend Belege für die Regel enthält. - 4. Wenn das Verhalten korrekt ist, verschiebe den geprüften Quellcode in **Admin → policy editor** und veröffentliche eine Version. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - Konventionsdateien unter `.failproofai/policies/` werden ohne `--custom` geladen. Behalte einen expliziten Installationsbefehl in CI, wenn die Validierung bei einem defekten Modul fehlschlagen soll. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -Dies trifft auf `production/config.yml`, `/srv/production/config.yml`, `/srv/production` und `C:\\production\\config.yml` sowohl für `Write` als auch für `Edit` zu. Namen wie `production-backup` werden nicht erfasst, da `production` ein vollständiges Pfadsegment sein muss. - -Validiere und installiere eine explizite Datei: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -Der Richtlinienkontext enthält den Ereignistyp, die normalisierte Nutzlast, den Tool-Namen und die Tool-Eingabe, Sitzungsmetadaten, Parameter sowie das Quell-CLI, sofern verfügbar. - -## Fehlerpfade testen - -Führe die Validierung nach dem Ändern der Einstiegsdatei oder eines lokalen Moduls, das sie importiert, erneut aus: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -Der strikte CLI-Pfad schlägt bei fehlenden Dateien, Syntaxfehlern, nicht auflösbaren Importen, Ausnahmen auf oberster Ebene und Zeitüberschreitungen beim Modulladen fehl. Zur Durchsetzungszeit wird eine defekte benutzerdefinierte Datei protokolliert und übersprungen, damit eingebaute Richtlinien weiter ausgeführt werden können. Behandle jede Ladewarnung als Verlust der erwarteten Durchsetzung und löse in Produktionsprotokollen einen Alarm aus. - -Verwende global eindeutige Namen für explizite, konventionsbasierte und Cloud-verwaltete Richtlinien. Halte Richtlinienfunktionen deterministisch, begrenze externe Aufrufe mit kurzen Timeouts und gib auf jedem Pfad ein bewusstes `allow`, `instruct` oder `deny` zurück. - - - Eine benutzerdefinierte Richtlinie ist Durchsetzungscode. Teste fehlende Felder, alternative Tool-Namen und fehlerhafte Eingaben – nicht nur die erwartete Übereinstimmung. - \ No newline at end of file diff --git a/docs/de/policies/fleet.mdx b/docs/de/policies/fleet.mdx deleted file mode 100644 index 09bd69af6..000000000 --- a/docs/de/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Richtlinien auf Maschinen ausrollen" -description: "Behalten Sie den Überblick, welche Maschinen registriert sind, aktuelle Konfigurationen ausführen und die vorgesehenen Richtlinienversionen durchsetzen." -icon: "network" ---- - -Die Fleet-Abdeckung beantwortet die Frage, ob eine Richtlinie dort vorhanden ist, wo das Risiko besteht. Verfolgen Sie Maschinen anhand einer stabilen ID und einer lesbaren Bezeichnung, und vergleichen Sie dann den zugewiesenen und gemeldeten Deployment-Status. - -## Abdeckung prüfen - - - - 1. Gehen Sie zu **Admin → Durchsetzung** und überprüfen Sie die Gesamtzahlen für „Durchsetzend" und „Beobachtend". - 2. Suchen Sie nach einer Maschine anhand von ID oder Bezeichnung, oder filtern Sie nach Maschinen ohne zugewiesene Richtlinie. - 3. Erweitern Sie eine Zeile, um zugewiesene Richtlinien, gemeldetes Deployment, letzten Check-in und Verlauf zu vergleichen. - 4. Aktualisieren Sie die Ansicht nach dem Abfrageintervall der Maschine, wenn ein angewendetes Deployment noch aussteht. - - ![Das Enforcement-Fleet mit Richtlinienabdeckung, Deployment-Status der Maschine sowie Beobachtungs- und Durchsetzungszuweisungen.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Verwenden Sie `fp events --agent-id --since 24h`, um zu bestätigen, dass die Agentaktivität der Maschine die Cloud erreicht. - - - -Nutzen Sie die Abdeckungsansichten, um Folgendes zu ermitteln: - -- Maschinen, die das neueste Deployment nie abgerufen haben -- Registrierte Maschinen, die keine Aktivität mehr melden -- Eine Richtlinie, die der falschen Umgebung oder Gruppe zugewiesen wurde -- Versionsabweichungen nach einem unterbrochenen Update - -Benennen Sie eine Maschine um, ohne sie neu zu verbinden: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Lokalen Status prüfen: - -```bash -failproofai config --status -``` - - - Verwenden Sie Bezeichnungen, die Workload und Umgebung identifizieren. Hostnamen allein sind nach Autoscaling oder Maschinenaustausch oft nicht ausreichend. - \ No newline at end of file diff --git a/docs/docs.json b/docs/docs.json index cda19dd2f..2c90430ab 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -112,7 +112,6 @@ "sessions/policy-decisions", "sessions/tools", "sessions/errors", - "sessions/evaluations", "sessions/metrics", "sessions/dashboards", "sessions/queries" @@ -121,6 +120,7 @@ { "group": "Plug in your agent", "pages": [ + "start/integrations", "start/integrations/custom-agents", "start/integrations/langchain", "start/integrations/crewai", @@ -134,6 +134,16 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "evaluations/overview", + "evaluations/write", + "evaluations/test", + "evaluations/deploy", + "sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -157,17 +167,23 @@ { "group": "Prevent repeat failures", "pages": [ - "policies/overview", - "policies/builtin", - "policies/builtin-catalog", - "policies/custom", - "policies/packs", - "policies/publish-a-pack", - "policies/local-configuration", + "policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "policies/editor", + "policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ + "policies/test", "policies/deploy", - "policies/fleet", "policies/rollback", + "policies/publish-a-pack", "policies/failure-behavior" ] } @@ -210,6 +226,7 @@ "pages": [ "reference/failproof-cli", "reference/local-dashboard", + "policies/local-configuration", "reference/cloud-cli", "reference/http-api", "reference/events-and-configuration", @@ -274,7 +291,6 @@ "zh/sessions/policy-decisions", "zh/sessions/tools", "zh/sessions/errors", - "zh/sessions/evaluations", "zh/sessions/metrics", "zh/sessions/dashboards", "zh/sessions/queries" @@ -283,6 +299,7 @@ { "group": "Plug in your agent", "pages": [ + "zh/start/integrations", "zh/start/integrations/custom-agents", "zh/start/integrations/langchain", "zh/start/integrations/crewai", @@ -296,6 +313,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "zh/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -319,17 +342,22 @@ { "group": "Prevent repeat failures", "pages": [ - "zh/policies/overview", - "zh/policies/builtin", - "zh/policies/builtin-catalog", - "zh/policies/custom", - "zh/policies/packs", - "zh/policies/publish-a-pack", - "zh/policies/local-configuration", + "zh/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "zh/policies/editor", + "zh/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "zh/policies/deploy", - "zh/policies/fleet", "zh/policies/rollback", + "zh/policies/publish-a-pack", "zh/policies/failure-behavior" ] } @@ -372,6 +400,7 @@ "pages": [ "zh/reference/failproof-cli", "zh/reference/local-dashboard", + "zh/policies/local-configuration", "zh/reference/cloud-cli", "zh/reference/http-api", "zh/reference/events-and-configuration", @@ -430,7 +459,6 @@ "ja/sessions/policy-decisions", "ja/sessions/tools", "ja/sessions/errors", - "ja/sessions/evaluations", "ja/sessions/metrics", "ja/sessions/dashboards", "ja/sessions/queries" @@ -439,6 +467,7 @@ { "group": "Plug in your agent", "pages": [ + "ja/start/integrations", "ja/start/integrations/custom-agents", "ja/start/integrations/langchain", "ja/start/integrations/crewai", @@ -452,6 +481,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "ja/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -475,17 +510,22 @@ { "group": "Prevent repeat failures", "pages": [ - "ja/policies/overview", - "ja/policies/builtin", - "ja/policies/builtin-catalog", - "ja/policies/custom", - "ja/policies/packs", - "ja/policies/publish-a-pack", - "ja/policies/local-configuration", + "ja/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "ja/policies/editor", + "ja/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "ja/policies/deploy", - "ja/policies/fleet", "ja/policies/rollback", + "ja/policies/publish-a-pack", "ja/policies/failure-behavior" ] } @@ -528,6 +568,7 @@ "pages": [ "ja/reference/failproof-cli", "ja/reference/local-dashboard", + "ja/policies/local-configuration", "ja/reference/cloud-cli", "ja/reference/http-api", "ja/reference/events-and-configuration", @@ -586,7 +627,6 @@ "ko/sessions/policy-decisions", "ko/sessions/tools", "ko/sessions/errors", - "ko/sessions/evaluations", "ko/sessions/metrics", "ko/sessions/dashboards", "ko/sessions/queries" @@ -595,6 +635,7 @@ { "group": "Plug in your agent", "pages": [ + "ko/start/integrations", "ko/start/integrations/custom-agents", "ko/start/integrations/langchain", "ko/start/integrations/crewai", @@ -608,6 +649,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "ko/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -631,17 +678,22 @@ { "group": "Prevent repeat failures", "pages": [ - "ko/policies/overview", - "ko/policies/builtin", - "ko/policies/builtin-catalog", - "ko/policies/custom", - "ko/policies/packs", - "ko/policies/publish-a-pack", - "ko/policies/local-configuration", + "ko/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "ko/policies/editor", + "ko/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "ko/policies/deploy", - "ko/policies/fleet", "ko/policies/rollback", + "ko/policies/publish-a-pack", "ko/policies/failure-behavior" ] } @@ -684,6 +736,7 @@ "pages": [ "ko/reference/failproof-cli", "ko/reference/local-dashboard", + "ko/policies/local-configuration", "ko/reference/cloud-cli", "ko/reference/http-api", "ko/reference/events-and-configuration", @@ -742,7 +795,6 @@ "es/sessions/policy-decisions", "es/sessions/tools", "es/sessions/errors", - "es/sessions/evaluations", "es/sessions/metrics", "es/sessions/dashboards", "es/sessions/queries" @@ -751,6 +803,7 @@ { "group": "Plug in your agent", "pages": [ + "es/start/integrations", "es/start/integrations/custom-agents", "es/start/integrations/langchain", "es/start/integrations/crewai", @@ -764,6 +817,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "es/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -787,17 +846,22 @@ { "group": "Prevent repeat failures", "pages": [ - "es/policies/overview", - "es/policies/builtin", - "es/policies/builtin-catalog", - "es/policies/custom", - "es/policies/packs", - "es/policies/publish-a-pack", - "es/policies/local-configuration", + "es/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "es/policies/editor", + "es/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "es/policies/deploy", - "es/policies/fleet", "es/policies/rollback", + "es/policies/publish-a-pack", "es/policies/failure-behavior" ] } @@ -840,6 +904,7 @@ "pages": [ "es/reference/failproof-cli", "es/reference/local-dashboard", + "es/policies/local-configuration", "es/reference/cloud-cli", "es/reference/http-api", "es/reference/events-and-configuration", @@ -898,7 +963,6 @@ "pt-br/sessions/policy-decisions", "pt-br/sessions/tools", "pt-br/sessions/errors", - "pt-br/sessions/evaluations", "pt-br/sessions/metrics", "pt-br/sessions/dashboards", "pt-br/sessions/queries" @@ -907,6 +971,7 @@ { "group": "Plug in your agent", "pages": [ + "pt-br/start/integrations", "pt-br/start/integrations/custom-agents", "pt-br/start/integrations/langchain", "pt-br/start/integrations/crewai", @@ -920,6 +985,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "pt-br/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -943,17 +1014,22 @@ { "group": "Prevent repeat failures", "pages": [ - "pt-br/policies/overview", - "pt-br/policies/builtin", - "pt-br/policies/builtin-catalog", - "pt-br/policies/custom", - "pt-br/policies/packs", - "pt-br/policies/publish-a-pack", - "pt-br/policies/local-configuration", + "pt-br/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "pt-br/policies/editor", + "pt-br/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "pt-br/policies/deploy", - "pt-br/policies/fleet", "pt-br/policies/rollback", + "pt-br/policies/publish-a-pack", "pt-br/policies/failure-behavior" ] } @@ -996,6 +1072,7 @@ "pages": [ "pt-br/reference/failproof-cli", "pt-br/reference/local-dashboard", + "pt-br/policies/local-configuration", "pt-br/reference/cloud-cli", "pt-br/reference/http-api", "pt-br/reference/events-and-configuration", @@ -1054,7 +1131,6 @@ "de/sessions/policy-decisions", "de/sessions/tools", "de/sessions/errors", - "de/sessions/evaluations", "de/sessions/metrics", "de/sessions/dashboards", "de/sessions/queries" @@ -1063,6 +1139,7 @@ { "group": "Plug in your agent", "pages": [ + "de/start/integrations", "de/start/integrations/custom-agents", "de/start/integrations/langchain", "de/start/integrations/crewai", @@ -1076,6 +1153,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "de/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -1099,17 +1182,22 @@ { "group": "Prevent repeat failures", "pages": [ - "de/policies/overview", - "de/policies/builtin", - "de/policies/builtin-catalog", - "de/policies/custom", - "de/policies/packs", - "de/policies/publish-a-pack", - "de/policies/local-configuration", + "de/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "de/policies/editor", + "de/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "de/policies/deploy", - "de/policies/fleet", "de/policies/rollback", + "de/policies/publish-a-pack", "de/policies/failure-behavior" ] } @@ -1152,6 +1240,7 @@ "pages": [ "de/reference/failproof-cli", "de/reference/local-dashboard", + "de/policies/local-configuration", "de/reference/cloud-cli", "de/reference/http-api", "de/reference/events-and-configuration", @@ -1210,7 +1299,6 @@ "fr/sessions/policy-decisions", "fr/sessions/tools", "fr/sessions/errors", - "fr/sessions/evaluations", "fr/sessions/metrics", "fr/sessions/dashboards", "fr/sessions/queries" @@ -1219,6 +1307,7 @@ { "group": "Plug in your agent", "pages": [ + "fr/start/integrations", "fr/start/integrations/custom-agents", "fr/start/integrations/langchain", "fr/start/integrations/crewai", @@ -1232,6 +1321,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "fr/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -1255,17 +1350,22 @@ { "group": "Prevent repeat failures", "pages": [ - "fr/policies/overview", - "fr/policies/builtin", - "fr/policies/builtin-catalog", - "fr/policies/custom", - "fr/policies/packs", - "fr/policies/publish-a-pack", - "fr/policies/local-configuration", + "fr/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "fr/policies/editor", + "fr/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "fr/policies/deploy", - "fr/policies/fleet", "fr/policies/rollback", + "fr/policies/publish-a-pack", "fr/policies/failure-behavior" ] } @@ -1308,6 +1408,7 @@ "pages": [ "fr/reference/failproof-cli", "fr/reference/local-dashboard", + "fr/policies/local-configuration", "fr/reference/cloud-cli", "fr/reference/http-api", "fr/reference/events-and-configuration", @@ -1366,7 +1467,6 @@ "ru/sessions/policy-decisions", "ru/sessions/tools", "ru/sessions/errors", - "ru/sessions/evaluations", "ru/sessions/metrics", "ru/sessions/dashboards", "ru/sessions/queries" @@ -1375,6 +1475,7 @@ { "group": "Plug in your agent", "pages": [ + "ru/start/integrations", "ru/start/integrations/custom-agents", "ru/start/integrations/langchain", "ru/start/integrations/crewai", @@ -1388,6 +1489,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "ru/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -1411,17 +1518,22 @@ { "group": "Prevent repeat failures", "pages": [ - "ru/policies/overview", - "ru/policies/builtin", - "ru/policies/builtin-catalog", - "ru/policies/custom", - "ru/policies/packs", - "ru/policies/publish-a-pack", - "ru/policies/local-configuration", + "ru/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "ru/policies/editor", + "ru/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "ru/policies/deploy", - "ru/policies/fleet", "ru/policies/rollback", + "ru/policies/publish-a-pack", "ru/policies/failure-behavior" ] } @@ -1464,6 +1576,7 @@ "pages": [ "ru/reference/failproof-cli", "ru/reference/local-dashboard", + "ru/policies/local-configuration", "ru/reference/cloud-cli", "ru/reference/http-api", "ru/reference/events-and-configuration", @@ -1522,7 +1635,6 @@ "hi/sessions/policy-decisions", "hi/sessions/tools", "hi/sessions/errors", - "hi/sessions/evaluations", "hi/sessions/metrics", "hi/sessions/dashboards", "hi/sessions/queries" @@ -1531,6 +1643,7 @@ { "group": "Plug in your agent", "pages": [ + "hi/start/integrations", "hi/start/integrations/custom-agents", "hi/start/integrations/langchain", "hi/start/integrations/crewai", @@ -1544,6 +1657,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "hi/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -1567,17 +1686,22 @@ { "group": "Prevent repeat failures", "pages": [ - "hi/policies/overview", - "hi/policies/builtin", - "hi/policies/builtin-catalog", - "hi/policies/custom", - "hi/policies/packs", - "hi/policies/publish-a-pack", - "hi/policies/local-configuration", + "hi/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "hi/policies/editor", + "hi/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "hi/policies/deploy", - "hi/policies/fleet", "hi/policies/rollback", + "hi/policies/publish-a-pack", "hi/policies/failure-behavior" ] } @@ -1620,6 +1744,7 @@ "pages": [ "hi/reference/failproof-cli", "hi/reference/local-dashboard", + "hi/policies/local-configuration", "hi/reference/cloud-cli", "hi/reference/http-api", "hi/reference/events-and-configuration", @@ -1678,7 +1803,6 @@ "tr/sessions/policy-decisions", "tr/sessions/tools", "tr/sessions/errors", - "tr/sessions/evaluations", "tr/sessions/metrics", "tr/sessions/dashboards", "tr/sessions/queries" @@ -1687,6 +1811,7 @@ { "group": "Plug in your agent", "pages": [ + "tr/start/integrations", "tr/start/integrations/custom-agents", "tr/start/integrations/langchain", "tr/start/integrations/crewai", @@ -1700,6 +1825,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "tr/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -1723,17 +1854,22 @@ { "group": "Prevent repeat failures", "pages": [ - "tr/policies/overview", - "tr/policies/builtin", - "tr/policies/builtin-catalog", - "tr/policies/custom", - "tr/policies/packs", - "tr/policies/publish-a-pack", - "tr/policies/local-configuration", + "tr/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "tr/policies/editor", + "tr/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "tr/policies/deploy", - "tr/policies/fleet", "tr/policies/rollback", + "tr/policies/publish-a-pack", "tr/policies/failure-behavior" ] } @@ -1776,6 +1912,7 @@ "pages": [ "tr/reference/failproof-cli", "tr/reference/local-dashboard", + "tr/policies/local-configuration", "tr/reference/cloud-cli", "tr/reference/http-api", "tr/reference/events-and-configuration", @@ -1834,7 +1971,6 @@ "vi/sessions/policy-decisions", "vi/sessions/tools", "vi/sessions/errors", - "vi/sessions/evaluations", "vi/sessions/metrics", "vi/sessions/dashboards", "vi/sessions/queries" @@ -1843,6 +1979,7 @@ { "group": "Plug in your agent", "pages": [ + "vi/start/integrations", "vi/start/integrations/custom-agents", "vi/start/integrations/langchain", "vi/start/integrations/crewai", @@ -1856,6 +1993,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "vi/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -1879,17 +2022,22 @@ { "group": "Prevent repeat failures", "pages": [ - "vi/policies/overview", - "vi/policies/builtin", - "vi/policies/builtin-catalog", - "vi/policies/custom", - "vi/policies/packs", - "vi/policies/publish-a-pack", - "vi/policies/local-configuration", + "vi/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "vi/policies/editor", + "vi/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "vi/policies/deploy", - "vi/policies/fleet", "vi/policies/rollback", + "vi/policies/publish-a-pack", "vi/policies/failure-behavior" ] } @@ -1932,6 +2080,7 @@ "pages": [ "vi/reference/failproof-cli", "vi/reference/local-dashboard", + "vi/policies/local-configuration", "vi/reference/cloud-cli", "vi/reference/http-api", "vi/reference/events-and-configuration", @@ -1990,7 +2139,6 @@ "it/sessions/policy-decisions", "it/sessions/tools", "it/sessions/errors", - "it/sessions/evaluations", "it/sessions/metrics", "it/sessions/dashboards", "it/sessions/queries" @@ -1999,6 +2147,7 @@ { "group": "Plug in your agent", "pages": [ + "it/start/integrations", "it/start/integrations/custom-agents", "it/start/integrations/langchain", "it/start/integrations/crewai", @@ -2012,6 +2161,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "it/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -2035,17 +2190,22 @@ { "group": "Prevent repeat failures", "pages": [ - "it/policies/overview", - "it/policies/builtin", - "it/policies/builtin-catalog", - "it/policies/custom", - "it/policies/packs", - "it/policies/publish-a-pack", - "it/policies/local-configuration", + "it/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "it/policies/editor", + "it/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "it/policies/deploy", - "it/policies/fleet", "it/policies/rollback", + "it/policies/publish-a-pack", "it/policies/failure-behavior" ] } @@ -2088,6 +2248,7 @@ "pages": [ "it/reference/failproof-cli", "it/reference/local-dashboard", + "it/policies/local-configuration", "it/reference/cloud-cli", "it/reference/http-api", "it/reference/events-and-configuration", @@ -2146,7 +2307,6 @@ "ar/sessions/policy-decisions", "ar/sessions/tools", "ar/sessions/errors", - "ar/sessions/evaluations", "ar/sessions/metrics", "ar/sessions/dashboards", "ar/sessions/queries" @@ -2155,6 +2315,7 @@ { "group": "Plug in your agent", "pages": [ + "ar/start/integrations", "ar/start/integrations/custom-agents", "ar/start/integrations/langchain", "ar/start/integrations/crewai", @@ -2168,6 +2329,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "ar/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -2191,17 +2358,22 @@ { "group": "Prevent repeat failures", "pages": [ - "ar/policies/overview", - "ar/policies/builtin", - "ar/policies/builtin-catalog", - "ar/policies/custom", - "ar/policies/packs", - "ar/policies/publish-a-pack", - "ar/policies/local-configuration", + "ar/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "ar/policies/editor", + "ar/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "ar/policies/deploy", - "ar/policies/fleet", "ar/policies/rollback", + "ar/policies/publish-a-pack", "ar/policies/failure-behavior" ] } @@ -2244,6 +2416,7 @@ "pages": [ "ar/reference/failproof-cli", "ar/reference/local-dashboard", + "ar/policies/local-configuration", "ar/reference/cloud-cli", "ar/reference/http-api", "ar/reference/events-and-configuration", @@ -2302,7 +2475,6 @@ "he/sessions/policy-decisions", "he/sessions/tools", "he/sessions/errors", - "he/sessions/evaluations", "he/sessions/metrics", "he/sessions/dashboards", "he/sessions/queries" @@ -2311,6 +2483,7 @@ { "group": "Plug in your agent", "pages": [ + "he/start/integrations", "he/start/integrations/custom-agents", "he/start/integrations/langchain", "he/start/integrations/crewai", @@ -2324,6 +2497,12 @@ "tab": "Find failures", "icon": "scan-search", "groups": [ + { + "group": "Evaluate agents", + "pages": [ + "he/sessions/evaluations" + ] + }, { "group": "Find and manage failures", "pages": [ @@ -2347,17 +2526,22 @@ { "group": "Prevent repeat failures", "pages": [ - "he/policies/overview", - "he/policies/builtin", - "he/policies/builtin-catalog", - "he/policies/custom", - "he/policies/packs", - "he/policies/publish-a-pack", - "he/policies/local-configuration", + "he/policies/overview" + ] + }, + { + "group": "Get a policy", + "pages": [ "he/policies/editor", + "he/policies/packs" + ] + }, + { + "group": "Ship a policy", + "pages": [ "he/policies/deploy", - "he/policies/fleet", "he/policies/rollback", + "he/policies/publish-a-pack", "he/policies/failure-behavior" ] } @@ -2400,6 +2584,7 @@ "pages": [ "he/reference/failproof-cli", "he/reference/local-dashboard", + "he/policies/local-configuration", "he/reference/cloud-cli", "he/reference/http-api", "he/reference/events-and-configuration", @@ -2507,6 +2692,306 @@ { "source": "/start/integrations/how-it-works", "destination": "/start/integrations/custom-agents" + }, + { + "source": "/policies/builtin", + "destination": "/policies/packs" + }, + { + "source": "/zh/policies/builtin", + "destination": "/zh/policies/packs" + }, + { + "source": "/ja/policies/builtin", + "destination": "/ja/policies/packs" + }, + { + "source": "/ko/policies/builtin", + "destination": "/ko/policies/packs" + }, + { + "source": "/es/policies/builtin", + "destination": "/es/policies/packs" + }, + { + "source": "/pt-br/policies/builtin", + "destination": "/pt-br/policies/packs" + }, + { + "source": "/de/policies/builtin", + "destination": "/de/policies/packs" + }, + { + "source": "/fr/policies/builtin", + "destination": "/fr/policies/packs" + }, + { + "source": "/ru/policies/builtin", + "destination": "/ru/policies/packs" + }, + { + "source": "/hi/policies/builtin", + "destination": "/hi/policies/packs" + }, + { + "source": "/tr/policies/builtin", + "destination": "/tr/policies/packs" + }, + { + "source": "/vi/policies/builtin", + "destination": "/vi/policies/packs" + }, + { + "source": "/it/policies/builtin", + "destination": "/it/policies/packs" + }, + { + "source": "/ar/policies/builtin", + "destination": "/ar/policies/packs" + }, + { + "source": "/he/policies/builtin", + "destination": "/he/policies/packs" + }, + { + "source": "/policies/builtin-catalog", + "destination": "/policies/packs" + }, + { + "source": "/zh/policies/builtin-catalog", + "destination": "/zh/policies/packs" + }, + { + "source": "/ja/policies/builtin-catalog", + "destination": "/ja/policies/packs" + }, + { + "source": "/ko/policies/builtin-catalog", + "destination": "/ko/policies/packs" + }, + { + "source": "/es/policies/builtin-catalog", + "destination": "/es/policies/packs" + }, + { + "source": "/pt-br/policies/builtin-catalog", + "destination": "/pt-br/policies/packs" + }, + { + "source": "/de/policies/builtin-catalog", + "destination": "/de/policies/packs" + }, + { + "source": "/fr/policies/builtin-catalog", + "destination": "/fr/policies/packs" + }, + { + "source": "/ru/policies/builtin-catalog", + "destination": "/ru/policies/packs" + }, + { + "source": "/hi/policies/builtin-catalog", + "destination": "/hi/policies/packs" + }, + { + "source": "/tr/policies/builtin-catalog", + "destination": "/tr/policies/packs" + }, + { + "source": "/vi/policies/builtin-catalog", + "destination": "/vi/policies/packs" + }, + { + "source": "/it/policies/builtin-catalog", + "destination": "/it/policies/packs" + }, + { + "source": "/ar/policies/builtin-catalog", + "destination": "/ar/policies/packs" + }, + { + "source": "/he/policies/builtin-catalog", + "destination": "/he/policies/packs" + }, + { + "source": "/policies/custom", + "destination": "/policies/editor" + }, + { + "source": "/zh/policies/custom", + "destination": "/zh/policies/editor" + }, + { + "source": "/ja/policies/custom", + "destination": "/ja/policies/editor" + }, + { + "source": "/ko/policies/custom", + "destination": "/ko/policies/editor" + }, + { + "source": "/es/policies/custom", + "destination": "/es/policies/editor" + }, + { + "source": "/pt-br/policies/custom", + "destination": "/pt-br/policies/editor" + }, + { + "source": "/de/policies/custom", + "destination": "/de/policies/editor" + }, + { + "source": "/fr/policies/custom", + "destination": "/fr/policies/editor" + }, + { + "source": "/ru/policies/custom", + "destination": "/ru/policies/editor" + }, + { + "source": "/hi/policies/custom", + "destination": "/hi/policies/editor" + }, + { + "source": "/tr/policies/custom", + "destination": "/tr/policies/editor" + }, + { + "source": "/vi/policies/custom", + "destination": "/vi/policies/editor" + }, + { + "source": "/it/policies/custom", + "destination": "/it/policies/editor" + }, + { + "source": "/ar/policies/custom", + "destination": "/ar/policies/editor" + }, + { + "source": "/he/policies/custom", + "destination": "/he/policies/editor" + }, + { + "source": "/policies/fleet", + "destination": "/policies/deploy" + }, + { + "source": "/zh/policies/fleet", + "destination": "/zh/policies/deploy" + }, + { + "source": "/ja/policies/fleet", + "destination": "/ja/policies/deploy" + }, + { + "source": "/ko/policies/fleet", + "destination": "/ko/policies/deploy" + }, + { + "source": "/es/policies/fleet", + "destination": "/es/policies/deploy" + }, + { + "source": "/pt-br/policies/fleet", + "destination": "/pt-br/policies/deploy" + }, + { + "source": "/de/policies/fleet", + "destination": "/de/policies/deploy" + }, + { + "source": "/fr/policies/fleet", + "destination": "/fr/policies/deploy" + }, + { + "source": "/ru/policies/fleet", + "destination": "/ru/policies/deploy" + }, + { + "source": "/hi/policies/fleet", + "destination": "/hi/policies/deploy" + }, + { + "source": "/tr/policies/fleet", + "destination": "/tr/policies/deploy" + }, + { + "source": "/vi/policies/fleet", + "destination": "/vi/policies/deploy" + }, + { + "source": "/it/policies/fleet", + "destination": "/it/policies/deploy" + }, + { + "source": "/ar/policies/fleet", + "destination": "/ar/policies/deploy" + }, + { + "source": "/he/policies/fleet", + "destination": "/he/policies/deploy" + }, + { + "source": "/built-in-policies", + "destination": "/policies/packs" + }, + { + "source": "/zh/built-in-policies", + "destination": "/zh/policies/packs" + }, + { + "source": "/ja/built-in-policies", + "destination": "/ja/policies/packs" + }, + { + "source": "/ko/built-in-policies", + "destination": "/ko/policies/packs" + }, + { + "source": "/es/built-in-policies", + "destination": "/es/policies/packs" + }, + { + "source": "/pt-br/built-in-policies", + "destination": "/pt-br/policies/packs" + }, + { + "source": "/de/built-in-policies", + "destination": "/de/policies/packs" + }, + { + "source": "/fr/built-in-policies", + "destination": "/fr/policies/packs" + }, + { + "source": "/ru/built-in-policies", + "destination": "/ru/policies/packs" + }, + { + "source": "/hi/built-in-policies", + "destination": "/hi/policies/packs" + }, + { + "source": "/tr/built-in-policies", + "destination": "/tr/policies/packs" + }, + { + "source": "/vi/built-in-policies", + "destination": "/vi/policies/packs" + }, + { + "source": "/it/built-in-policies", + "destination": "/it/policies/packs" + }, + { + "source": "/ar/built-in-policies", + "destination": "/ar/policies/packs" + }, + { + "source": "/he/built-in-policies", + "destination": "/he/policies/packs" } ] } diff --git a/docs/es/policies/builtin-catalog.mdx b/docs/es/policies/builtin-catalog.mdx deleted file mode 100644 index 96584d219..000000000 --- a/docs/es/policies/builtin-catalog.mdx +++ /dev/null @@ -1,132 +0,0 @@ ---- -title: "Catálogo de políticas integradas" -description: "Revisa cada política integrada de Failproof AI, su disparador, estado recomendado y parámetros configurables." -icon: "list-checks" ---- - -El paquete instalado es la fuente de verdad para la disponibilidad de políticas. Ejecuta `failproofai policies` tras cada actualización, ya que las entradas del catálogo y el comportamiento pueden cambiar con la versión del paquete. - -## Línea base recomendada - -La selección recomendada por la configuración guiada activa actualmente sanitizadores de secretos, protecciones de entorno, autoprotección, guardas contra comandos catastróficos y seguridad de ramas protegidas: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` está **siempre activo**. Se incluye arriba por -completitud, pero se registra en cada evaluación independientemente de si aparece en -tu conjunto habilitado, y no puede desactivarse ni pausarse — una guarda contra -que el agente desactive la aplicación que el propio agente puede desactivar no es una guarda. - -La selección recomendada es deliberadamente más reducida que **Todo**. Las políticas de infraestructura y flujo de trabajo pueden interrumpir trabajo válido y deben habilitarse solo en los repositorios y máquinas que las necesiten. - -## Secretos y entorno - -| Política | Disparador | Resultado | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | Redacta JWTs de la salida de herramientas antes de que el modelo los vea. | -| `sanitize-api-keys` | `PostToolUse` | Redacta claves comunes de OpenAI, Anthropic, GitHub, AWS, Stripe y Google. | -| `sanitize-connection-strings` | `PostToolUse` | Redacta cadenas de conexión a bases de datos que contengan credenciales. | -| `sanitize-private-key-content` | `PostToolUse` | Redacta cuerpos de claves privadas PEM. | -| `sanitize-bearer-tokens` | `PostToolUse` | Redacta tokens de autorización bearer. | -| `protect-env-vars` | `PreToolUse` en herramientas de shell | Bloquea comandos que exponen variables de entorno. | -| `block-env-files` | `PreToolUse` | Bloquea lecturas y escrituras de archivos `.env`. | -| `block-read-outside-cwd` | `PreToolUse` en herramientas de lectura, glob, grep o shell | Restringe las lecturas al directorio de trabajo de la sesión. | -| `block-secrets-write` | `PreToolUse` en herramientas de escritura | Bloquea escrituras en nombres de archivos comunes de claves secretas y credenciales. | - -## Comandos peligrosos e infraestructura - -| Política | Disparador | Resultado | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | Bloquea `sudo` a menos que coincida un patrón de allow. | -| `block-curl-pipe-sh` | `PreToolUse` | Bloquea scripts descargados canalizados directamente a un shell. | -| `block-rm-rf` | `PreToolUse` | Bloquea patrones de eliminación recursiva catastrófica. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Siempre activo, no puede desactivarse.** Bloquea toda invocación de la CLI de Failproof AI, autopausa y desinstalación mediante gestores de paquetes. | -| `block-kubectl` | `PreToolUse` | Controla comandos de Kubernetes. | -| `block-terraform` | `PreToolUse` | Controla comandos de Terraform y OpenTofu. | -| `block-aws-cli` | `PreToolUse` | Controla comandos de la CLI de AWS. | -| `block-gcloud` | `PreToolUse` | Controla comandos de la CLI de Google Cloud. | -| `block-az-cli` | `PreToolUse` | Controla comandos de la CLI de Azure. | -| `block-helm` | `PreToolUse` | Controla comandos de Helm. | -| `block-gh-pipeline` | `PreToolUse` | Controla operaciones mutantes de flujos de trabajo, ejecuciones, fusiones, releases, caché y secretos de la CLI de GitHub. | - -## Seguridad de Git y bases de datos - -| Política | Disparador | Resultado | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Bloquea pushes directos a las ramas protegidas configuradas. | -| `block-force-push` | `PreToolUse` | Bloquea force-pushes; `--force-with-lease` permanece permitido en la implementación actual. | -| `block-work-on-main` | `PreToolUse` | Bloquea commits y fusiones en ramas protegidas. | -| `warn-git-amend` | `PreToolUse` | Advierte antes de reescribir un commit con `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | Advierte antes de eliminar o limpiar stashes de forma permanente. | -| `warn-all-files-staged` | `PreToolUse` | Advierte sobre el uso amplio de `git add -A`, `git add .` o `git add --all`. | -| `warn-destructive-sql` | `PreToolUse` | Advierte sobre `DROP`, `TRUNCATE` y `DELETE` sin `WHERE` a través de clientes de base de datos reconocidos. | -| `warn-schema-alteration` | `PreToolUse` | Advierte sobre operaciones reconocidas de `ALTER TABLE` en columnas y renombrados. | - -## Paquetes, comportamiento del sistema y bucles del agente - -| Política | Disparador | Resultado | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Advierte antes de publicar en registros de paquetes. | -| `warn-global-package-install` | `PreToolUse` | Advierte antes de instalar paquetes de forma global. | -| `prefer-package-manager` | `PreToolUse` | Instruye al agente para que use un gestor de paquetes permitido. | -| `warn-large-file-write` | `PreToolUse` en herramientas de escritura | Advierte cuando se supera el umbral de tamaño de archivo configurado. | -| `warn-background-process` | `PreToolUse` | Advierte sobre patrones de procesos en segundo plano desasociados o de larga duración. | -| `warn-repeated-tool-calls` | `PreToolUse` | Advierte tras tres o más llamadas idénticas a una herramienta. | - -## Flujo de trabajo al finalizar la tarea - -Estas políticas requieren un harness que emita un evento `Stop` compatible. - -| Política | Resultado | -| --- | --- | -| `require-commit-before-stop` | Rechaza la finalización mientras haya trabajo rastreado sin confirmar. | -| `require-push-before-stop` | Rechaza la finalización mientras haya commits que solo existan en local. | -| `require-pr-before-stop` | Requiere un pull request para la rama actual. | -| `require-no-conflicts-before-stop` | Requiere una fusión limpia contra la rama base configurada. | -| `require-ci-green-before-stop` | Requiere que las comprobaciones de CI del HEAD actual finalicen con éxito. | - -## Referencia de parámetros - -Configura los parámetros bajo el objeto `policyParams` del ámbito seleccionado. Los tipos son validados por cada política. - -| Política | Parámetro | Tipo y valor por defecto | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; las entradas contienen `regex` y `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Bloqueadores de infraestructura | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - Un patrón de allow amplía lo que un agente puede hacer. Prueba la tokenización exacta y las variantes de comandos en el harness de destino antes de desplegarlo en una flota. - \ No newline at end of file diff --git a/docs/es/policies/builtin.mdx b/docs/es/policies/builtin.mdx deleted file mode 100644 index aed73c03c..000000000 --- a/docs/es/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Políticas integradas" -description: "Activa salvaguardas mantenidas para los modos de fallo más comunes de los agentes." -icon: "library" ---- - -Las políticas integradas cubren el manejo de secretos, archivos de entorno, comandos de shell destructivos, ramas protegidas, herramientas de nube e infraestructura, publicación de paquetes, llamadas repetidas y verificaciones de flujo de trabajo al final de tareas. - -## Activar y verificar una política integrada - - - - 1. Instala la política en una máquina conectada con el CLI local. - 2. Ejecuta una acción de prueba segura en el agente instrumentado. - 3. Ve a **Observe → policy** y filtra por nombre de política, entorno de máquina o decisión. - 4. Abre la sesión vinculada para confirmar la entrada de herramienta coincidente y el motivo devuelto. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - Elimínala con `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -Lista las políticas disponibles en tu versión instalada: - -```bash -failproofai policies -``` - -Activa una política para un proyecto: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Activa varias políticas para los entornos seleccionados: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Algunas políticas aceptan parámetros o están marcadas como beta. Revisa la descripción, el alcance de coincidencia y el comportamiento predeterminado antes de implementarlas. Una política que protege un flujo de trabajo puede bloquear operaciones válidas en otro. - - - Revisa las 40 políticas actuales, sus disparadores, la línea base recomendada y los parámetros disponibles. - - - - Prefiere el alcance de proyecto para expectativas específicas del repositorio y el alcance de usuario para los requisitos de seguridad en toda la máquina. - \ No newline at end of file diff --git a/docs/es/policies/custom.mdx b/docs/es/policies/custom.mdx deleted file mode 100644 index bf1d2602f..000000000 --- a/docs/es/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Políticas personalizadas" -description: "Escribe una política para un modo de fallo único en tu flujo de trabajo de agente." -icon: "shield-plus" ---- - -Crea un archivo con extensión `policies.js`, `policies.mjs` o `policies.ts` dentro de `.failproofai/policies/`. Los archivos de convención se cargan automáticamente en el ámbito de proyecto y de usuario. - -## Prueba la política antes de publicarla en Cloud - - - - 1. Instala la política personalizada en una máquina de prueba y ejecuta tanto una acción que coincida como una que no deba coincidir. - 2. Ve a **Observar → política** y compara las dos decisiones. - 3. Abre cada sesión vinculada y verifica que el payload del evento contiene suficiente evidencia para la regla. - 4. Cuando el comportamiento sea correcto, mueve el código revisado a **Admin → editor de políticas** y publica una versión. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - Los archivos de convención bajo `.failproofai/policies/` se cargan sin necesidad de `--custom`. Mantén un comando de instalación explícito en CI cuando la validación deba fallar ante un módulo roto. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -Esto coincide con `production/config.yml`, `/srv/production/config.yml`, `/srv/production` y `C:\\production\\config.yml` tanto para `Write` como para `Edit`. No coincide con nombres como `production-backup`, ya que `production` debe ser un segmento completo de la ruta. - -Valida e instala un archivo explícito: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -El contexto de la política incluye el tipo de evento, el payload normalizado, el nombre e input de la herramienta, los metadatos de la sesión, los parámetros y el CLI de origen cuando esté disponible. - -## Prueba los caminos de fallo - -Ejecuta la validación después de modificar el archivo de entrada o cualquier módulo local que importe: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -La ruta CLI con --strict falla ante archivos inexistentes, errores de sintaxis, importaciones no resueltas, excepciones a nivel superior y tiempos de espera en la carga de módulos. En el momento de la aplicación, un archivo personalizado con errores se registra y se omite para que las políticas integradas puedan continuar. Trata cualquier advertencia de carga como una pérdida de la aplicación esperada y genera una alerta en los registros de producción. - -Usa nombres globalmente únicos entre políticas explícitas, de convención y gestionadas desde Cloud. Mantén las funciones de política deterministas, limita las llamadas externas con tiempos de espera cortos y devuelve un allow, instruct o deny intencionado en cada camino de ejecución. - - - Una política personalizada es código de aplicación. Prueba campos faltantes, nombres de herramienta alternativos y entradas malformadas, no solo la coincidencia esperada. - \ No newline at end of file diff --git a/docs/es/policies/fleet.mdx b/docs/es/policies/fleet.mdx deleted file mode 100644 index 0455b7d64..000000000 --- a/docs/es/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Desplegar políticas en máquinas" -description: "Conoce qué máquinas están registradas, actualizadas y aplicando las versiones de política previstas." -icon: "network" ---- - -La cobertura de flota responde si existe una política donde existe el riesgo. Realiza un seguimiento de las máquinas mediante un ID estable y una etiqueta legible por humanos, y luego compara el estado de despliegue asignado y el reportado. - -## Verificar cobertura - - - - 1. Ve a **Admin → enforcement** y revisa los totales de aplicación y observación. - 2. Busca una máquina por ID o etiqueta, o filtra las máquinas que no tienen una política asignada. - 3. Expande una fila para comparar las políticas asignadas, el despliegue reportado, el último check-in y el historial. - 4. Actualiza tras el intervalo de sondeo de la máquina cuando un despliegue aplicado permanezca pendiente. - - ![El panel de flota de Enforcement mostrando cobertura de políticas, estado de despliegue de máquinas y asignaciones de observación y aplicación.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Usa `fp events --agent-id --since 24h` para confirmar que la actividad del agente de la máquina llega a Cloud. - - - -Usa las vistas de cobertura para encontrar: - -- Máquinas que nunca obtuvieron el último despliegue -- Máquinas registradas que dejaron de reportar actividad -- Una política asignada al entorno o cohorte incorrecto -- Deriva de versiones tras una actualización interrumpida - -Renombra una máquina sin reconectarla: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Comprueba el estado local: - -```bash -failproofai config --status -``` - - - Usa etiquetas que identifiquen la carga de trabajo y el entorno. Los nombres de host por sí solos suelen ser insuficientes tras un autoescalado o reemplazo de máquina. - \ No newline at end of file diff --git a/docs/evaluations/deploy.mdx b/docs/evaluations/deploy.mdx new file mode 100644 index 000000000..93bf669f7 --- /dev/null +++ b/docs/evaluations/deploy.mdx @@ -0,0 +1,55 @@ +--- +title: "Deploy and version an evaluation" +description: "Deploy an immutable version, see what is live, publish new versions, roll back, and score sessions you already have." +icon: "cloud-upload" +--- + +## Deploy it + +Select **deploy `@`** at the bottom of the authoring page. The version is immutable once published: from then on, every session that finishes, and that its condition applies to, is scored by it. + +## See what is live + +**Analyze → eval authoring** lists your organization's hosted definitions, the evaluations the managed evaluator runs for it. Each row shows: + +- its name, key, version, and result type +- its source checksum, which tells deployed revisions apart without opening the code +- whether it is **conditional** or runs on **all completed sessions** — the condition is what scopes an evaluation to particular agents or environments +- its timeout, its labels, and when it last changed + +![The hosted definitions list: each evaluation's name, key, version, result type, checksum, timeout, and scope, with new version and enable or disable.](/images/dashboard/eval-definitions.png) + +Search the list, or filter it by state. Evaluations your own worker registers are not listed here; their results carry a **customer** tag on the [evaluations page](/sessions/evaluations), and hosted ones carry **managed**. + +An organization can have up to 100 different hosted evaluations enabled at once. + +## Publish a new version + +Select **new version** on a row. The authoring page opens with that version's code; change it, test it, and deploy it. Its key and result type carry over and cannot change. + +Publishing a successor disables its predecessor and keeps it on the list. Results keep the version that produced them, so a chart shows exactly when the new logic took over. + +## Roll back + +Select **disable** on the current version and **enable** on the one you want back. Nothing is deleted, and every result stays as it was. + +## Stop an evaluation + +Select **disable**. With no enabled version, it stops running on new sessions. To stop an evaluation your own worker runs, stop registering it: remove it from the worker, or stop the worker. + +## Score sessions you already have + +Evaluation runs forward: a version deployed now never scores a session that ended before it. To score history, open **score sessions you already have** on the eval authoring page, choose a window of up to 90 days and, optionally, a single evaluation, and count before you run. The count is exactly what will run, and each session-and-evaluation pair in it is a billable evaluation. + +It fills gaps only. A session that already has a result for that evaluation keeps it, and running the same window twice scores nothing new. + +To score one session again — after a fix, or for a session that never ended cleanly — select **re-evaluate** on its page. The new result is added to the session's history; earlier ones stay. + +## Permissions + +| Permission | Lets you | +| --- | --- | +| `evaluations:read` | See results, and open the eval authoring page | +| `evaluations:trigger` | See, deploy, version, enable, and disable hosted definitions; test them; score history; re-evaluate a session | +| `events:read` | Test against real sessions, and ground drafts in your payload keys, on top of `evaluations:trigger` | +| `evaluations:run` | Run your own evaluator worker | diff --git a/docs/evaluations/overview.mdx b/docs/evaluations/overview.mdx new file mode 100644 index 000000000..14ac945f6 --- /dev/null +++ b/docs/evaluations/overview.mdx @@ -0,0 +1,44 @@ +--- +title: "Evaluate agents" +description: "Score every finished session with evaluations you define: hosted Python checks, or LLM judges in your own worker." +icon: "gauge" +--- + +An evaluation scores a finished agent session. When a session ends, every enabled evaluation that applies to it runs and records what it found, with reasoning you can read beside the trace: + +- a **score** from 0 to 1, optionally marked passed or failed +- a **metric**, such as a count, a duration, or a cost, with its unit +- an **assertion**, which passed or did not + +## Two kinds of evaluator + +| | Hosted Python | Your own worker | +| --- | --- | --- | +| Written | In the dashboard, under **Analyze → eval authoring** | In Python, with the [Evaluator SDK](/reference/evaluator-sdk) | +| Runs | On Failproof AI's managed evaluator, in a sandbox | On your infrastructure | +| Best for | Deterministic, code-based checks | LLM judges, model calls, packages, secrets, network access, heavy processing | + +Hosted Python is deliberately small: one expression, no imports, no network. Anything that needs a model — an LLM judge scoring whether an answer was relevant, say — runs in your own worker instead. Neither kind needs an inbound connection: workers claim finished sessions and submit results over outbound HTTPS. + +## Each organization evaluates its own agents + +Evaluations belong to the organization that defines them. Each organization on an instance writes its own — its own checks, conditions, thresholds, and labels — versions and deploys them without affecting any other, and sees only its own results. Filter those results by agent, environment, evaluation, and time, or ask the assistant about them. + +## From first draft to live scores + + + + Describe what to measure and let the assistant draft it, or write it yourself. See [Write an evaluation](/evaluations/write). + + + Run it against real sessions before it goes live; nothing is stored. See [Test an evaluation](/evaluations/test). + + + Deploy an immutable version, publish new ones as it evolves, and roll back to an earlier one. See [Deploy and version](/evaluations/deploy). + + + Chart scores over time, compare agents and environments, and ask the assistant. See [Read evaluation results](/sessions/evaluations). + + + +Evaluation runs forward: a version deployed now scores the sessions that finish from now on. To score sessions you already have, [backfill them](/evaluations/deploy#score-sessions-you-already-have). diff --git a/docs/evaluations/test.mdx b/docs/evaluations/test.mdx new file mode 100644 index 000000000..3649f587e --- /dev/null +++ b/docs/evaluations/test.mdx @@ -0,0 +1,29 @@ +--- +title: "Test an evaluation" +description: "Run an evaluation against your real sessions before deploying it. Nothing is stored." +icon: "flask-conical" +--- + +**test this evaluation**, on the authoring page, runs the code against real sessions of yours on the evaluator fleet without deploying it. Nothing is stored: a failure here is a preview, and deploying is always allowed. + + + + Select **check** to compile the code and condition against the sandbox's rules without running them on any session. + + + Narrow the matching sessions by agent, environment, time, or session id, and tick up to 10. Include sessions the evaluation should fail as well as ones it should pass. + + + Select **run against N sessions**, and read each row. + + + +| Row | What it means | +| --- | --- | +| **ok** | It ran. The row lists every score, metric, and assertion it returned, and how long it took. | +| **skipped** | The condition returned `False`, so the evaluation did not run. That is a skip, not a failure. | +| Failed | It raised, timed out, or used something the sandbox refuses. The row says which, and **Fix it** hands the error to the assistant when it can help. | + +![The test this evaluation panel: three sessions picked by agent, two ok and one skipped because its condition returned False.](/images/dashboard/eval-test.png) + +A result stops being current the moment you edit the code; it is dimmed rather than reused. diff --git a/docs/evaluations/write.mdx b/docs/evaluations/write.mdx new file mode 100644 index 000000000..03a5c8d8e --- /dev/null +++ b/docs/evaluations/write.mdx @@ -0,0 +1,76 @@ +--- +title: "Write an evaluation" +description: "Describe what to measure and let the assistant draft a hosted Python evaluation, or write the code yourself. LLM judges run in your own worker." +icon: "file-pen-line" +--- + +Hosted evaluations are small, deterministic Python, written in the dashboard and run on Failproof AI's evaluator fleet. Heavier logic — an LLM judge, a package, a secret, a network call — runs in [your own worker](#write-it-in-your-own-worker) instead. + +## Draft it from a description + +1. Go to **Analyze → eval authoring** and select **new eval**. +2. Describe what to measure in plain English, or choose from **start from an example…**, and select **draft**. +3. Review the fields and the code it fills in, then [test it](/evaluations/test) and [deploy it](/evaluations/deploy). + +![The eval authoring page with a drafted evaluation: the description, the assistant's notes on the draft, and the name, key, version, result, timeout, labels, and condition fields.](/images/dashboard/eval-authoring-draft.png) + +The draft is grounded in your organization's own events: the page reads which payload keys your sessions carried over the last seven days, so the code reads keys that exist rather than guesses. Before handing the draft over, the assistant tests it against up to five of your recent sessions, repairs anything it can prove is broken — for up to three rounds — and checks once that the code measures what you asked for. Keep the description specific: broad prompts are slower and can time out. Review the code either way; deploying is never blocked. + +## Set the fields + +| Field | What it is | +| --- | --- | +| name | What people see. Editable later | +| key | The stable identifier its results chart under, such as `code_assistant_quality_gate` | +| version | Any version string without spaces, such as `1.0.0` | +| result | **score** (0 to 1), **metric** (a number with a unit), or **assertion** (passed or not) | +| timeout seconds | Default 30. The sandbox stops any single run at 60 | +| labels | Up to 20, comma-separated. Editable later | +| condition | Optional. A Python expression; the evaluation runs only on sessions where it is `True` | + +Use the condition to scope an evaluation to the agents and environments it is meant for: + +```python +session.agent_id == "code-assistant" and session.environment == "production" +``` + +The key, version, result type, condition, and code are immutable once deployed: to change any of them, publish a new version. The name, labels, and whether it is enabled stay editable. + +## Write the code yourself + +The **evaluator code** is one Python expression that returns `EvalResult(...)`, with `session` in scope. This one scores the share of tool results that came back ok: + +```python +EvalResult( + score=Score( + len([e for e in session.events_of_type("tool_result") if e.payload.get("status") == "ok"]) + / max(1, session.count("tool_result")) + ), + metrics={"tool_calls": Metric(session.count("tool_use"), unit="calls")}, + reasoning="Share of tool results that came back ok.", +) +``` + +A result leads with the evaluation's own key, in its declared type: `score=` for a score evaluation, or a `metrics` or `assertions` entry named after the key for a metric or an assertion evaluation. Other metrics and assertions ride along with it, up to 25 results in a run. + +| In scope | Gives you | +| --- | --- | +| `session` | `session_id`, `agent_id`, `environment`, `started_at`, `ended_at`, `event_count`, and `events`, plus `count(event_type)` and `events_of_type(event_type)` | +| Each event | `id`, `ts`, `event_type`, and `payload` | +| Result types | `EvalResult`, `Score`, `Metric`, `Assertion`, and `ConditionResult` for a condition | +| Builtins | `abs`, `all`, `any`, `bool`, `dict`, `float`, `int`, `len`, `list`, `max`, `min`, `range`, `round`, `set`, `sorted`, `str`, `sum`, `tuple` | + +Nothing else is reachable: no imports, and no attributes beyond that session data and plain string and dictionary methods such as `get`, `lower`, and `split`, which must be called rather than referenced. Payload keys are whatever your agents send — `status` above is only an example — so read them off a real session. **format** tidies the code and **fix** asks the assistant to repair it. The code can be up to 128 KiB, and the condition up to 16 KiB. + +![The evaluator code editor, with format and fix, showing the assertions of a drafted evaluation.](/images/dashboard/eval-authoring-code.png) + +## Write it in your own worker + +When an evaluation needs a model, a package, a secret, or the network, write it with the [Evaluator SDK](/reference/evaluator-sdk) and run it on your own infrastructure. It uses the same result types, and its results appear beside hosted ones, tagged **customer**: + +```python +@app.eval("answer_relevance", version="judge-v1", labels=["llm_judge"], timeout_seconds=30) +async def answer_relevance(session): + value, reasoning = await ask_judge(session) # your LLM call: a 0-1 score and why + return EvalResult(score=Score(value, passed=value >= 0.7), reasoning=reasoning) +``` diff --git a/docs/fr/policies/builtin-catalog.mdx b/docs/fr/policies/builtin-catalog.mdx deleted file mode 100644 index 9707296f3..000000000 --- a/docs/fr/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "Catalogue des politiques intégrées" -description: "Consultez chaque politique intégrée de Failproof AI, son déclencheur, son état recommandé et ses paramètres configurables." -icon: "list-checks" ---- - -Le package installé fait foi pour la disponibilité des politiques. Exécutez `failproofai policies` après chaque mise à jour, car les entrées du catalogue et leur comportement peuvent évoluer avec la version du package. - -## Référence de base recommandée - -La sélection recommandée lors de la configuration guidée active actuellement les désinfectants de secrets, les protections d'environnement, l'auto-protection, les gardes contre les commandes catastrophiques et la sécurité des branches protégées : - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` est **toujours actif**. Il est listé ci-dessus par souci d'exhaustivité, mais il s'enregistre à chaque évaluation qu'il figure ou non dans votre ensemble activé, et il ne peut pas être désactivé ni mis en pause — un garde contre la désactivation de l'application par l'agent, que l'agent peut lui-même désactiver, n'est pas un garde. - -La recommandation est délibérément plus restreinte que **Tout**. Les politiques d'infrastructure et de workflow peuvent interrompre un travail valide et doivent être activées uniquement pour les dépôts et les machines qui en ont besoin. - -## Secrets et environnement - -| Politique | Déclencheur | Résultat | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | Masque les JWT dans la sortie des outils avant que le modèle ne les voie. | -| `sanitize-api-keys` | `PostToolUse` | Masque les clés OpenAI, Anthropic, GitHub, AWS, Stripe et Google les plus courantes. | -| `sanitize-connection-strings` | `PostToolUse` | Masque les chaînes de connexion à des bases de données contenant des identifiants. | -| `sanitize-private-key-content` | `PostToolUse` | Masque les corps de clés privées PEM. | -| `sanitize-bearer-tokens` | `PostToolUse` | Masque les jetons bearer d'autorisation. | -| `protect-env-vars` | `PreToolUse` sur les outils shell | Bloque les commandes qui affichent les variables d'environnement. | -| `block-env-files` | `PreToolUse` | Bloque les lectures et écritures des fichiers `.env`. | -| `block-read-outside-cwd` | `PreToolUse` sur les outils de lecture, glob, grep ou shell | Limite les lectures au répertoire de travail de la session. | -| `block-secrets-write` | `PreToolUse` sur les outils d'écriture | Bloque les écritures vers les noms de fichiers courants de clés secrètes et d'identifiants. | - -## Commandes dangereuses et infrastructure - -| Politique | Déclencheur | Résultat | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | Bloque `sudo` sauf si un motif d'autorisation correspond. | -| `block-curl-pipe-sh` | `PreToolUse` | Bloque les scripts téléchargés et directement transmis à un shell. | -| `block-rm-rf` | `PreToolUse` | Bloque les motifs de suppression récursive catastrophique. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Toujours actif, ne peut pas être désactivé.** Bloque toute invocation de la CLI Failproof AI, toute mise en pause automatique et toute désinstallation par le gestionnaire de packages. | -| `block-kubectl` | `PreToolUse` | Contrôle les commandes Kubernetes. | -| `block-terraform` | `PreToolUse` | Contrôle les commandes Terraform et OpenTofu. | -| `block-aws-cli` | `PreToolUse` | Contrôle les commandes AWS CLI. | -| `block-gcloud` | `PreToolUse` | Contrôle les commandes Google Cloud CLI. | -| `block-az-cli` | `PreToolUse` | Contrôle les commandes Azure CLI. | -| `block-helm` | `PreToolUse` | Contrôle les commandes Helm. | -| `block-gh-pipeline` | `PreToolUse` | Contrôle les opérations GitHub CLI de workflow, d'exécution, de fusion, de release, de cache et de secrets entraînant des modifications. | - -## Sécurité Git et base de données - -| Politique | Déclencheur | Résultat | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Bloque les poussées directes vers les branches protégées configurées. | -| `block-force-push` | `PreToolUse` | Bloque les force-push ; `--force-with-lease` reste autorisé par l'implémentation actuelle. | -| `block-work-on-main` | `PreToolUse` | Bloque les commits et fusions sur les branches protégées. | -| `warn-git-amend` | `PreToolUse` | Avertit avant de réécrire un commit avec `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | Avertit avant de supprimer ou vider définitivement des stashes. | -| `warn-all-files-staged` | `PreToolUse` | Avertit lors d'un `git add -A`, `git add .` ou `git add --all` général. | -| `warn-destructive-sql` | `PreToolUse` | Avertit lors d'un `DROP`, `TRUNCATE` ou `DELETE` sans clause `WHERE` via des clients de base de données reconnus. | -| `warn-schema-alteration` | `PreToolUse` | Avertit lors d'opérations `ALTER TABLE` reconnues sur des colonnes ou des renommages. | - -## Packages, comportement système et boucles d'agent - -| Politique | Déclencheur | Résultat | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Avertit avant de publier sur des registres de packages. | -| `warn-global-package-install` | `PreToolUse` | Avertit avant l'installation globale d'un package. | -| `prefer-package-manager` | `PreToolUse` | Indique à l'agent d'utiliser un gestionnaire de packages autorisé. | -| `warn-large-file-write` | `PreToolUse` sur les outils d'écriture | Avertit au-dessus du seuil de taille de fichier configuré. | -| `warn-background-process` | `PreToolUse` | Avertit sur les motifs de processus en arrière-plan détachés ou de longue durée. | -| `warn-repeated-tool-calls` | `PreToolUse` | Avertit après trois appels d'outil identiques ou plus. | - -## Workflow de fin de tâche - -Ces politiques nécessitent un harnais qui émet un événement `Stop` compatible. - -| Politique | Résultat | -| --- | --- | -| `require-commit-before-stop` | Refuse la complétion si du travail suivi reste non commité. | -| `require-push-before-stop` | Refuse la complétion si des commits restent uniquement en local. | -| `require-pr-before-stop` | Exige une pull request pour la branche courante. | -| `require-no-conflicts-before-stop` | Exige une fusion propre par rapport à la branche de base configurée. | -| `require-ci-green-before-stop` | Exige que les vérifications CI sur le HEAD courant se terminent avec succès. | - -## Référence des paramètres - -Configurez les paramètres sous l'objet `policyParams` du scope sélectionné. Les types sont validés par chaque politique. - -| Politique | Paramètre | Type et valeur par défaut | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]` ; les entrées contiennent `regex` et `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Bloqueurs d'infrastructure | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"` ; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - Un motif d'autorisation élargit ce qu'un agent peut faire. Testez la tokenisation exacte et les variantes de commandes sur le harnais cible avant de le déployer sur une flotte. - \ No newline at end of file diff --git a/docs/fr/policies/builtin.mdx b/docs/fr/policies/builtin.mdx deleted file mode 100644 index 60c286aeb..000000000 --- a/docs/fr/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Politiques intégrées" -description: "Activez des garde-fous maintenus pour les modes d'échec courants des agents." -icon: "library" ---- - -Les politiques intégrées couvrent la gestion des secrets, les fichiers d'environnement, les commandes shell destructives, les branches protégées, les outils cloud et d'infrastructure, la publication de paquets, les appels répétés et les vérifications de workflow en fin de tâche. - -## Activer et vérifier une politique intégrée - - - - 1. Installez la politique sur une machine connectée avec la CLI locale. - 2. Exécutez une action de test sûre dans l'agent instrumenté. - 3. Accédez à **Observer → politique** et filtrez par nom de politique, environnement machine ou décision. - 4. Ouvrez la session liée pour confirmer l'entrée d'outil correspondante et la raison retournée. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - Supprimez-la avec `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -Listez les politiques disponibles dans votre version installée : - -```bash -failproofai policies -``` - -Activez une politique pour un projet : - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Activez plusieurs politiques pour certains harnais : - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Certaines politiques acceptent des paramètres ou sont marquées comme bêta. Examinez la description, le périmètre de correspondance et le comportement par défaut avant le déploiement. Une politique qui protège un workflow peut bloquer des opérations valides dans un autre. - - - Consultez l'ensemble des 40 politiques actuelles, leurs déclencheurs, la base de référence recommandée et leurs paramètres. - - - - Préférez le périmètre projet pour les attentes spécifiques au dépôt, et le périmètre utilisateur pour les exigences de sécurité à l'échelle de la machine. - \ No newline at end of file diff --git a/docs/fr/policies/custom.mdx b/docs/fr/policies/custom.mdx deleted file mode 100644 index a6e6a0ed3..000000000 --- a/docs/fr/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Politiques personnalisées" -description: "Écrivez une politique pour un mode d'échec propre à votre workflow d'agent." -icon: "shield-plus" ---- - -Créez un fichier se terminant par `policies.js`, `policies.mjs` ou `policies.ts` dans `.failproofai/policies/`. Les fichiers de convention se chargent automatiquement aux portées projet et utilisateur. - -## Tester la politique avant la publication Cloud - - - - 1. Installez la politique personnalisée sur une machine de test et déclenchez à la fois une action correspondante et une non-correspondance légitime. - 2. Rendez-vous dans **Observe → policy** et comparez les deux décisions. - 3. Ouvrez chaque session liée et vérifiez que le payload d'événement contient suffisamment d'informations pour la règle. - 4. Lorsque le comportement est correct, déplacez la source vérifiée dans **Admin → policy editor** et publiez une version. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - Les fichiers de convention dans `.failproofai/policies/` se chargent sans `--custom`. Conservez une commande d'installation explicite en CI lorsque la validation doit échouer sur un module défectueux. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -Ce code correspond à `production/config.yml`, `/srv/production/config.yml`, `/srv/production` et `C:\\production\\config.yml` pour les deux outils `Write` et `Edit`. Il ne correspond pas à des noms tels que `production-backup`, car `production` doit constituer un segment de chemin complet. - -Valider et installer un fichier explicite : - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -Le contexte de la politique inclut le type d'événement, le payload normalisé, le nom et les entrées de l'outil, les métadonnées de session, les paramètres, ainsi que le CLI source lorsqu'il est disponible. - -## Tester les chemins d'échec - -Exécutez la validation après avoir modifié le fichier d'entrée ou tout module local qu'il importe : - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -Le chemin CLI strict échoue en cas de fichiers manquants, d'erreurs de syntaxe, d'imports non résolus, d'exceptions au niveau supérieur et de délais d'expiration au chargement du module. Au moment de l'application, un fichier personnalisé défectueux est journalisé et ignoré afin que les politiques intégrées puissent continuer à fonctionner. Traitez tout avertissement de chargement comme une perte d'application attendue et signalez-le dans les journaux de production. - -Utilisez des noms globalement uniques pour les politiques explicites, de convention et gérées par Cloud. Gardez les fonctions de politique déterministes, encadrez les appels externes avec des délais courts, et retournez un `allow`, `instruct` ou `deny` intentionnel sur chaque chemin d'exécution. - - - Une politique personnalisée est du code d'application. Testez les champs manquants, les noms d'outils alternatifs et les entrées malformées — pas uniquement la correspondance attendue. - \ No newline at end of file diff --git a/docs/fr/policies/fleet.mdx b/docs/fr/policies/fleet.mdx deleted file mode 100644 index 7512ed8e1..000000000 --- a/docs/fr/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Déployer des politiques sur les machines" -description: "Savoir quelles machines sont enrôlées, à jour et appliquent les versions de politique souhaitées." -icon: "network" ---- - -La couverture de parc répond à la question de savoir si une politique existe là où le risque existe. Suivez les machines par identifiant stable et libellé lisible, puis comparez leur état de déploiement assigné et rapporté. - -## Vérifier la couverture - - - - 1. Accédez à **Admin → enforcement** et consultez les totaux en mode application et observation. - 2. Recherchez une machine par identifiant ou libellé, ou filtrez pour trouver les machines sans politique. - 3. Développez une ligne pour comparer les politiques assignées, l'état de déploiement rapporté, le dernier check-in et l'historique. - 4. Actualisez après l'intervalle d'interrogation de la machine lorsqu'un déploiement appliqué reste en attente. - - ![Le parc Enforcement affichant la couverture des politiques, l'état de déploiement des machines, et les assignations en observation et en application.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Utilisez `fp events --agent-id --since 24h` pour confirmer que l'activité de l'agent de la machine est bien transmise au Cloud. - - - -Utilisez les vues de couverture pour identifier : - -- Les machines qui n'ont jamais récupéré le dernier déploiement -- Les machines enrôlées qui ont cessé de rapporter leur activité -- Une politique assignée au mauvais environnement ou cohorte -- Une dérive de version après une mise à jour interrompue - -Renommer une machine sans la reconnecter : - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Vérifier l'état local : - -```bash -failproofai config --status -``` - - - Utilisez des libellés qui identifient la charge de travail et l'environnement. Les noms d'hôtes seuls sont souvent insuffisants après une mise à l'échelle automatique ou un remplacement de machine. - \ No newline at end of file diff --git a/docs/he/admin/overview.mdx b/docs/he/admin/overview.mdx index e549af854..ffe88835f 100644 --- a/docs/he/admin/overview.mdx +++ b/docs/he/admin/overview.mdx @@ -1,5 +1,4 @@ --- ---- title: "ניהול" description: "נהל גישה, שימוש, ארגונים וביטחון ללא עירוב שלהם בזרימת העבודה של אמינות." icon: "settings-2" diff --git a/docs/he/audits/agent-contracts.mdx b/docs/he/audits/agent-contracts.mdx index bb2e66a8c..671c42b06 100644 --- a/docs/he/audits/agent-contracts.mdx +++ b/docs/he/audits/agent-contracts.mdx @@ -1,5 +1,4 @@ --- ---- title: "הקשר סוכן" description: "ספר לביקורות מה כל סוכן חייב לעשות, להפיק, ולעולם לא לעשות." icon: "bot" diff --git a/docs/he/audits/local-audit.mdx b/docs/he/audits/local-audit.mdx index 2e09ecbb4..50eda92f2 100644 --- a/docs/he/audits/local-audit.mdx +++ b/docs/he/audits/local-audit.mdx @@ -1,5 +1,4 @@ --- ---- title: "审核本地代理历史记录" description: "离线扫描受支持的代理 CLI 历史记录，并在本地审查风险或浪费的行为。" icon: "laptop-minimal-check" diff --git a/docs/he/audits/overview.mdx b/docs/he/audits/overview.mdx index 8dc687f4d..c73b7eb66 100644 --- a/docs/he/audits/overview.mdx +++ b/docs/he/audits/overview.mdx @@ -1,5 +1,4 @@ --- ---- title: "ביקורות" description: "בדוק אוכלוסיית פגיעויות מוגדרת לכשלים שעקבות בלבד לא יגידו עליהם." icon: "scan-search" diff --git a/docs/he/audits/recipes.mdx b/docs/he/audits/recipes.mdx index 2dd3dc4a3..5466ac6cc 100644 --- a/docs/he/audits/recipes.mdx +++ b/docs/he/audits/recipes.mdx @@ -1,5 +1,4 @@ --- ---- title: "מתכונים ביקורת" description: "יעדים התחלתיים לחקירות נפוצות של כשלים בסוכנים." icon: "book-open-check" diff --git a/docs/he/index.mdx b/docs/he/index.mdx index ede5a62cb..dbcf5ee98 100644 --- a/docs/he/index.mdx +++ b/docs/he/index.mdx @@ -1,5 +1,4 @@ --- ---- title: "הפוך את הסוכן שלך לבטוח מטעויות" description: "נראות והטלת אכיפה על כל ההרתמה שהסוכנים שלך מריצים - CLIs קידוד, שערי צ'אט, עוזרים עצמאיים והסוכנים המותאמים שלך." icon: "shield-check" diff --git a/docs/he/policies/builtin-catalog.mdx b/docs/he/policies/builtin-catalog.mdx deleted file mode 100644 index f7805ef7c..000000000 --- a/docs/he/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "קטלוג מדיניויות מובנה" -description: "עיין בכל מדיניות Failproof AI מובנה, בטריגר שלה, במצב מומלץ ובפרמטרים הניתנים להגדרה." -icon: "list-checks" ---- - -החבילה המותקנת היא מקור האמת לזמינות מדיניויות. הרץ `failproofai policies` לאחר כל שדרוג כי ערכי הקטלוג וההתנהגות עשויים להשתנות עם גרסת החבילה. - -## קו בסיס מומלץ - -הבחירה המומלצת של ההתקנה המודרכת מפעילה כרגע מטהרי סודות, הגנות סביבתיות, הגנה עצמית, שומרי פקודות קטסטרופליות והגנת ענפים מוגנים: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` **תמיד פועל**. הוא מופיע למעלה למטרות השלמות, אך הוא נרשם בכל הערכה בין אם הוא מופיע בערכה המופעלת שלך או לא, ולא ניתן להשבית אותו או להשהות — הגנה כנגד סוכן המכבה את האכיפה שהסוכן יכול להכבות אינה הגנה. - -המומלץ הוא בכוונה צר יותר מ**הכל**. מדיניויות תשתית וזרימת עבודה יכולות להפריע לעבודה תקינה ויש להפעילן במאגרים ובמכונות הזקוקות להן. - -## סודות וסביבה - -| מדיניות | טריגר | תוצאה | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | הסר JWTs מפלט כלי לפני שהמודל רואה אותם. | -| `sanitize-api-keys` | `PostToolUse` | הסר מפתחות OpenAI, Anthropic, GitHub, AWS, Stripe ו-Google נפוצים. | -| `sanitize-connection-strings` | `PostToolUse` | הסר מחרוזות חיבור מסד נתונים המכילות אישורים. | -| `sanitize-private-key-content` | `PostToolUse` | הסר גופי מפתח פרטי PEM. | -| `sanitize-bearer-tokens` | `PostToolUse` | הסר אסימוני הרשאה של מנשאים. | -| `protect-env-vars` | `PreToolUse` בכלי מעטפת | חסום פקודות שמשחקות משתני סביבה. | -| `block-env-files` | `PreToolUse` | חסום קריאה וכתיבה של קבצי `.env`. | -| `block-read-outside-cwd` | `PreToolUse` בקריאה, glob, grep או כלי מעטפת | שמור קריאות בתוך ספריית העבודה של ההפעלה. | -| `block-secrets-write` | `PreToolUse` בכלי כתיבה | חסום כתיבות לשמות קבצים וסכמות אישור של מפתחות סודיים נפוצות. | - -## פקודות מסוכנות ותשתית - -| מדיניות | טריגר | תוצאה | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | חסום `sudo` אלא אם דפוס הרשאה תואם. | -| `block-curl-pipe-sh` | `PreToolUse` | חסום סקריפטים שהורדו בצינור ישירות למעטפת. | -| `block-rm-rf` | `PreToolUse` | חסום דפוסי מחיקה רקורסיבית קטסטרופליים. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **תמיד פועל, לא ניתן להשבית.** חסום כל הפעלה של CLI של Failproof AI, השהייה עצמית והסרת מנהל חבילות. | -| `block-kubectl` | `PreToolUse` | שער פקודות Kubernetes. | -| `block-terraform` | `PreToolUse` | שער פקודות Terraform ו-OpenTofu. | -| `block-aws-cli` | `PreToolUse` | שער פקודות AWS CLI. | -| `block-gcloud` | `PreToolUse` | שער פקודות Google Cloud CLI. | -| `block-az-cli` | `PreToolUse` | שער פקודות Azure CLI. | -| `block-helm` | `PreToolUse` | שער פקודות Helm. | -| `block-gh-pipeline` | `PreToolUse` | שער פעולות זיהום GitHub CLI של זרימת עבודה, הרצה, מיזוג, שחרור, cache וסוד. | - -## בטיחות Git ומסד נתונים - -| מדיניות | טריגר | תוצאה | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | חסום דחיפות ישירות לענפים מוגנים מוגדרים. | -| `block-force-push` | `PreToolUse` | חסום דחיפות כוח; `--force-with-lease` נשאר מותר ביישום הנוכחי. | -| `block-work-on-main` | `PreToolUse` | חסום commits ומיזוגים בענפים מוגנים. | -| `warn-git-amend` | `PreToolUse` | הזהר לפני כתיבה מחדש של commit עם `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | הזהר לפני השלכה קבועה או ניקוי stashes. | -| `warn-all-files-staged` | `PreToolUse` | הזהר בברור `git add -A`, `git add .` או `git add --all`. | -| `warn-destructive-sql` | `PreToolUse` | הזהר בנוגע ל-`DROP`, `TRUNCATE` ו-`DELETE` ללא `WHERE` דרך לקוחות מסד נתונים מוכרים. | -| `warn-schema-alteration` | `PreToolUse` | הזהר בנוגע ל-`ALTER TABLE` הכרה ופעולות שינוי שם. | - -## חבילות, התנהגות מערכת וולולים של סוכנים - -| מדיניות | טריגר | תוצאה | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | הזהר לפני פרסום לרושמי חבילות. | -| `warn-global-package-install` | `PreToolUse` | הזהר לפני התקנת חבילה גלובלית. | -| `prefer-package-manager` | `PreToolUse` | הנחה את הסוכן להשתמש במנהל חבילות מותר. | -| `warn-large-file-write` | `PreToolUse` בכלי כתיבה | הזהר מעל סף גודל הקובץ המוגדר. | -| `warn-background-process` | `PreToolUse` | הזהר בדפוסי תהליכים רקע מופרדים או ארוכי חיים. | -| `warn-repeated-tool-calls` | `PreToolUse` | הזהר לאחר שלוש או יותר קריאות כלי זהות. | - -## זרימת עבודה של סיום משימה - -מדיניויות אלה דורשות כרוכה הפולטת אירוע `Stop` תואם. - -| מדיניות | תוצאה | -| --- | --- | -| `require-commit-before-stop` | סרב השלמה בעודם עבודה עקובה לא מחוייבת. | -| `require-push-before-stop` | סרב השלמה בעוד commits נשארות מקומיות בלבד. | -| `require-pr-before-stop` | דרוש בקשת משיכה לענף הנוכחי. | -| `require-no-conflicts-before-stop` | דרוש מיזוג נקי כנגד ענף בסיס מוגדר. | -| `require-ci-green-before-stop` | דרוש בדיקות CI של current-HEAD להשלמה בהצלחה. | - -## ייחוס פרמטרים - -הגדר פרמטרים תחת אובייקט `policyParams` של ההיקף שנבחר. סוגים מאומתים על ידי כל מדיניות. - -| מדיניות | פרמטר | סוג וברירת מחדל | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; ערכים מכילים `regex` ו-`label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| חוסמי תשתית | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - דפוס הרשאה מרחיב את מה שסוכן עשוי לעשות. בדוק את הטוקניזציה המדויקת וגרסאות פקודות בכרוכה היעד לפני פריסה שלה על פני צי. - \ No newline at end of file diff --git a/docs/he/policies/builtin.mdx b/docs/he/policies/builtin.mdx deleted file mode 100644 index 95fa65e7f..000000000 --- a/docs/he/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "מדיניות מובנות" -description: "הפעל הגנות תחזוקה עבור מצבי כשל נפוצים של סוכנים." -icon: "library" ---- - -מדיניות מובנות כוללת טיפול בסודות, קבצי סביבה, פקודות shell הרסניות, ענפים מוגנים, כלים בענן ותשתיות, הוצאת חבילות, קריאות חוזרות, ובדיקות זרימת עבודה של סיום משימה. - -## הפעלה ואימות של מדיניות מובנית - - - - 1. התקן את המדיניות על מכונה מחוברת עם CLI המקומי. - 2. הפעל פעולת בדיקה בטוחה בסוכן המאומת. - 3. עבור אל **Observe → policy** וסנן לפי שם המדיניות, סביבת המכונה או ההחלטה. - 4. פתח את ההפעלה המקושרת כדי לאשר את קלט הכלי התואם והסיבה שהוחזרה. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - הסר אותה עם `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -הצג את המדיניות הזמינות בגרסה המותקנת שלך: - -```bash -failproofai policies -``` - -הפעל מדיניות אחת עבור פרויקט: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -הפעל מספר מדיניות עבור מכלים נבחרים: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -חלק מהמדיניות מקבלות פרמטרים או מסומנות כבטא. בדוק את התיאור, התאם את ההיקף וההתנהגות המוגדרת כברירת מחדל לפני הנפוץ. מדיניות המגנה על זרימת עבודה אחת עשויה לחסום פעולות חוקיות בזרימה אחרת. - - - בדוק את כל 40 המדיניות הנוכחיות, הטריגרים שלהן, קו בסיס מומלץ ופרמטרים. - - - - העדף היקף פרויקט לציפיות ספציפיות למאגר וקו היקף משתמש לדרישות בטיחות כלל-מכונה. - \ No newline at end of file diff --git a/docs/he/policies/custom.mdx b/docs/he/policies/custom.mdx deleted file mode 100644 index 7735b35c4..000000000 --- a/docs/he/policies/custom.mdx +++ /dev/null @@ -1,73 +0,0 @@ ---- ---- -title: "מדיניויות מותאמות" -description: "כתוב מדיניות עבור מצב כשל ייחודי לזרימת העבודה של הסוכן שלך." -icon: "shield-plus" ---- - -צור קובץ בסיום `policies.js`, `policies.mjs`, או `policies.ts` תחת `.failproofai/policies/`. קובצי קונוונציה נטענים באופן אוטומטי בהיקף פרויקט וסטטוס משתמש. - -## בדוק את המדיניות לפני פרסום בענן - - - - 1. התקן את המדיניות המותאמת על מכונת בדיקה אחת וערוך הן פעולה תואמת והן אי-התאמה לגיטימית. - 2. עבור אל **Observe → policy** והשווה בין שתי ההחלטות. - 3. פתח כל סשן קשור וודא שמטען האירוע מכיל מספיק הוכחות לכלל. - 4. כאשר ההתנהגות נכונה, העבר את הקוד הנבדק אל **Admin → policy editor** ופרסם גרסה. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - קובצי קונוונציה תחת `.failproofai/policies/` נטענים ללא `--custom`. שמור על פקודת התקנה מפורשת ב-CI כאשר אימות צריך להיכשל על מודול שבור. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -זה תואם ל-`production/config.yml`, `/srv/production/config.yml`, `/srv/production`, ו-`C:\\production\\config.yml` עבור `Write` ו-`Edit`. זה אינו תואם לשמות כגון `production-backup` מכיוון ש-`production` חייב להיות קטע נתיב שלם. - -אמת והתקן קובץ מפורש: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -הקשר של המדיניות כולל את סוג האירוע, מטען מנורמל, שם וקלט הכלי, מטא-נתונים של סשן, פרמטרים, ו-CLI מקור כאשר זמין. - -## בדוק נתיבי כשל - -הפעל אימות לאחר שינוי קובץ הכניסה או כל מודול מקומי שהוא מייבא: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -נתיב ה-CLI הקפדני נכשל עבור קובצים חסרים, שגיאות תחביר, יבוא שלא פתור, חריגים ברמה עליונה וזמנים פגומים בטעינת מודול. בזמן אכיפה, קובץ מותאם שבור מופעל ודלג כך שמדיניויות מובנות יכולות להמשיך. התייחס לכל אזהרת טעינה כאובדן אכיפה צפויה והתריע על כך בתיקיות ייצור. - -השתמש בשמות ייחודיים ברחבי העולם בין מדיניויות מפורשות, קונוונציה וגם מנוהלות בענן. שמור על פונקציות מדיניות דטרמיניסטיות, כבול קריאות חיצוניות עם זמנים פגומים קצרים, והחזר `allow`, `instruct`, או `deny` בכוונה בכל נתיב. - - - מדיניות מותאמת היא קוד אכיפה. בדוק שדות חסרים, שמות כלים חלופיים, וקלט שגוי—לא רק ההתאמה הצפויה. - \ No newline at end of file diff --git a/docs/he/policies/fleet.mdx b/docs/he/policies/fleet.mdx deleted file mode 100644 index 609f2893e..000000000 --- a/docs/he/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "פרוס מדיניות למכונות" -description: "דע אילו מכונות רשומות, עדכניות ואוכפות את גרסאות המדיניות המכוונות." -icon: "network" ---- - -כיסוי הצי משקף האם מדיניות קיימת במקום שבו קיימת הסיכון. עקוב אחר מכונות לפי מזהה יציב ותווית קריאה, ואז השווה את מצב הפריסה שלהן שהוקצה והדווח. - -## בדוק כיסוי - - - - 1. עבור אל **Admin → enforcement** וסקור את הסכומים של אוכפים ומשקיפים. - 2. חפש מכונה לפי מזהה או תווית, או סנן מכונות שחסרה להן מדיניות. - 3. הרחב שורה כדי להשוות בין מדיניות מוקצה, פריסה דווחת, בדיקה אחרונה והיסטוריה. - 4. רענן לאחר מרווח הסקר של המכונה כאשר פריסה מיושמת נשארת בתלייה. - - ![צי enforcement המציג כיסוי מדיניות, מצב פריסה של מכונה והקצאות צפייה ואכיפה.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - השתמש ב-`fp events --agent-id --since 24h` כדי לאשר שפעילות סוכן המכונה מגיעה ל-Cloud. - - - -השתמש בתצוגות כיסוי כדי למצוא: - -- מכונות שלעולם לא משכו את הפריסה העדכנית ביותר -- מכונות רשומות שהפסיקו לדווח על פעילות -- מדיניות שהוקצתה לסביבה או קוהורטה לא נכונה -- סחיפת גרסה לאחר עדכון מופרע - -שנה שם של מכונה ללא התחברות מחדש: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -בדוק מצב מקומי: - -```bash -failproofai config --status -``` - - - השתמש בתויות המזהות עומס עבודה וסביבה. שמות מארחים לבדם לעתים קרובות אינם מספיקים לאחר שינוי גודל אוטומטי או החלפת מכונה. - \ No newline at end of file diff --git a/docs/he/reference/self-hosting.mdx b/docs/he/reference/self-hosting.mdx index 4f0b72ed5..1df6da179 100644 --- a/docs/he/reference/self-hosting.mdx +++ b/docs/he/reference/self-hosting.mdx @@ -1,5 +1,4 @@ --- ---- title: "Self-host Failproof AI Cloud" description: "פרוס את מישור הבקרה של Failproof AI על קלסטר Kubernetes המנוהל על ידי הלקוח." icon: "cloud-cog" diff --git a/docs/he/sessions/evaluations.mdx b/docs/he/sessions/evaluations.mdx index 328d60cb1..2b37f70d3 100644 --- a/docs/he/sessions/evaluations.mdx +++ b/docs/he/sessions/evaluations.mdx @@ -1,5 +1,4 @@ --- ---- title: "הערכות מקוונות" description: "דירוג הפעלות חיות ושהסתיימו לאיכות, ציות, עלות ותופעת השהיה." icon: "gauge" diff --git a/docs/he/sessions/live-events.mdx b/docs/he/sessions/live-events.mdx index 5e24741da..59b9c5ea1 100644 --- a/docs/he/sessions/live-events.mdx +++ b/docs/he/sessions/live-events.mdx @@ -1,5 +1,4 @@ --- ---- title: "אירועים בזמן אמת" description: "צפו בפעילות הסוכן בזמן ביצוע הסשן." icon: "radio" diff --git a/docs/he/sessions/models.mdx b/docs/he/sessions/models.mdx index 72f10ff2d..94666771c 100644 --- a/docs/he/sessions/models.mdx +++ b/docs/he/sessions/models.mdx @@ -1,5 +1,4 @@ --- ---- title: "Models" description: "השווה בין latency של מודלים, tokens, שימוש בקונטקסט, והפצת תעבורה." icon: "cpu" diff --git a/docs/he/start/integrations.mdx b/docs/he/start/integrations.mdx index f88a2c9fe..3c67d164c 100644 --- a/docs/he/start/integrations.mdx +++ b/docs/he/start/integrations.mdx @@ -1,5 +1,4 @@ --- ---- title: "הוספת כלים לסוכן שלך" sidebarTitle: "Frameworks" description: "חבר כל framework סוכן נתמך ל-Failproof AI בקריאה אחת." diff --git a/docs/he/start/integrations/crewai.mdx b/docs/he/start/integrations/crewai.mdx index c58033a42..b7d5b367d 100644 --- a/docs/he/start/integrations/crewai.mdx +++ b/docs/he/start/integrations/crewai.mdx @@ -1,5 +1,4 @@ --- ---- title: "CrewAI" sidebarTitle: "CrewAI" description: "ניהול צוות, זרימות, סוכנים לפי תפקיד, כלים, זיכרון ומשוב אנושי." diff --git a/docs/he/start/integrations/llamaindex.mdx b/docs/he/start/integrations/llamaindex.mdx index 267cc8c80..e36fb1d6b 100644 --- a/docs/he/start/integrations/llamaindex.mdx +++ b/docs/he/start/integrations/llamaindex.mdx @@ -1,5 +1,4 @@ --- ---- title: "LlamaIndex" sidebarTitle: "LlamaIndex" description: "Instrument workflows, steps, function agents, and retrievers." diff --git a/docs/he/start/integrations/pydantic-ai.mdx b/docs/he/start/integrations/pydantic-ai.mdx index 685b36d59..ca0aced10 100644 --- a/docs/he/start/integrations/pydantic-ai.mdx +++ b/docs/he/start/integrations/pydantic-ai.mdx @@ -1,5 +1,4 @@ --- ---- title: "Pydantic AI" sidebarTitle: "Pydantic AI" description: "הגן על אג'נטים מוקלדים, כלים, קריאות מודל וניסיונות חוזרים." diff --git a/docs/hi/admin/keys-and-permissions.mdx b/docs/hi/admin/keys-and-permissions.mdx index 405a74a4b..ba9fd66d5 100644 --- a/docs/hi/admin/keys-and-permissions.mdx +++ b/docs/hi/admin/keys-and-permissions.mdx @@ -1,5 +1,4 @@ --- ---- title: "कुंजियाँ और अनुमतियाँ" description: "मशीनों, स्वचालन और ऑपरेटरों के लिए स्कोप की गई API कुंजियाँ बनाएँ।" icon: "key-round" diff --git a/docs/hi/policies/builtin-catalog.mdx b/docs/hi/policies/builtin-catalog.mdx deleted file mode 100644 index 130c3610b..000000000 --- a/docs/hi/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "बिल्ट-इन पॉलिसी कैटलॉग" -description: "हर बिल्ट-इन Failproof AI पॉलिसी, इसके ट्रिगर, अनुशंसित स्थिति और कॉन्फ़िगर करने योग्य पैरामीटर की समीक्षा करें।" -icon: "list-checks" ---- - -इंस्टॉल किया गया पैकेज पॉलिसी उपलब्धता के लिए सत्य का स्रोत है। हर अपग्रेड के बाद `failproofai policies` चलाएं क्योंकि कैटलॉग प्रविष्टियां और व्यवहार पैकेज संस्करण के साथ बदल सकते हैं। - -## अनुशंसित बेसलाइन - -गाइडेड सेटअप का अनुशंसित चयन वर्तमान में सीक्रेट सैनिटाइज़र, पर्यावरण सुरक्षा, स्व-सुरक्षा, विनाशकारी-कमांड गार्ड और संरक्षित-शाखा सुरक्षा सक्षम करता है: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` **हमेशा चालू रहता है**। यह पूर्णता के लिए ऊपर सूचीबद्ध है, लेकिन यह हर मूल्यांकन पर पंजीकृत होता है चाहे वह आपके सक्षम सेट में दिखाई दे या नहीं, और इसे अक्षम या रोका नहीं जा सकता — यह एजेंट को प्रवर्तन बंद करने से बचाने के लिए एक सुरक्षा है जिसे एजेंट बंद कर सकता है एक गार्ड नहीं है। - -अनुशंसित जानबूझकर **Everything** से संकीर्ण है। बुनियादी ढांचे और वर्कफ़्लो पॉलिसीएं वैध कार्य को बाधित कर सकती हैं और उन्हें उन रिपॉजिटरी और मशीनों के लिए सक्षम किया जाना चाहिए जिन्हें उनकी आवश्यकता है। - -## सीक्रेट और पर्यावरण - -| पॉलिसी | ट्रिगर | परिणाम | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | मॉडल द्वारा देखे जाने से पहले टूल आउटपुट से JWTs को हटाएं। | -| `sanitize-api-keys` | `PostToolUse` | सामान्य OpenAI, Anthropic, GitHub, AWS, Stripe और Google कुंजियों को हटाएं। | -| `sanitize-connection-strings` | `PostToolUse` | क्रेडेंशियल युक्त डेटाबेस कनेक्शन स्ट्रिंग को हटाएं। | -| `sanitize-private-key-content` | `PostToolUse` | PEM प्राइवेट-की निकायों को हटाएं। | -| `sanitize-bearer-tokens` | `PostToolUse` | प्राधिकरण बेयरर टोकन को हटाएं। | -| `protect-env-vars` | शेल टूल पर `PreToolUse` | पर्यावरण चर डंप करने वाली कमांड को ब्लॉक करें। | -| `block-env-files` | `PreToolUse` | `.env` फाइलों को पढ़ने और लिखने को ब्लॉक करें। | -| `block-read-outside-cwd` | पढ़ने, glob, grep या शेल टूल पर `PreToolUse` | सेशन वर्किंग डायरेक्टरी के अंदर पढ़ने को रखें। | -| `block-secrets-write` | लिखने वाले टूल पर `PreToolUse` | सामान्य सीक्रेट-की और क्रेडेंशियल फाइलनामों को लिखने को ब्लॉक करें। | - -## खतरनाक कमांड और बुनियादी ढांचा - -| पॉलिसी | ट्रिगर | परिणाम | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | जब तक allow पैटर्न मेल न खाए तब तक `sudo` को ब्लॉक करें। | -| `block-curl-pipe-sh` | `PreToolUse` | डाउनलोड की गई स्क्रिप्ट को सीधे शेल में पाइप करने को ब्लॉक करें। | -| `block-rm-rf` | `PreToolUse` | विनाशकारी रिकर्सिव डिलीशन पैटर्न को ब्लॉक करें। | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **हमेशा चालू, अक्षम नहीं किया जा सकता।** हर Failproof AI CLI आह्वान, स्व-रुकावट और पैकेज-मैनेजर अनइंस्टॉल को ब्लॉक करें। | -| `block-kubectl` | `PreToolUse` | Kubernetes कमांड को गेट करें। | -| `block-terraform` | `PreToolUse` | Terraform और OpenTofu कमांड को गेट करें। | -| `block-aws-cli` | `PreToolUse` | AWS CLI कमांड को गेट करें। | -| `block-gcloud` | `PreToolUse` | Google Cloud CLI कमांड को गेट करें। | -| `block-az-cli` | `PreToolUse` | Azure CLI कमांड को गेट करें। | -| `block-helm` | `PreToolUse` | Helm कमांड को गेट करें। | -| `block-gh-pipeline` | `PreToolUse` | उत्परिवर्तनशील GitHub CLI वर्कफ़्लो, रन, मर्ज, रिलीज़, कैश और सीक्रेट ऑपरेशन को गेट करें। | - -## Git और डेटाबेस सुरक्षा - -| पॉलिसी | ट्रिगर | परिणाम | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | कॉन्फ़िगर की गई संरक्षित शाखाओं को सीधे पुश करने को ब्लॉक करें। | -| `block-force-push` | `PreToolUse` | फोर्स-पुश को ब्लॉक करें; `--force-with-lease` वर्तमान कार्यान्वयन द्वारा अनुमति दी गई रहती है। | -| `block-work-on-main` | `PreToolUse` | संरक्षित शाखाओं पर कमिट और मर्ज को ब्लॉक करें। | -| `warn-git-amend` | `PreToolUse` | `--amend` से कमिट को फिर से लिखने से पहले चेतावनी दें। | -| `warn-git-stash-drop` | `PreToolUse` | स्टैश को स्थायी रूप से ड्रॉप या क्लियर करने से पहले चेतावनी दें। | -| `warn-all-files-staged` | `PreToolUse` | व्यापक `git add -A`, `git add .` या `git add --all` पर चेतावनी दें। | -| `warn-destructive-sql` | `PreToolUse` | मान्यता प्राप्त डेटाबेस क्लाइंट के माध्यम से `WHERE` बिना `DROP`, `TRUNCATE` और `DELETE` पर चेतावनी दें। | -| `warn-schema-alteration` | `PreToolUse` | मान्यता प्राप्त `ALTER TABLE` कॉलम और नाम परिवर्तन ऑपरेशन पर चेतावनी दें। | - -## पैकेज, सिस्टम व्यवहार और एजेंट लूप - -| पॉलिसी | ट्रिगर | परिणाम | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | पैकेज रजिस्ट्रीज़ में प्रकाशित करने से पहले चेतावनी दें। | -| `warn-global-package-install` | `PreToolUse` | ग्लोबल पैकेज इंस्टॉलेशन से पहले चेतावनी दें। | -| `prefer-package-manager` | `PreToolUse` | एजेंट को अनुमत पैकेज मैनेजर का उपयोग करने का निर्देश दें। | -| `warn-large-file-write` | लिखने वाले टूल पर `PreToolUse` | कॉन्फ़िगर की गई फाइल-आकार थ्रेसहोल्ड से ऊपर चेतावनी दें। | -| `warn-background-process` | `PreToolUse` | अलग किए गए या दीर्घकालीन बैकग्राउंड-प्रक्रिया पैटर्न पर चेतावनी दें। | -| `warn-repeated-tool-calls` | `PreToolUse` | तीन या अधिक समान टूल कॉल के बाद चेतावनी दें। | - -## कार्य-के-अंत वर्कफ़्लो - -ये पॉलिसीएं एक हार्नेस की आवश्यकता है जो एक संगत `Stop` इवेंट भेजता है। - -| पॉलिसी | परिणाम | -| --- | --- | -| `require-commit-before-stop` | जब तक ट्रैक किया गया कार्य अनकमिटेड है तब तक समापन को अस्वीकार करें। | -| `require-push-before-stop` | जब तक कमिट लोकल-ओनली रहें तब तक समापन को अस्वीकार करें। | -| `require-pr-before-stop` | वर्तमान शाखा के लिए एक पुल रिक्वेस्ट की आवश्यकता करें। | -| `require-no-conflicts-before-stop` | कॉन्फ़िगर की गई बेस शाखा के विरुद्ध एक स्वच्छ मर्ज की आवश्यकता करें। | -| `require-ci-green-before-stop` | वर्तमान-HEAD CI चेक को सफलतापूर्वक पूरा करने की आवश्यकता करें। | - -## पैरामीटर संदर्भ - -चयनित स्कोप के `policyParams` ऑब्जेक्ट के तहत पैरामीटर कॉन्फ़िगर करें। प्रकारों को प्रत्येक पॉलिसी द्वारा मान्य किया जाता है। - -| पॉलिसी | पैरामीटर | प्रकार और डिफ़ॉल्ट | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; प्रविष्टियों में `regex` और `label` हैं | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| बुनियादी ढांचा ब्लॉकर | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - एक allow पैटर्न एजेंट को जो कर सकता है उसे व्यापक बनाता है। तैनाती से पहले सटीक टोकनाइज़ेशन और कमांड वेरिएंट को लक्ष्य हार्नेस पर परीक्षण करें। - \ No newline at end of file diff --git a/docs/hi/policies/builtin.mdx b/docs/hi/policies/builtin.mdx deleted file mode 100644 index 9e67d5ca1..000000000 --- a/docs/hi/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "अंतर्निर्मित नीतियाँ" -description: "एजेंट की सामान्य विफलता के तरीकों के लिए रक्षा उपाय सक्षम करें।" -icon: "library" ---- - -अंतर्निर्मित नीतियाँ गुप्त प्रबंधन, पर्यावरण फ़ाइलें, विनाशकारी शेल कमांड, सुरक्षित शाखाएँ, क्लाउड और बुनियादी ढाँचा उपकरण, पैकेज प्रकाशन, दोहराई गई कॉल, और कार्य-समाप्ति वर्कफ़्लो जांच को कवर करती हैं। - -## अंतर्निर्मित नीति को सक्षम और सत्यापित करें - - - - 1. स्थानीय CLI के साथ जुड़ी हुई मशीन पर नीति को स्थापित करें। - 2. instrumented एजेंट में एक सुरक्षित परीक्षण क्रिया चलाएँ। - 3. **Observe → policy** पर जाएँ और नीति के नाम, मशीन पर्यावरण, या निर्णय के आधार पर फ़िल्टर करें। - 4. जुड़ा हुआ सत्र खोलें ताकि मिलान किए गए टूल इनपुट और दिया गया कारण की पुष्टि की जा सके। - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - इसे `failproofai policy remove block-rm-rf --cli claude --scope project` के साथ हटाएँ। - - - -आपके स्थापित संस्करण में उपलब्ध नीतियों की सूची: - -```bash -failproofai policies -``` - -किसी प्रोजेक्ट के लिए एक नीति सक्षम करें: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -चयनित harness के लिए कई नीतियाँ सक्षम करें: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -कुछ नीतियाँ मापदंड स्वीकार करती हैं या बीटा चिह्नित हैं। rollout से पहले विवरण, मेल scope, और डिफ़ॉल्ट व्यवहार की समीक्षा करें। एक नीति जो एक वर्कफ़्लो की सुरक्षा करती है, दूसरे में वैध संचालन को अवरुद्ध कर सकती है। - - - सभी 40 वर्तमान नीतियों, उनके ट्रिगर, अनुशंसित आधारभूत, और मापदंडों की समीक्षा करें। - - - - रिपॉजिटरी-विशिष्ट अपेक्षाओं के लिए प्रोजेक्ट scope को प्राथमिकता दें और मशीन-व्यापी सुरक्षा आवश्यकताओं के लिए उपयोगकर्ता scope को प्राथमिकता दें। - \ No newline at end of file diff --git a/docs/hi/policies/custom.mdx b/docs/hi/policies/custom.mdx deleted file mode 100644 index 7c04250fe..000000000 --- a/docs/hi/policies/custom.mdx +++ /dev/null @@ -1,73 +0,0 @@ ---- ---- -title: "कस्टम नीतियां" -description: "अपने agent वर्कफ़्लो के लिए अद्वितीय विफलता मोड के लिए एक नीति लिखें।" -icon: "shield-plus" ---- - -`.failproofai/policies/` के तहत `policies.js`, `policies.mjs`, या `policies.ts` में समाप्त होने वाली फ़ाइल बनाएं। कन्वेंशन फ़ाइलें परियोजना और उपयोगकर्ता स्कोप पर स्वचालित रूप से लोड होती हैं। - -## Cloud प्रकाशन से पहले नीति का परीक्षण करें - - - - 1. कस्टम नीति को एक परीक्षण मशीन पर स्थापित करें और एक मेल खाने वाली क्रिया और एक वैध गैर-मिलान दोनों को ट्रिगर करें। - 2. **Observe → policy** पर जाएं और दोनों निर्णयों की तुलना करें। - 3. प्रत्येक लिंक किए गए सत्र को खोलें और सत्यापित करें कि ईवेंट पेलोड में नियम के लिए पर्याप्त साक्ष्य है। - 4. जब व्यवहार सही हो, समीक्षित स्रोत को **Admin → policy editor** में स्थानांतरित करें और एक संस्करण प्रकाशित करें। - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - `.failproofai/policies/` के तहत कन्वेंशन फ़ाइलें `--custom` के बिना लोड होती हैं। CI में एक स्पष्ट इंस्टॉल कमांड रखें जब सत्यापन एक टूटे हुए मॉड्यूल पर विफल होना चाहिए। - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -यह `production/config.yml`, `/srv/production/config.yml`, `/srv/production`, और `C:\\production\\config.yml` को `Write` और `Edit` दोनों के लिए मेल खाता है। यह `production-backup` जैसे नाम से नहीं मेल खाता क्योंकि `production` को एक पूर्ण पथ सेगमेंट होना चाहिए। - -एक स्पष्ट फ़ाइल को सत्यापित और स्थापित करें: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -नीति संदर्भ में ईवेंट प्रकार, सामान्यीकृत पेलोड, टूल नाम और इनपुट, सत्र मेटाडेटा, पैरामीटर, और उपलब्ध होने पर स्रोत CLI शामिल है। - -## विफलता पथों का परीक्षण करें - -प्रविष्टि फ़ाइल या किसी भी स्थानीय मॉड्यूल को बदलने के बाद सत्यापन चलाएं जो इसे आयात करता है: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -कड़ी CLI पथ लापता फ़ाइलों, सिंटैक्स त्रुटियों, अनसुलझे आयातों, शीर्ष-स्तरीय अपवादों, और मॉड्यूल-लोड टाइमआउट के लिए विफल हो जाती है। प्रवर्तन समय पर, एक टूटी हुई कस्टम फ़ाइल को लॉग किया जाता है और छोड़ दिया जाता है ताकि अंतर्निहित नीतियां जारी रह सकें। किसी भी लोड चेतावनी को प्रत्याशित प्रवर्तन की हानि के रूप में मानें और उत्पादन लॉग में इस पर सतर्क करें। - -स्पष्ट, कन्वेंशन, और Cloud-प्रबंधित नीतियों में विश्व स्तर पर अद्वितीय नामों का उपयोग करें। नीति कार्यों को नियतात्मक रखें, बाहरी कॉलों को छोटे टाइमआउट के साथ बाध्य करें, और प्रत्येक पथ पर एक इरादतन `allow`, `instruct`, या `deny` लौटाएं। - - - एक कस्टम नीति प्रवर्तन कोड है। लापता फ़ील्ड, वैकल्पिक टूल नामों, और विकृत इनपुट का परीक्षण करें—केवल अपेक्षित मिलान नहीं। - \ No newline at end of file diff --git a/docs/hi/policies/failure-behavior.mdx b/docs/hi/policies/failure-behavior.mdx index 1c635b52e..42bccfe67 100644 --- a/docs/hi/policies/failure-behavior.mdx +++ b/docs/hi/policies/failure-behavior.mdx @@ -1,5 +1,4 @@ --- ---- title: "विफलता व्यवहार" description: "समझें कि जब नीति मूल्यांकन या स्थानीय daemon अनुपलब्ध हो तो क्या होता है।" icon: "shield-alert" diff --git a/docs/hi/policies/fleet.mdx b/docs/hi/policies/fleet.mdx deleted file mode 100644 index 49118afa5..000000000 --- a/docs/hi/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "मशीनों पर नीतियां तैनात करें" -description: "जानें कि कौन सी मशीनें नामांकित हैं, वर्तमान हैं, और इच्छित नीति संस्करणों को लागू कर रही हैं।" -icon: "network" ---- - -फ़्लीट कवरेज यह उत्तर देता है कि क्या कोई नीति उस स्थान पर मौजूद है जहां जोखिम मौजूद है। स्थिर ID और मानव-पठनीय लेबल के आधार पर मशीनों को ट्रैक करें, फिर उनकी नियत और रिपोर्ट की गई तैनाती स्थिति की तुलना करें। - -## कवरेज जांचें - - - - 1. **Admin → enforcement** पर जाएं और लागू करने और देखने वाली कुल संख्या की समीक्षा करें। - 2. ID या लेबल के आधार पर एक मशीन खोजें, या नीति न रखने वाली मशीनों के लिए फ़िल्टर करें। - 3. नियत नीतियों, रिपोर्ट की गई तैनाती, अंतिम चेक-इन और इतिहास की तुलना करने के लिए एक पंक्ति का विस्तार करें। - 4. जब लागू की गई तैनाती लंबित रहे तो मशीन के पोलिंग अंतराल के बाद रीफ्रेश करें। - - ![लागू करने वाली फ़्लीट को दिखाने वाला Enforcement फ़्लीट, नीति कवरेज, मशीन तैनाती स्थिति, और देखें और लागू असाइनमेंट।](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - मशीन की एजेंट गतिविधि क्लाउड तक पहुंचती है यह पुष्टि करने के लिए `fp events --agent-id --since 24h` का उपयोग करें। - - - -कवरेज दृश्यों का उपयोग करके खोजें: - -- वे मशीनें जिन्होंने कभी नवीनतम तैनाती नहीं खींची -- नामांकित मशीनें जिन्होंने गतिविधि की रिपोर्टिंग बंद कर दी -- एक नीति जो गलत वातावरण या कोहोर्ट को निर्दिष्ट है -- एक बाधित अपडेट के बाद संस्करण बहाव - -बिना इसे फिर से कनेक्ट किए एक मशीन का नाम बदलें: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -स्थानीय स्थिति जांचें: - -```bash -failproofai config --status -``` - - - ऐसे लेबल का उपयोग करें जो वर्कलोड और वातावरण की पहचान करते हैं। ऑटोस्केलिंग या मशीन प्रतिस्थापन के बाद होस्टनाम अकेले अक्सर अपर्याप्त होते हैं। - \ No newline at end of file diff --git a/docs/hi/reference/local-dashboard.mdx b/docs/hi/reference/local-dashboard.mdx index c6d49c557..bb30c11f6 100644 --- a/docs/hi/reference/local-dashboard.mdx +++ b/docs/hi/reference/local-dashboard.mdx @@ -1,5 +1,4 @@ --- ---- title: "लोकल डैशबोर्ड" description: "लोकल प्रोजेक्ट, सेशन, पॉलिसी एक्टिविटी, कॉन्फ़िगरेशन, ऑडिट और शेड्यूल्ड स्कैन की समीक्षा करें।" icon: "monitor-cog" diff --git a/docs/hi/sessions/evaluations.mdx b/docs/hi/sessions/evaluations.mdx index 5f5e4fae5..4eb1d3d31 100644 --- a/docs/hi/sessions/evaluations.mdx +++ b/docs/hi/sessions/evaluations.mdx @@ -1,5 +1,4 @@ --- ---- title: "ऑनलाइन मूल्यांकन" description: "गुणवत्ता, अनुपालन, लागत और विलंबता के लिए लाइव और पूर्ण सत्रों को स्कोर करें।" icon: "gauge" diff --git a/docs/hi/start/integrations.mdx b/docs/hi/start/integrations.mdx index 7f4e3eebc..336abed0e 100644 --- a/docs/hi/start/integrations.mdx +++ b/docs/hi/start/integrations.mdx @@ -1,5 +1,4 @@ --- ---- title: "अपने एजेंट को इंस्ट्रूमेंट करें" sidebarTitle: "Frameworks" description: "किसी भी समर्थित एजेंट फ्रेमवर्क को Failproof AI से एक कॉल के साथ कनेक्ट करें।" diff --git a/docs/hi/start/integrations/crewai.mdx b/docs/hi/start/integrations/crewai.mdx index 0ed13312e..d09708955 100644 --- a/docs/hi/start/integrations/crewai.mdx +++ b/docs/hi/start/integrations/crewai.mdx @@ -1,5 +1,4 @@ --- ---- title: "CrewAI" sidebarTitle: "CrewAI" description: "Crews, flows, agents को भूमिका, tools, memory और human feedback द्वारा instrument करें।" diff --git a/docs/images/dashboard/eval-authoring-code.png b/docs/images/dashboard/eval-authoring-code.png new file mode 100644 index 000000000..c16be3853 Binary files /dev/null and b/docs/images/dashboard/eval-authoring-code.png differ diff --git a/docs/images/dashboard/eval-authoring-draft.png b/docs/images/dashboard/eval-authoring-draft.png new file mode 100644 index 000000000..c40385a6b Binary files /dev/null and b/docs/images/dashboard/eval-authoring-draft.png differ diff --git a/docs/images/dashboard/eval-definitions.png b/docs/images/dashboard/eval-definitions.png new file mode 100644 index 000000000..f209f8469 Binary files /dev/null and b/docs/images/dashboard/eval-definitions.png differ diff --git a/docs/images/dashboard/eval-test.png b/docs/images/dashboard/eval-test.png new file mode 100644 index 000000000..a949d2366 Binary files /dev/null and b/docs/images/dashboard/eval-test.png differ diff --git a/docs/images/dashboard/evaluations-assistant.png b/docs/images/dashboard/evaluations-assistant.png new file mode 100644 index 000000000..b23db07cb Binary files /dev/null and b/docs/images/dashboard/evaluations-assistant.png differ diff --git a/docs/images/dashboard/evaluations-chart.png b/docs/images/dashboard/evaluations-chart.png new file mode 100644 index 000000000..08002d949 Binary files /dev/null and b/docs/images/dashboard/evaluations-chart.png differ diff --git a/docs/images/dashboard/policy-backtest.png b/docs/images/dashboard/policy-backtest.png new file mode 100644 index 000000000..3ff3ebcf4 Binary files /dev/null and b/docs/images/dashboard/policy-backtest.png differ diff --git a/docs/it/admin/overview.mdx b/docs/it/admin/overview.mdx index 3bbdb458d..ee1356207 100644 --- a/docs/it/admin/overview.mdx +++ b/docs/it/admin/overview.mdx @@ -1,5 +1,4 @@ --- ---- title: "Amministrazione" description: "Gestisci accesso, utilizzo, organizzazioni e sicurezza senza mischiarli nel flusso di lavoro di affidabilità." icon: "settings-2" diff --git a/docs/it/audits/agent-contracts.mdx b/docs/it/audits/agent-contracts.mdx index 05743e556..161e0ff22 100644 --- a/docs/it/audits/agent-contracts.mdx +++ b/docs/it/audits/agent-contracts.mdx @@ -1,5 +1,4 @@ --- ---- title: "Contesto dell'agente" description: "Indica ai controlli cosa deve fare, produrre e non fare mai ogni agente." icon: "bot" diff --git a/docs/it/policies/builtin-catalog.mdx b/docs/it/policies/builtin-catalog.mdx deleted file mode 100644 index 0105ad69e..000000000 --- a/docs/it/policies/builtin-catalog.mdx +++ /dev/null @@ -1,133 +0,0 @@ ---- -title: "Catalogo delle policy integrate" -description: "Esamina ogni policy integrata di Failproof AI, il suo trigger, lo stato consigliato e i parametri configurabili." -icon: "list-checks" ---- - -Il pacchetto installato è la fonte autorevole per la disponibilità delle policy. Esegui `failproofai policies` dopo ogni aggiornamento perché le voci del catalogo e il comportamento possono cambiare con la versione del pacchetto. - -## Baseline consigliata - -La selezione consigliata della configurazione guidata attualmente abilita i sanitizzatori di segreti, le protezioni dell'ambiente, l'auto-protezione, i guard per comandi catastrofici e la sicurezza dei rami protetti: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` è **sempre attiva**. È elencata sopra per -completezza, ma si registra ad ogni valutazione indipendentemente dal fatto che -compaia nel tuo set abilitato, e non può essere disabilitata o messa in pausa — una protezione -contro l'agente che disattiva l'applicazione della normativa non è una protezione se -l'agente può disattivarla. - -La baseline consigliata è volutamente più ristretta di **Everything**. Le policy di infrastruttura e workflow possono interrompere il lavoro valido e dovrebbero essere abilitate nei repository e nelle macchine che le richiedono. - -## Segreti e ambiente - -| Policy | Trigger | Risultato | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | Oscura i JWT dall'output dello strumento prima che il modello li veda. | -| `sanitize-api-keys` | `PostToolUse` | Oscura le chiavi comuni di OpenAI, Anthropic, GitHub, AWS, Stripe e Google. | -| `sanitize-connection-strings` | `PostToolUse` | Oscura le stringhe di connessione al database contenenti credenziali. | -| `sanitize-private-key-content` | `PostToolUse` | Oscura i corpi delle chiavi private PEM. | -| `sanitize-bearer-tokens` | `PostToolUse` | Oscura i token bearer di autorizzazione. | -| `protect-env-vars` | `PreToolUse` su strumenti shell | Blocca i comandi che scaricano le variabili d'ambiente. | -| `block-env-files` | `PreToolUse` | Blocca le letture e le scritture dei file `.env`. | -| `block-read-outside-cwd` | `PreToolUse` su strumenti read, glob, grep o shell | Mantieni le letture all'interno della directory di lavoro della sessione. | -| `block-secrets-write` | `PreToolUse` su strumenti di scrittura | Blocca le scritture ai nomi di file comuni di chiavi segrete e credenziali. | - -## Comandi pericolosi e infrastruttura - -| Policy | Trigger | Risultato | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | Blocca `sudo` a meno che non corrisponda un pattern di autorizzazione. | -| `block-curl-pipe-sh` | `PreToolUse` | Blocca gli script scaricati reindirizzati direttamente a una shell. | -| `block-rm-rf` | `PreToolUse` | Blocca i pattern di eliminazione ricorsiva catastrofica. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Sempre attiva, non può essere disabilitata.** Blocca ogni invocazione della CLI di Failproof AI, l'auto-pausa e la disinstallazione del gestore di pacchetti. | -| `block-kubectl` | `PreToolUse` | Controlla i comandi Kubernetes. | -| `block-terraform` | `PreToolUse` | Controlla i comandi Terraform e OpenTofu. | -| `block-aws-cli` | `PreToolUse` | Controlla i comandi AWS CLI. | -| `block-gcloud` | `PreToolUse` | Controlla i comandi Google Cloud CLI. | -| `block-az-cli` | `PreToolUse` | Controlla i comandi Azure CLI. | -| `block-helm` | `PreToolUse` | Controlla i comandi Helm. | -| `block-gh-pipeline` | `PreToolUse` | Controlla le operazioni di workflow, run, merge, release, cache e secret di GitHub CLI che modificano lo stato. | - -## Sicurezza Git e database - -| Policy | Trigger | Risultato | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Blocca i push diretti ai rami protetti configurati. | -| `block-force-push` | `PreToolUse` | Blocca i force-push; `--force-with-lease` rimane consentito dall'implementazione attuale. | -| `block-work-on-main` | `PreToolUse` | Blocca i commit e i merge sui rami protetti. | -| `warn-git-amend` | `PreToolUse` | Avvisa prima di riscrivere un commit con `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | Avvisa prima di eliminare o cancellare definitivamente gli stash. | -| `warn-all-files-staged` | `PreToolUse` | Avvisa su ampi `git add -A`, `git add .` o `git add --all`. | -| `warn-destructive-sql` | `PreToolUse` | Avvisa su `DROP`, `TRUNCATE` e `DELETE` senza `WHERE` attraverso i client di database riconosciuti. | -| `warn-schema-alteration` | `PreToolUse` | Avvisa su riconosciute operazioni di colonna e rinomina di `ALTER TABLE`. | - -## Pacchetti, comportamento del sistema e loop dell'agente - -| Policy | Trigger | Risultato | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Avvisa prima di pubblicare sui registri dei pacchetti. | -| `warn-global-package-install` | `PreToolUse` | Avvisa prima dell'installazione globale di pacchetti. | -| `prefer-package-manager` | `PreToolUse` | Istruisce l'agente a utilizzare un gestore di pacchetti consentito. | -| `warn-large-file-write` | `PreToolUse` su strumenti di scrittura | Avvisa al di sopra della soglia di dimensione del file configurata. | -| `warn-background-process` | `PreToolUse` | Avvisa sui pattern di processi in background staccati o di lunga durata. | -| `warn-repeated-tool-calls` | `PreToolUse` | Avvisa dopo tre o più chiamate identiche dello strumento. | - -## Flusso di lavoro di fine attività - -Queste policy richiedono un harness che emetta un evento `Stop` compatibile. - -| Policy | Risultato | -| --- | --- | -| `require-commit-before-stop` | Rifiuta il completamento mentre rimane lavoro tracciato non committato. | -| `require-push-before-stop` | Rifiuta il completamento mentre i commit rimangono solo locali. | -| `require-pr-before-stop` | Richiede una pull request per il ramo corrente. | -| `require-no-conflicts-before-stop` | Richiede un merge pulito contro il ramo base configurato. | -| `require-ci-green-before-stop` | Richiede il completamento con successo dei controlli CI dell'HEAD corrente. | - -## Riferimento dei parametri - -Configura i parametri sotto l'oggetto `policyParams` dello scope selezionato. I tipi sono validati da ogni policy. - -| Policy | Parametro | Tipo e default | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; le voci contengono `regex` e `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Bloccatori infrastruttura | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - Un pattern di autorizzazione amplia quello che un agente può fare. Verifica la tokenizzazione esatta e le varianti di comando sull'harness target prima di distribuirlo su una flotta. - \ No newline at end of file diff --git a/docs/it/policies/builtin.mdx b/docs/it/policies/builtin.mdx deleted file mode 100644 index 14d5760b7..000000000 --- a/docs/it/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Politiche integrate" -description: "Abilita protezioni mantenute per i comuni modi di errore degli agenti." -icon: "library" ---- - -Le politiche integrate coprono la gestione dei segreti, i file di ambiente, i comandi shell distruttivi, i rami protetti, gli strumenti cloud e infrastruttura, la pubblicazione di pacchetti, le chiamate ripetute e i controlli del flusso di lavoro di fine attività. - -## Abilitare e verificare una politica integrata - - - - 1. Installa la politica su una macchina connessa con la CLI locale. - 2. Esegui un'azione di test sicura nell'agente strumentato. - 3. Vai a **Observe → policy** e filtra per nome della politica, ambiente della macchina o decisione. - 4. Apri la sessione collegata per confermare l'input dello strumento corrispondente e il motivo restituito. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - Rimuovila con `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -Elenca le politiche disponibili nella tua versione installata: - -```bash -failproofai policies -``` - -Abilita una politica per un progetto: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Abilita più politiche per strumenti selezionati: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Alcune politiche accettano parametri o sono contrassegnate come beta. Rivedi la descrizione, l'ambito di corrispondenza e il comportamento predefinito prima del rollout. Una politica che protegge un flusso di lavoro potrebbe bloccare operazioni valide in un altro. - - - Rivedi tutte le 40 politiche attuali, i loro trigger, la baseline consigliata e i parametri. - - - - Preferisci l'ambito del progetto per le aspettative specifiche del repository e l'ambito utente per i requisiti di sicurezza a livello di macchina. - \ No newline at end of file diff --git a/docs/it/policies/custom.mdx b/docs/it/policies/custom.mdx deleted file mode 100644 index f74a96264..000000000 --- a/docs/it/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Criteri personalizzati" -description: "Scrivi un criterio per una modalità di errore univoca nel flusso di lavoro del tuo agente." -icon: "shield-plus" ---- - -Crea un file che termina con `policies.js`, `policies.mjs` o `policies.ts` nella cartella `.failproofai/policies/`. I file per convenzione si caricano automaticamente a livello di progetto e utente. - -## Test del criterio prima della pubblicazione Cloud - - - - 1. Installa il criterio personalizzato su una macchina di test e attiva sia un'azione corrispondente che una non-corrispondenza legittima. - 2. Vai a **Observe → policy** e confronta le due decisioni. - 3. Apri ogni sessione collegata e verifica che il payload dell'evento contenga prove sufficienti per la regola. - 4. Quando il comportamento è corretto, sposta il codice sorgente controllato in **Admin → policy editor** e pubblica una versione. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - I file per convenzione in `.failproofai/policies/` si caricano senza `--custom`. Mantieni un comando di installazione esplicito in CI quando la convalida deve fallire su un modulo difettoso. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -Questo corrisponde a `production/config.yml`, `/srv/production/config.yml`, `/srv/production` e `C:\\production\\config.yml` sia per `Write` che per `Edit`. Non corrisponde a nomi come `production-backup` perché `production` deve essere un segmento di percorso completo. - -Convalida e installa un file esplicito: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -Il contesto della policy include il tipo di evento, il payload normalizzato, il nome e l'input dello strumento, i metadati della sessione, i parametri e la CLI di origine quando disponibile. - -## Test dei percorsi di errore - -Esegui la convalida dopo aver modificato il file di entry o qualsiasi modulo locale che importa: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -Il percorso CLI rigoroso fallisce per file mancanti, errori di sintassi, importazioni non risolte, eccezioni di livello superiore e timeout di caricamento del modulo. Al momento dell'applicazione, un file personalizzato difettoso viene registrato e saltato in modo che i criteri incorporati possano continuare. Considera qualsiasi avviso di caricamento come una perdita dell'applicazione prevista e generale un avviso nei log di produzione. - -Utilizza nomi globalmente univoci su tutti i criteri espliciti, per convenzione e gestiti da Cloud. Mantieni le funzioni della policy deterministiche, limita le chiamate esterne con timeout brevi e restituisci un intenzionale `allow`, `instruct` o `deny` su ogni percorso. - - - Un criterio personalizzato è codice di applicazione. Testa campi mancanti, nomi di strumenti alternativi e input non valido—non solo la corrispondenza prevista. - \ No newline at end of file diff --git a/docs/it/policies/fleet.mdx b/docs/it/policies/fleet.mdx deleted file mode 100644 index af98bd080..000000000 --- a/docs/it/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Distribuisci politiche alle macchine" -description: "Conosci quali macchine sono iscritte, aggiornate e applicano le versioni di politica previste." -icon: "network" ---- - -La copertura della flotta risponde alla domanda se esiste una politica dove esiste il rischio. Traccia le macchine tramite ID stabile e un'etichetta leggibile dall'uomo, quindi confronta lo stato di distribuzione assegnato e segnalato. - -## Controlla la copertura - - - - 1. Vai a **Admin → enforcement** e rivedi i totali in applicazione e osservazione. - 2. Cerca una macchina per ID o etichetta, oppure filtra per le macchine a cui manca una politica. - 3. Espandi una riga per confrontare le politiche assegnate, la distribuzione segnalata, l'ultimo check-in e la cronologia. - 4. Aggiorna dopo l'intervallo di polling della macchina quando una distribuzione applicata rimane in sospeso. - - ![La flotta Enforcement che mostra la copertura delle politiche, lo stato di distribuzione della macchina e gli assegnamenti di osservazione e applicazione.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Usa `fp events --agent-id --since 24h` per confermare che l'attività dell'agente della macchina raggiunge il Cloud. - - - -Usa le viste di copertura per trovare: - -- Macchine che non hanno mai richiesto la distribuzione più recente -- Macchine iscritte che hanno smesso di segnalare attività -- Una politica assegnata all'ambiente o alla coorte sbagliati -- Variazione di versione dopo un aggiornamento interrotto - -Rinomina una macchina senza riconnetterla: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Controlla lo stato locale: - -```bash -failproofai config --status -``` - - - Usa etichette che identifichino il carico di lavoro e l'ambiente. I nomi host da soli sono spesso insufficienti dopo il ridimensionamento automatico o la sostituzione della macchina. - \ No newline at end of file diff --git a/docs/it/policies/local-configuration.mdx b/docs/it/policies/local-configuration.mdx index 549621077..a758d9606 100644 --- a/docs/it/policies/local-configuration.mdx +++ b/docs/it/policies/local-configuration.mdx @@ -1,5 +1,4 @@ --- ---- title: "Configurazione locale" description: "Controlla l'ambito delle policy, i parametri, i file personalizzati e le impostazioni di Failproof AI a livello di macchina." icon: "file-cog" diff --git a/docs/it/reference/evaluator-sdk.mdx b/docs/it/reference/evaluator-sdk.mdx index 693304db9..0fc260696 100644 --- a/docs/it/reference/evaluator-sdk.mdx +++ b/docs/it/reference/evaluator-sdk.mdx @@ -1,5 +1,4 @@ --- ---- title: "Evaluator SDK" description: "Costruisci un servizio che valuta le sessioni di Failproof AI in modo sincrono o asincrono." icon: "gauge" diff --git a/docs/it/reference/harnesses.mdx b/docs/it/reference/harnesses.mdx index 701f74256..5839de80c 100644 --- a/docs/it/reference/harnesses.mdx +++ b/docs/it/reference/harnesses.mdx @@ -1,5 +1,4 @@ --- ---- title: "Harness per agent" description: "Cattura sessioni e applica policy su tutti i 12 harness per agent supportati." icon: "plug-zap" diff --git a/docs/it/reference/local-dashboard.mdx b/docs/it/reference/local-dashboard.mdx index e2a200ef4..d00b81828 100644 --- a/docs/it/reference/local-dashboard.mdx +++ b/docs/it/reference/local-dashboard.mdx @@ -1,5 +1,4 @@ --- ---- title: "Dashboard locale" description: "Rivedi progetti locali, sessioni, attività delle policy, configurazione, audit e scansioni pianificate." icon: "monitor-cog" diff --git a/docs/it/sessions/assistant.mdx b/docs/it/sessions/assistant.mdx index ac6815ebd..ad66c6b21 100644 --- a/docs/it/sessions/assistant.mdx +++ b/docs/it/sessions/assistant.mdx @@ -1,5 +1,4 @@ --- ---- title: "Failproof Assistant" description: "Analizza e gestisci Failproof AI in linguaggio naturale, da domande e query a dashboard e audit." icon: "message-square-text" diff --git a/docs/it/sessions/queries.mdx b/docs/it/sessions/queries.mdx index a474e1f37..9d760555d 100644 --- a/docs/it/sessions/queries.mdx +++ b/docs/it/sessions/queries.mdx @@ -1,5 +1,4 @@ --- ---- title: "Query" description: "Esplora i dati di sessione, evento e valutazione con SQL riutilizzabile." icon: "database" diff --git a/docs/it/start/quickstarts/langchain.mdx b/docs/it/start/quickstarts/langchain.mdx index 473332274..97db6c747 100644 --- a/docs/it/start/quickstarts/langchain.mdx +++ b/docs/it/start/quickstarts/langchain.mdx @@ -1,5 +1,4 @@ --- ---- title: "LangChain e LangGraph" description: "Installa, strumenta e visualizza la tua prima traccia." icon: "/images/frameworks/langchain.svg" diff --git a/docs/ja/policies/builtin-catalog.mdx b/docs/ja/policies/builtin-catalog.mdx deleted file mode 100644 index 77146af0c..000000000 --- a/docs/ja/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "ビルトインポリシーカタログ" -description: "Failproof AI の全ビルトインポリシー、そのトリガー、推奨状態、および設定可能なパラメーターを確認できます。" -icon: "list-checks" ---- - -インストール済みパッケージがポリシーの利用可否に関する唯一の情報源です。パッケージをアップグレードするたびに `failproofai policies` を実行してください。カタログのエントリや動作はパッケージバージョンによって変わることがあります。 - -## 推奨ベースライン - -ガイド付きセットアップの推奨選択では、現在、シークレットサニタイザー、環境保護、自己保護、致命的コマンドガード、保護ブランチの安全策が有効化されます： - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` は**常に有効**です。上記に記載しているのは網羅性のためであり、有効なセットに含まれているかどうかに関係なく、すべての評価時に登録されます。無効化や一時停止はできません。エージェントが無効にできる「エージェントによる強制のオフ」に対するガードは、ガードとして機能しないからです。 - -推奨設定は意図的に**すべて**よりも範囲を絞っています。インフラや開発ワークフローのポリシーは正当な作業を中断させる可能性があるため、それらを必要とするリポジトリやマシンに限定して有効化すべきです。 - -## シークレットと環境 - -| ポリシー | トリガー | 結果 | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | モデルに渡る前にツール出力から JWT を削除する。 | -| `sanitize-api-keys` | `PostToolUse` | OpenAI、Anthropic、GitHub、AWS、Stripe、Google の主要なキーを削除する。 | -| `sanitize-connection-strings` | `PostToolUse` | 認証情報を含むデータベース接続文字列を削除する。 | -| `sanitize-private-key-content` | `PostToolUse` | PEM 秘密鍵の本文を削除する。 | -| `sanitize-bearer-tokens` | `PostToolUse` | Authorization ベアラートークンを削除する。 | -| `protect-env-vars` | シェルツール上の `PreToolUse` | 環境変数をダンプするコマンドをブロックする。 | -| `block-env-files` | `PreToolUse` | `.env` ファイルの読み取りおよび書き込みをブロックする。 | -| `block-read-outside-cwd` | 読み取り・glob・grep・シェルツール上の `PreToolUse` | 読み取りをセッションの作業ディレクトリ内に限定する。 | -| `block-secrets-write` | 書き込みツール上の `PreToolUse` | 一般的なシークレットキーや認証情報ファイル名への書き込みをブロックする。 | - -## 危険なコマンドとインフラ - -| ポリシー | トリガー | 結果 | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`、`PermissionRequest` | allow パターンに一致しない限り `sudo` をブロックする。 | -| `block-curl-pipe-sh` | `PreToolUse` | ダウンロードしたスクリプトをシェルに直接パイプすることをブロックする。 | -| `block-rm-rf` | `PreToolUse` | 致命的な再帰削除パターンをブロックする。 | -| `block-failproofai-commands` | `PreToolUse`、`PermissionRequest` | **常に有効、無効化不可。** Failproof AI CLI のすべての呼び出し、自己一時停止、パッケージマネージャーによるアンインストールをブロックする。 | -| `block-kubectl` | `PreToolUse` | Kubernetes コマンドをゲートする。 | -| `block-terraform` | `PreToolUse` | Terraform および OpenTofu コマンドをゲートする。 | -| `block-aws-cli` | `PreToolUse` | AWS CLI コマンドをゲートする。 | -| `block-gcloud` | `PreToolUse` | Google Cloud CLI コマンドをゲートする。 | -| `block-az-cli` | `PreToolUse` | Azure CLI コマンドをゲートする。 | -| `block-helm` | `PreToolUse` | Helm コマンドをゲートする。 | -| `block-gh-pipeline` | `PreToolUse` | GitHub CLI の変更を伴うワークフロー・実行・マージ・リリース・キャッシュ・シークレット操作をゲートする。 | - -## Git とデータベースの安全性 - -| ポリシー | トリガー | 結果 | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | 設定された保護ブランチへの直接プッシュをブロックする。 | -| `block-force-push` | `PreToolUse` | フォースプッシュをブロックする。現在の実装では `--force-with-lease` は引き続き許可される。 | -| `block-work-on-main` | `PreToolUse` | 保護ブランチへのコミットおよびマージをブロックする。 | -| `warn-git-amend` | `PreToolUse` | `--amend` によるコミットの書き換え前に警告する。 | -| `warn-git-stash-drop` | `PreToolUse` | スタッシュを永続的にドロップまたはクリアする前に警告する。 | -| `warn-all-files-staged` | `PreToolUse` | 広範な `git add -A`、`git add .`、`git add --all` に対して警告する。 | -| `warn-destructive-sql` | `PreToolUse` | 認識されたデータベースクライアント経由で `WHERE` なしの `DROP`、`TRUNCATE`、`DELETE` が実行される前に警告する。 | -| `warn-schema-alteration` | `PreToolUse` | 認識された `ALTER TABLE` のカラム操作やリネーム操作に対して警告する。 | - -## パッケージ、システム動作、エージェントループ - -| ポリシー | トリガー | 結果 | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | パッケージレジストリへの公開前に警告する。 | -| `warn-global-package-install` | `PreToolUse` | グローバルパッケージのインストール前に警告する。 | -| `prefer-package-manager` | `PreToolUse` | 許可されたパッケージマネージャーを使用するようエージェントに指示する。 | -| `warn-large-file-write` | 書き込みツール上の `PreToolUse` | 設定されたファイルサイズしきい値を超えた場合に警告する。 | -| `warn-background-process` | `PreToolUse` | デタッチまたは長期実行のバックグラウンドプロセスパターンに対して警告する。 | -| `warn-repeated-tool-calls` | `PreToolUse` | 同一のツール呼び出しが 3 回以上繰り返された場合に警告する。 | - -## タスク終了時のワークフロー - -これらのポリシーは、互換性のある `Stop` イベントを発行するハーネスを必要とします。 - -| ポリシー | 結果 | -| --- | --- | -| `require-commit-before-stop` | トラッキングされた作業がコミットされていない間は完了を拒否する。 | -| `require-push-before-stop` | コミットがローカルのみの間は完了を拒否する。 | -| `require-pr-before-stop` | 現在のブランチに対してプルリクエストを要求する。 | -| `require-no-conflicts-before-stop` | 設定されたベースブランチに対してクリーンなマージを要求する。 | -| `require-ci-green-before-stop` | 現在の HEAD の CI チェックが正常に完了することを要求する。 | - -## パラメーターリファレンス - -選択したスコープの `policyParams` オブジェクト配下でパラメーターを設定します。型は各ポリシーによって検証されます。 - -| ポリシー | パラメーター | 型とデフォルト値 | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`、`[]`；エントリには `regex` と `label` が含まれる | -| `block-read-outside-cwd` | `allowPaths` | `string[]`、`[]` | -| `block-sudo` | `allowPatterns` | `string[]`、`[]` | -| `block-rm-rf` | `allowPaths` | `string[]`、`[]` | -| インフラブロッカー | `allowPatterns` | `string[]`、`[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`、`[]` | -| `block-push-master` | `protectedBranches` | `string[]`、`["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`、`["main", "master"]` | -| `prefer-package-manager` | `allowed`、`blocked` | `string[]`、`[]` | -| `warn-large-file-write` | `thresholdKb` | `number`、`1024` | -| `require-push-before-stop` | `remote`、`baseBranch` | `string`、`"origin"`；`string`、`"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`、`"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`、`"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - allow パターンはエージェントが実行できる範囲を広げます。フリートへの展開前に、対象ハーネス上でトークン化の詳細とコマンドのバリエーションを正確にテストしてください。 - \ No newline at end of file diff --git a/docs/ja/policies/builtin.mdx b/docs/ja/policies/builtin.mdx deleted file mode 100644 index 2b9a136d6..000000000 --- a/docs/ja/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "組み込みポリシー" -description: "一般的なエージェント障害モードに対する管理されたガードレールを有効にします。" -icon: "library" ---- - -組み込みポリシーは、シークレット処理、環境ファイル、破壊的なシェルコマンド、保護されたブランチ、クラウドおよびインフラツール、パッケージ公開、繰り返し呼び出し、タスク終了時のワークフローチェックをカバーします。 - -## 組み込みポリシーの有効化と検証 - - - - 1. ローカル CLI を使用して、接続済みのマシンにポリシーをインストールします。 - 2. インストルメント済みのエージェントで安全なテストアクションを実行します。 - 3. **Observe → policy** に移動し、ポリシー名、マシン環境、または決定でフィルタリングします。 - 4. リンクされたセッションを開いて、マッチしたツール入力と返された理由を確認します。 - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - `failproofai policy remove block-rm-rf --cli claude --scope project` で削除します。 - - - -インストール済みバージョンで利用可能なポリシーを一覧表示します。 - -```bash -failproofai policies -``` - -プロジェクトに対して 1 つのポリシーを有効にします。 - -```bash -failproofai policy add block-rm-rf --scope project -``` - -選択したハーネスに対して複数のポリシーを有効にします。 - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -一部のポリシーはパラメータを受け付けるか、ベータとしてマークされています。ロールアウト前に説明、マッチスコープ、デフォルト動作を確認してください。あるワークフローを保護するポリシーが、別のワークフローで有効な操作をブロックする場合があります。 - - - 現在の 40 件のポリシーすべて、トリガー、推奨ベースライン、パラメータを確認します。 - - - - リポジトリ固有の要件にはプロジェクトスコープを、マシン全体の安全要件にはユーザースコープを優先してください。 - \ No newline at end of file diff --git a/docs/ja/policies/custom.mdx b/docs/ja/policies/custom.mdx deleted file mode 100644 index 9153bb5e5..000000000 --- a/docs/ja/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "カスタムポリシー" -description: "エージェントワークフロー固有の障害モードに対応するポリシーを作成します。" -icon: "shield-plus" ---- - -`.failproofai/policies/` 以下に `policies.js`、`policies.mjs`、または `policies.ts` で終わるファイルを作成してください。規約ファイルはプロジェクトスコープおよびユーザースコープで自動的に読み込まれます。 - -## クラウド公開前にポリシーをテストする - - - - 1. カスタムポリシーをテストマシン1台にインストールし、マッチするアクションと正当なノンマッチの両方をトリガーします。 - 2. **Observe → policy** に移動して、2つの判定結果を比較します。 - 3. リンクされた各セッションを開き、イベントペイロードにルールの根拠となる十分な証跡が含まれているか確認します。 - 4. 動作が正しいことを確認したら、レビュー済みのソースを **Admin → policy editor** に移してバージョンを公開します。 - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - `.failproofai/policies/` 以下の規約ファイルは `--custom` なしで読み込まれます。壊れたモジュールでバリデーションを失敗させたい CI 環境では、明示的なインストールコマンドを残してください。 - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -これは `production/config.yml`、`/srv/production/config.yml`、`/srv/production`、`C:\\production\\config.yml` に対して `Write` と `Edit` の両方でマッチします。`production` はパスセグメント全体でなければならないため、`production-backup` のような名前にはマッチしません。 - -明示的なファイルを検証してインストールするには： - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -ポリシーコンテキストには、イベントタイプ、正規化されたペイロード、ツール名と入力、セッションメタデータ、パラメーター、および利用可能な場合はソース CLI が含まれます。 - -## 失敗パスのテスト - -エントリファイルまたはそれがインポートするローカルモジュールを変更した後は、バリデーションを実行してください： - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -strict な CLI パスは、ファイルが見つからない場合、構文エラー、未解決のインポート、トップレベルの例外、モジュール読み込みタイムアウトで失敗します。実行時に壊れたカスタムファイルはログに記録されてスキップされるため、組み込みポリシーは継続して機能します。読み込み警告は期待される実行の損失として扱い、本番ログでアラートを設定してください。 - -明示的・規約・クラウド管理のポリシーを通じてグローバルに一意な名前を使用してください。ポリシー関数は決定論的に保ち、外部呼び出しには短いタイムアウトを設定し、すべてのパスで意図的な `allow`、`instruct`、または `deny` を返すようにしてください。 - - - カスタムポリシーは実行コードです。期待されるマッチだけでなく、フィールドの欠如、代替ツール名、不正な入力についてもテストしてください。 - \ No newline at end of file diff --git a/docs/ja/policies/fleet.mdx b/docs/ja/policies/fleet.mdx deleted file mode 100644 index 5f364d2fd..000000000 --- a/docs/ja/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "マシンへのポリシーのデプロイ" -description: "登録済みのマシン、最新状態のマシン、および意図したポリシーバージョンを適用中のマシンを把握します。" -icon: "network" ---- - -フリートカバレッジは、リスクが存在する場所にポリシーが存在するかどうかを確認するためのものです。安定したIDと人が読めるラベルでマシンを追跡し、割り当て済みのデプロイ状態と報告されたデプロイ状態を比較します。 - -## カバレッジの確認 - - - - 1. **Admin → enforcement** に移動し、適用中（enforcing）と監視中（observing）の合計を確認します。 - 2. IDまたはラベルでマシンを検索するか、ポリシーが割り当てられていないマシンをフィルタリングします。 - 3. 行を展開して、割り当て済みポリシー、報告されたデプロイ状態、最終チェックイン、および履歴を比較します。 - 4. 適用済みのデプロイが保留中のままになっている場合は、マシンのポーリング間隔が経過した後に更新します。 - - ![ポリシーカバレッジ、マシンのデプロイ状態、監視および適用の割り当てを表示するEnforcementフリート画面。](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - `fp events --agent-id --since 24h` を使用して、マシンのエージェントアクティビティがCloudに到達していることを確認します。 - - - -カバレッジビューを使用して以下を検出できます： - -- 最新のデプロイを一度も取得していないマシン -- アクティビティの報告を停止した登録済みマシン -- 誤った環境またはコホートに割り当てられたポリシー -- 中断されたアップデート後のバージョンのずれ - -再接続せずにマシンの名前を変更する： - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -ローカル状態を確認する： - -```bash -failproofai config --status -``` - - - ワークロードと環境を識別するラベルを使用してください。ホスト名だけでは、オートスケーリングやマシン入れ替え後に不十分になることがよくあります。 - \ No newline at end of file diff --git a/docs/ko/policies/builtin-catalog.mdx b/docs/ko/policies/builtin-catalog.mdx deleted file mode 100644 index 7d23bd8be..000000000 --- a/docs/ko/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "기본 제공 정책 카탈로그" -description: "Failproof AI의 모든 기본 제공 정책, 트리거, 권장 상태, 설정 가능한 파라미터를 확인하세요." -icon: "list-checks" ---- - -설치된 패키지가 정책 가용성의 기준입니다. 업그레이드할 때마다 `failproofai policies`를 실행하세요. 카탈로그 항목과 동작은 패키지 버전에 따라 변경될 수 있습니다. - -## 권장 기준 설정 - -가이드 설정의 권장 선택 항목은 현재 시크릿 삭제, 환경 보호, 자기 보호, 치명적 명령 차단, 보호 브랜치 안전 기능을 활성화합니다: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands`는 **항상 활성화**되어 있습니다. 완전성을 위해 위에 나열되었지만, 활성화된 목록에 포함 여부와 관계없이 모든 평가 시 등록되며, 비활성화하거나 일시 중지할 수 없습니다. 에이전트가 끌 수 있는 강제 수단은 진정한 보호 수단이 아닙니다. - -권장 설정은 의도적으로 **전체 활성화**보다 좁은 범위로 설정되어 있습니다. 인프라 및 워크플로 정책은 유효한 작업을 방해할 수 있으므로, 해당 정책이 필요한 저장소와 머신에서만 활성화해야 합니다. - -## 시크릿 및 환경 - -| 정책 | 트리거 | 결과 | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | 모델이 확인하기 전에 도구 출력에서 JWT를 삭제합니다. | -| `sanitize-api-keys` | `PostToolUse` | OpenAI, Anthropic, GitHub, AWS, Stripe, Google 등의 일반적인 API 키를 삭제합니다. | -| `sanitize-connection-strings` | `PostToolUse` | 자격 증명이 포함된 데이터베이스 연결 문자열을 삭제합니다. | -| `sanitize-private-key-content` | `PostToolUse` | PEM 개인 키 본문을 삭제합니다. | -| `sanitize-bearer-tokens` | `PostToolUse` | Authorization bearer 토큰을 삭제합니다. | -| `protect-env-vars` | 셸 도구에서 `PreToolUse` | 환경 변수를 덤프하는 명령을 차단합니다. | -| `block-env-files` | `PreToolUse` | `.env` 파일의 읽기 및 쓰기를 차단합니다. | -| `block-read-outside-cwd` | 읽기, glob, grep, 셸 도구에서 `PreToolUse` | 읽기 작업을 세션 작업 디렉토리 내로 제한합니다. | -| `block-secrets-write` | 쓰기 도구에서 `PreToolUse` | 일반적인 시크릿 키 및 자격 증명 파일명에 대한 쓰기를 차단합니다. | - -## 위험 명령 및 인프라 - -| 정책 | 트리거 | 결과 | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | 허용 패턴이 일치하지 않는 한 `sudo`를 차단합니다. | -| `block-curl-pipe-sh` | `PreToolUse` | 셸로 직접 파이프되는 다운로드 스크립트를 차단합니다. | -| `block-rm-rf` | `PreToolUse` | 치명적인 재귀 삭제 패턴을 차단합니다. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **항상 활성화, 비활성화 불가.** 모든 Failproof AI CLI 호출, 자기 일시 중지, 패키지 매니저 제거를 차단합니다. | -| `block-kubectl` | `PreToolUse` | Kubernetes 명령을 제어합니다. | -| `block-terraform` | `PreToolUse` | Terraform 및 OpenTofu 명령을 제어합니다. | -| `block-aws-cli` | `PreToolUse` | AWS CLI 명령을 제어합니다. | -| `block-gcloud` | `PreToolUse` | Google Cloud CLI 명령을 제어합니다. | -| `block-az-cli` | `PreToolUse` | Azure CLI 명령을 제어합니다. | -| `block-helm` | `PreToolUse` | Helm 명령을 제어합니다. | -| `block-gh-pipeline` | `PreToolUse` | GitHub CLI의 변경성 워크플로, 실행, 병합, 릴리스, 캐시, 시크릿 작업을 제어합니다. | - -## Git 및 데이터베이스 안전 - -| 정책 | 트리거 | 결과 | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | 설정된 보호 브랜치로의 직접 푸시를 차단합니다. | -| `block-force-push` | `PreToolUse` | 강제 푸시를 차단합니다. 현재 구현에서는 `--force-with-lease`는 허용됩니다. | -| `block-work-on-main` | `PreToolUse` | 보호 브랜치에서의 커밋 및 병합을 차단합니다. | -| `warn-git-amend` | `PreToolUse` | `--amend`로 커밋을 재작성하기 전에 경고합니다. | -| `warn-git-stash-drop` | `PreToolUse` | 스태시를 영구적으로 삭제하거나 초기화하기 전에 경고합니다. | -| `warn-all-files-staged` | `PreToolUse` | 광범위한 `git add -A`, `git add .`, `git add --all` 사용 시 경고합니다. | -| `warn-destructive-sql` | `PreToolUse` | 인식된 데이터베이스 클라이언트를 통해 `WHERE` 절 없이 `DROP`, `TRUNCATE`, `DELETE` 사용 시 경고합니다. | -| `warn-schema-alteration` | `PreToolUse` | 인식된 `ALTER TABLE` 컬럼 및 이름 변경 작업 시 경고합니다. | - -## 패키지, 시스템 동작, 에이전트 루프 - -| 정책 | 트리거 | 결과 | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | 패키지 레지스트리에 게시하기 전에 경고합니다. | -| `warn-global-package-install` | `PreToolUse` | 전역 패키지 설치 전에 경고합니다. | -| `prefer-package-manager` | `PreToolUse` | 허용된 패키지 매니저를 사용하도록 에이전트에게 지시합니다. | -| `warn-large-file-write` | 쓰기 도구에서 `PreToolUse` | 설정된 파일 크기 임계값을 초과하면 경고합니다. | -| `warn-background-process` | `PreToolUse` | 분리되거나 장기 실행되는 백그라운드 프로세스 패턴 시 경고합니다. | -| `warn-repeated-tool-calls` | `PreToolUse` | 동일한 도구 호출이 세 번 이상 반복되면 경고합니다. | - -## 작업 종료 워크플로 - -이 정책들은 호환되는 `Stop` 이벤트를 발생시키는 하네스가 필요합니다. - -| 정책 | 결과 | -| --- | --- | -| `require-commit-before-stop` | 추적된 작업이 커밋되지 않은 상태에서 완료를 거부합니다. | -| `require-push-before-stop` | 커밋이 로컬에만 존재하는 상태에서 완료를 거부합니다. | -| `require-pr-before-stop` | 현재 브랜치에 대한 풀 리퀘스트를 요구합니다. | -| `require-no-conflicts-before-stop` | 설정된 베이스 브랜치에 대해 충돌 없는 병합을 요구합니다. | -| `require-ci-green-before-stop` | 현재 HEAD의 CI 검사가 성공적으로 완료될 것을 요구합니다. | - -## 파라미터 참조 - -선택된 스코프의 `policyParams` 객체 아래에서 파라미터를 설정합니다. 각 정책에 의해 타입이 검증됩니다. - -| 정책 | 파라미터 | 타입 및 기본값 | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; 항목에 `regex` 및 `label` 포함 | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| 인프라 차단 정책 | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - 허용 패턴은 에이전트가 수행할 수 있는 작업의 범위를 넓힙니다. 플릿 전체에 배포하기 전에 대상 하네스에서 정확한 토크나이제이션과 명령 변형을 테스트하세요. - \ No newline at end of file diff --git a/docs/ko/policies/builtin.mdx b/docs/ko/policies/builtin.mdx deleted file mode 100644 index 11e4d015d..000000000 --- a/docs/ko/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "내장 정책" -description: "일반적인 에이전트 실패 모드에 대한 관리형 가드레일을 활성화합니다." -icon: "library" ---- - -내장 정책은 시크릿 처리, 환경 파일, 파괴적인 셸 명령, 보호된 브랜치, 클라우드 및 인프라 도구, 패키지 게시, 반복 호출, 작업 종료 워크플로우 검사를 다룹니다. - -## 내장 정책 활성화 및 검증 - - - - 1. 로컬 CLI로 연결된 머신에 정책을 설치합니다. - 2. 계측된 에이전트에서 안전한 테스트 작업을 실행합니다. - 3. **Observe → policy**로 이동하여 정책 이름, 머신 환경 또는 결정으로 필터링합니다. - 4. 연결된 세션을 열어 매칭된 도구 입력과 반환된 이유를 확인합니다. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - `failproofai policy remove block-rm-rf --cli claude --scope project`로 제거할 수 있습니다. - - - -설치된 버전에서 사용 가능한 정책 목록 확인: - -```bash -failproofai policies -``` - -프로젝트에 정책 하나 활성화: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -선택한 하네스에 여러 정책 활성화: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -일부 정책은 파라미터를 허용하거나 베타로 표시됩니다. 롤아웃 전에 설명, 매치 범위, 기본 동작을 검토하세요. 한 워크플로우를 보호하는 정책이 다른 워크플로우에서는 유효한 작업을 차단할 수 있습니다. - - - 현재 40개의 정책 전체와 각 정책의 트리거, 권장 기준선, 파라미터를 검토하세요. - - - - 저장소별 요구 사항에는 프로젝트 범위를, 머신 전체 안전 요구 사항에는 사용자 범위를 사용하는 것을 권장합니다. - \ No newline at end of file diff --git a/docs/ko/policies/custom.mdx b/docs/ko/policies/custom.mdx deleted file mode 100644 index 448c7600a..000000000 --- a/docs/ko/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "커스텀 정책" -description: "에이전트 워크플로우에 고유한 실패 모드에 대한 정책을 작성하세요." -icon: "shield-plus" ---- - -`.failproofai/policies/` 하위에 `policies.js`, `policies.mjs`, 또는 `policies.ts`로 끝나는 파일을 생성하세요. 컨벤션 파일은 프로젝트 및 사용자 범위에서 자동으로 로드됩니다. - -## 클라우드 게시 전 정책 테스트 - - - - 1. 테스트 머신 한 대에 커스텀 정책을 설치하고, 매칭되는 액션과 정상적인 비매칭 액션을 각각 트리거합니다. - 2. **Observe → policy**로 이동하여 두 결정을 비교합니다. - 3. 연결된 각 세션을 열어 이벤트 페이로드에 규칙을 판단할 충분한 근거가 포함되어 있는지 확인합니다. - 4. 동작이 올바르면, 검토한 소스를 **Admin → policy editor**로 이동하여 버전을 게시합니다. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - `.failproofai/policies/` 하위의 컨벤션 파일은 `--custom` 없이 로드됩니다. 손상된 모듈에서 유효성 검사가 실패해야 하는 CI 환경에서는 명시적인 설치 명령을 유지하세요. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -이 정책은 `Write`와 `Edit` 모두에 대해 `production/config.yml`, `/srv/production/config.yml`, `/srv/production`, `C:\\production\\config.yml`에 매칭됩니다. `production`은 완전한 경로 세그먼트여야 하므로 `production-backup`과 같은 이름은 매칭되지 않습니다. - -명시적 파일의 유효성을 검사하고 설치합니다: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -정책 컨텍스트에는 이벤트 유형, 정규화된 페이로드, 도구 이름과 입력, 세션 메타데이터, 파라미터, 그리고 가능한 경우 소스 CLI가 포함됩니다. - -## 실패 경로 테스트 - -엔트리 파일이나 해당 파일이 임포트하는 로컬 모듈을 변경한 후 유효성 검사를 실행하세요: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -strict CLI 경로는 파일 누락, 구문 오류, 해결되지 않은 임포트, 최상위 예외, 모듈 로드 타임아웃 시 실패합니다. 적용 시점에 손상된 커스텀 파일은 기록되고 건너뛰어지므로 내장 정책은 계속 동작합니다. 로드 경고는 예상되는 적용의 손실로 간주하고, 프로덕션 로그에서 알림을 설정하세요. - -명시적, 컨벤션, 클라우드 관리 정책 전반에 걸쳐 전역적으로 고유한 이름을 사용하세요. 정책 함수는 결정론적으로 유지하고, 외부 호출은 짧은 타임아웃으로 제한하며, 모든 경로에서 의도적인 `allow`, `instruct`, 또는 `deny`를 반환하세요. - - - 커스텀 정책은 적용 코드입니다. 예상되는 매칭뿐만 아니라 누락된 필드, 대체 도구 이름, 잘못된 형식의 입력에 대해서도 테스트하세요. - \ No newline at end of file diff --git a/docs/ko/policies/fleet.mdx b/docs/ko/policies/fleet.mdx deleted file mode 100644 index ca9d45058..000000000 --- a/docs/ko/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "머신에 정책 배포하기" -description: "등록된 머신이 어떤 것인지, 최신 상태인지, 의도한 정책 버전을 적용하고 있는지 파악합니다." -icon: "network" ---- - -플리트 커버리지는 위험이 존재하는 곳에 정책이 존재하는지를 확인합니다. 머신을 안정적인 ID와 사람이 읽을 수 있는 레이블로 추적한 다음, 할당된 배포 상태와 실제 보고된 배포 상태를 비교합니다. - -## 커버리지 확인 - - - - 1. **Admin → enforcement** 로 이동하여 적용(enforcing) 및 관찰(observing) 합계를 검토합니다. - 2. ID 또는 레이블로 머신을 검색하거나, 정책이 없는 머신을 필터링합니다. - 3. 행을 펼쳐 할당된 정책, 보고된 배포 상태, 마지막 체크인 시각, 히스토리를 비교합니다. - 4. 적용된 배포가 보류 상태로 남아 있을 경우, 머신의 폴링 간격이 지난 후 새로고침합니다. - - ![정책 커버리지, 머신 배포 상태, 관찰 및 적용 할당을 보여주는 Enforcement 플리트 화면.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - `fp events --agent-id --since 24h` 명령을 사용하여 해당 머신의 에이전트 활동이 Cloud에 도달하는지 확인하세요. - - - -커버리지 뷰를 활용해 다음 항목을 확인할 수 있습니다: - -- 최신 배포를 한 번도 가져오지 않은 머신 -- 등록은 됐지만 활동 보고가 중단된 머신 -- 잘못된 환경 또는 코호트에 할당된 정책 -- 업데이트 중단 이후 발생한 버전 불일치 - -머신을 재연결하지 않고 이름 변경하기: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -로컬 상태 확인하기: - -```bash -failproofai config --status -``` - - - 워크로드와 환경을 식별할 수 있는 레이블을 사용하세요. 호스트명만으로는 오토스케일링이나 머신 교체 후 식별이 어려운 경우가 많습니다. - \ No newline at end of file diff --git a/docs/policies/builtin-catalog.mdx b/docs/policies/builtin-catalog.mdx deleted file mode 100644 index 75029a507..000000000 --- a/docs/policies/builtin-catalog.mdx +++ /dev/null @@ -1,132 +0,0 @@ ---- -title: "Builtin policy catalog" -description: "Review every builtin Failproof AI policy, its trigger, recommended state, and configurable parameters." -icon: "list-checks" ---- - -The installed package is the source of truth for policy availability. Run `failproofai policies` after every upgrade because catalog entries and behavior can change with the package version. - -## Recommended baseline - -The guided setup's recommended selection currently enables secret sanitizers, environment protections, self-protection, catastrophic-command guards, and protected-branch safety: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` is **always on**. It is listed above for -completeness, but it registers on every evaluation whether or not it appears in -your enabled set, and it cannot be disabled or paused — a guard against the -agent switching off enforcement that the agent can switch off is not a guard. - -Recommended is deliberately narrower than **Everything**. Infrastructure and workflow policies can interrupt valid work and should be enabled for the repositories and machines that need them. - -## Secrets and environment - -| Policy | Trigger | Outcome | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | Redact JWTs from tool output before the model sees them. | -| `sanitize-api-keys` | `PostToolUse` | Redact common OpenAI, Anthropic, GitHub, AWS, Stripe, and Google keys. | -| `sanitize-connection-strings` | `PostToolUse` | Redact database connection strings containing credentials. | -| `sanitize-private-key-content` | `PostToolUse` | Redact PEM private-key bodies. | -| `sanitize-bearer-tokens` | `PostToolUse` | Redact authorization bearer tokens. | -| `protect-env-vars` | `PreToolUse` on shell tools | Block commands that dump environment variables. | -| `block-env-files` | `PreToolUse` | Block reads and writes of `.env` files. | -| `block-read-outside-cwd` | `PreToolUse` on read, glob, grep, or shell tools | Keep reads inside the session working directory. | -| `block-secrets-write` | `PreToolUse` on write tools | Block writes to common secret-key and credential filenames. | - -## Dangerous commands and infrastructure - -| Policy | Trigger | Outcome | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | Block `sudo` unless an allow pattern matches. | -| `block-curl-pipe-sh` | `PreToolUse` | Block downloaded scripts piped directly to a shell. | -| `block-rm-rf` | `PreToolUse` | Block catastrophic recursive deletion patterns. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Always on, cannot be disabled.** Block every Failproof AI CLI invocation, self-pause, and package-manager uninstall. | -| `block-kubectl` | `PreToolUse` | Gate Kubernetes commands. | -| `block-terraform` | `PreToolUse` | Gate Terraform and OpenTofu commands. | -| `block-aws-cli` | `PreToolUse` | Gate AWS CLI commands. | -| `block-gcloud` | `PreToolUse` | Gate Google Cloud CLI commands. | -| `block-az-cli` | `PreToolUse` | Gate Azure CLI commands. | -| `block-helm` | `PreToolUse` | Gate Helm commands. | -| `block-gh-pipeline` | `PreToolUse` | Gate mutating GitHub CLI workflow, run, merge, release, cache, and secret operations. | - -## Git and database safety - -| Policy | Trigger | Outcome | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Block direct pushes to configured protected branches. | -| `block-force-push` | `PreToolUse` | Block force-pushes; `--force-with-lease` remains allowed by the current implementation. | -| `block-work-on-main` | `PreToolUse` | Block commits and merges on protected branches. | -| `warn-git-amend` | `PreToolUse` | Warn before rewriting a commit with `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | Warn before permanently dropping or clearing stashes. | -| `warn-all-files-staged` | `PreToolUse` | Warn on broad `git add -A`, `git add .`, or `git add --all`. | -| `warn-destructive-sql` | `PreToolUse` | Warn on `DROP`, `TRUNCATE`, and `DELETE` without `WHERE` through recognized database clients. | -| `warn-schema-alteration` | `PreToolUse` | Warn on recognized `ALTER TABLE` column and rename operations. | - -## Packages, system behavior, and agent loops - -| Policy | Trigger | Outcome | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Warn before publishing to package registries. | -| `warn-global-package-install` | `PreToolUse` | Warn before global package installation. | -| `prefer-package-manager` | `PreToolUse` | Instruct the agent to use an allowed package manager. | -| `warn-large-file-write` | `PreToolUse` on write tools | Warn above the configured file-size threshold. | -| `warn-background-process` | `PreToolUse` | Warn on detached or long-lived background-process patterns. | -| `warn-repeated-tool-calls` | `PreToolUse` | Warn after three or more identical tool calls. | - -## End-of-task workflow - -These policies require a harness that emits a compatible `Stop` event. - -| Policy | Outcome | -| --- | --- | -| `require-commit-before-stop` | Refuse completion while tracked work remains uncommitted. | -| `require-push-before-stop` | Refuse completion while commits remain local-only. | -| `require-pr-before-stop` | Require a pull request for the current branch. | -| `require-no-conflicts-before-stop` | Require a clean merge against the configured base branch. | -| `require-ci-green-before-stop` | Require current-HEAD CI checks to complete successfully. | - -## Parameter reference - -Configure parameters under the selected scope's `policyParams` object. Types are validated by each policy. - -| Policy | Parameter | Type and default | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; entries contain `regex` and `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Infrastructure blockers | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - An allow pattern broadens what an agent may do. Test the exact tokenization and command variants on the target harness before deploying it across a fleet. - diff --git a/docs/policies/builtin.mdx b/docs/policies/builtin.mdx deleted file mode 100644 index 1dd24e61b..000000000 --- a/docs/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Builtin policies" -description: "Enable maintained guardrails for common agent failure modes." -icon: "library" ---- - -Builtin policies cover secret handling, environment files, destructive shell commands, protected branches, cloud and infrastructure tools, package publishing, repeated calls, and end-of-task workflow checks. - -## Enable and verify a builtin policy - - - - 1. Install the policy on a connected machine with the local CLI. - 2. Run a safe test action in the instrumented agent. - 3. Go to **Observe → policy** and filter by the policy name, machine environment, or decision. - 4. Open the linked session to confirm the matched tool input and returned reason. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - Remove it with `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -List the policies available in your installed version: - -```bash -failproofai policies -``` - -Enable one policy for a project: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Enable several policies for selected harnesses: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Some policies accept parameters or are marked beta. Review the description, match scope, and default behavior before rollout. A policy that protects one workflow may block valid operations in another. - - - Review all 40 current policies, their triggers, recommended baseline, and parameters. - - - - Prefer project scope for repository-specific expectations and user scope for machine-wide safety requirements. - diff --git a/docs/policies/custom.mdx b/docs/policies/custom.mdx deleted file mode 100644 index 61ea52783..000000000 --- a/docs/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Custom policies" -description: "Write a policy for a failure mode unique to your agent workflow." -icon: "shield-plus" ---- - -Create a file ending in `policies.js`, `policies.mjs`, or `policies.ts` under `.failproofai/policies/`. Convention files load automatically at project and user scope. - -## Test the policy before Cloud publication - - - - 1. Install the custom policy on one test machine and trigger both a matching action and a legitimate non-match. - 2. Go to **Observe → policy** and compare the two decisions. - 3. Open each linked session and verify the event payload contains enough evidence for the rule. - 4. When the behavior is correct, move the reviewed source into **Admin → policy editor** and publish a version. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - Convention files under `.failproofai/policies/` load without `--custom`. Keep an explicit install command in CI when validation should fail on a broken module. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -This matches `production/config.yml`, `/srv/production/config.yml`, `/srv/production`, and `C:\\production\\config.yml` for both `Write` and `Edit`. It does not match names such as `production-backup` because `production` must be a complete path segment. - -Validate and install an explicit file: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -The policy context includes the event type, normalized payload, tool name and input, session metadata, parameters, and source CLI when available. - -## Test failure paths - -Run validation after changing the entry file or any local module it imports: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -The strict CLI path fails for missing files, syntax errors, unresolved imports, top-level exceptions, and module-load timeouts. At enforcement time, a broken custom file is logged and skipped so builtin policies can continue. Treat any load warning as a loss of expected enforcement and alert on it in production logs. - -Use globally unique names across explicit, convention, and Cloud-managed policies. Keep policy functions deterministic, bound external calls with short timeouts, and return an intentional `allow`, `instruct`, or `deny` on every path. - - - A custom policy is enforcement code. Test missing fields, alternate tool names, and malformed input—not only the expected match. - diff --git a/docs/policies/deploy.mdx b/docs/policies/deploy.mdx index c918125dc..a6c7cfcfd 100644 --- a/docs/policies/deploy.mdx +++ b/docs/policies/deploy.mdx @@ -1,50 +1,93 @@ --- -title: "Deploy policies" -description: "Roll out a reviewed policy version to the intended machines." +title: "Deploy a policy" +description: "Put a tested policy version on machines in observe mode, enforce it, and confirm every machine picked it up." icon: "cloud-upload" --- -A deployment connects one or more policy versions to a target set of enrolled machines. +A deployment puts published policy versions on a machine, each with one of two effects: -## Apply a deployment +- **Observe** records what the policy would have done, and blocks nothing. +- **Enforce** acts on the decision: a `deny` blocks the call and an `instruct` steers the agent. + +## Add a machine + +A machine appears under **Admin → enforcement** once it is connected to Cloud. If the one you want is not there yet: + + + + 1. Go to **Administration → Keys** and create a key with `policies:pull`, so the machine can receive deployments, and `events:add`, so its decisions reach Cloud. + 2. Connect the machine with that key — [Connect a machine to Cloud](/start/setup#connect-a-machine-to-cloud) walks through it. + 3. Confirm it shows up under **Admin → enforcement**. + + + On the machine: + + ```bash + npm install -g failproofai + failproofai config + failproofai config --status + ``` + + In a terminal, `failproofai config` asks whether to connect to Cloud and takes the key at a masked prompt. Then confirm the machine is enrolled, from anywhere, with `fp fleet list`. + + + +## Deploy in observe mode 1. Go to **Admin → enforcement**, find the machine, and expand its row. - 2. Select **edit**, add the reviewed policy version, and choose **observe** or its enforcing effect. + 2. Select **edit**, add the tested policy version, and choose **observe**. 3. Apply the change, then wait for the machine's next check-in and confirm its deployment and coverage state. 4. Go to **Observe → policy** to inspect live decisions. ![The machine deployment editor with policy versions, enforce and observe effects, and the apply deployment action.](/images/dashboard/enforcement-editor.png) - Deploy from the CLI with `fp fleet`. Review the resulting set before applying it — `deploy` prints the full plan and asks **only on an interactive terminal without `--json`**. Under `--json`, with `--yes`, or with stdin redirected (a CI step, a script, an agent shelling out) it applies immediately with no plan and no prompt — so run `fp fleet show ` first if you want review: - ```bash fp fleet list fp fleet show - fp fleet deploy --add no-force-push + fp fleet deploy --add no-force-push:observe ``` - `fp fleet diff ` shows intent vs delivery (a machine reads as `behind` until it next polls), `fp fleet history ` lists the generations, and `fp fleet rollback ` reinstates one — it refuses if that generation names a policy since disabled or deleted. + The `:observe` suffix is what makes it observe: a bare `--add no-force-push` keeps the effect the machine already has for that policy, and otherwise enforces. Switch it to enforce later with `--add no-force-push:enforce`. + + `deploy` **replaces the machine's whole policy set** with the result. It prints the plan, then asks before applying — but only on an interactive terminal. With `--yes`, under `fp --json`, or with stdin redirected (a CI step, a script, an agent shelling out) it applies without asking; the plan is still printed, or returned as `plan` under `--json`. - Check the machine itself with `failproofai config --status`, and use `fp sessions --env production --since 24h` and `fp events --event-type hook_completed` after deployment to verify activity reaches Cloud. + On the machine, `failproofai policies` lists the Cloud-managed policies it is running and `failproofai config --status` shows its connection. Use `fp sessions --env production --since 24h` and `fp events --event-type hook_completed` to confirm its activity reaches Cloud. - - Deploy a reviewed version, not a mutable draft, starting with a non-production machine or small cohort whose sessions you can inspect. + + Pick the published version and the machines it should run on. - - Review matches, reasons, affected tools, and false positives without blocking work. + + Review matches, reasons, affected tools, and false positives while nothing is blocked. - - Promote after observed matches separate unsafe actions from valid ones, then confirm every intended machine has pulled the deployment and is reporting decisions. + + Switch the effect to enforce once the observed matches separate unsafe actions from valid ones, then confirm every intended machine pulled the change and is reporting decisions. -Machines need the `policies:pull` capability. Event reporting is separately controlled by `events:add`; verify both when you expect Cloud analysis and enforcement. +## Check coverage + +Coverage answers whether a policy is running where the risk is. + +1. Go to **Admin → enforcement** and review the enforcing and observing totals. +2. Search for a machine by ID or label, or filter for machines missing a policy. +3. Expand a row to compare assigned policies, reported deployment, last check-in, and history. +4. Refresh after the machine's polling interval when an applied deployment is still pending. + +![The Enforcement fleet showing policy coverage, machine deployment state, and observe and enforce assignments.](/images/dashboard/enforcement-fleet.png) + +Look for machines that never pulled the latest deployment, enrolled machines that stopped reporting, a policy assigned to the wrong environment, and version drift after an interrupted update. + +Label machines by workload and environment — hostnames alone rarely survive autoscaling or replacement: + +```bash +failproofai config --machine-label checkout-runner-03 +``` Enforcement management is an administrative Cloud workflow. Do not treat root-only enforcement routes as ordinary customer `/v1` API endpoints. diff --git a/docs/policies/editor.mdx b/docs/policies/editor.mdx index e0f0019d9..0363bee7c 100644 --- a/docs/policies/editor.mdx +++ b/docs/policies/editor.mdx @@ -1,49 +1,96 @@ --- -title: "Policy editor" -description: "Create and revise versioned policies from a confirmed failure mode." +title: "Write a policy" +description: "Let Failproof AI draft a policy from an audit finding, or write the source yourself, then review, test, and publish it." icon: "file-pen-line" --- -Use the policy editor to turn a finding or issue into a deployable rule. Keep authoring separate from deployment so a draft cannot silently change live behavior. +There are two ways to write a policy: let Failproof AI draft it from an audit finding, or write the source yourself. Nothing is published or deployed until you choose to. -When an issue has a repeatable action pattern, open it under **Analyze → issues** and select **generate policy**. Failproof AI first explains whether a policy can express the problem, then carries the reviewed intent and finding context into the editor. The generated source remains a draft until you publish it. +## Write a policy from an audit -## Publish a policy version +An audit finds a failure; a policy stops it happening again. Failproof AI drafts the policy from the finding's own evidence. + +### 1. Run an audit + +[Run an audit](/audits/run) over the sessions where the failure happens. Each finding carries its evidence sessions, a root cause, and a suggested prevention path. Work from a finding with a **repeatable action pattern** — a policy can only stop what it can recognize in a hook event. + +### 2. Generate the draft - 1. Go to **Admin → policy editor** and, in **compose**, describe the failure mode or paste the JavaScript policy source. - 2. Validate the source and fix every reported error. - 3. Enter the policy identity and publish it, then use **library** to compare or disable versions. - 4. Select **enforcement** when the version is ready for a machine rollout. + 1. Open the finding's issue under **Analyze → issues** and check its cited sessions, root cause, and recommendation. + 2. Select **generate policy**. Failproof AI first says whether a policy can express the problem at all. A **no policy** result means the fix is an alert, a workflow change, or a person — not a policy. + 3. Select **write this policy**. The issue title, finding, root cause, recommendation, and proposed enforcement intent become a draft in **Admin → policy editor**. Use **open the editor anyway** when you disagree with the candidacy check. ![The Policy editor compose view with policy identity, AI-assisted drafting, source validation, and publishing controls.](/images/dashboard/policy-editor.png) - Publish from the CLI with `fp policies publish`. It mints a **new version** and never edits one in place, and it parse-checks the source with node before sending — nothing downstream does, so a syntax error would otherwise surface on the machine at enforcement time: + Read the evidence, then draft with the assistant. `compose` prints source for you to review and publishes nothing: ```bash - fp policies test ./checkout.policy.mjs --command "git push --force" --expect deny - fp policies publish checkout-guard ./checkout.policy.mjs --description "Block force-push" + fp issues show + fp audits finding + fp policies compose "Block git push --force on release branches" ``` - Publishing deploys nothing — a new version sits unused until `fp fleet deploy` puts it on a machine. `fp policies compose ""` drafts source with the Cloud assistant and prints it for review rather than publishing it. - - To install a policy into a local agent CLI instead (not Cloud), use `failproofai policies --install --custom ./checkout.policies.ts --cli claude --scope project`. + `compose` needs a signed-in session (`fp login`) whose role has `policies:write`; it refuses API keys. -## Authoring checklist +### 3. Review the draft + +A draft is a starting point, not a verdict. Before publishing, check that it: + +1. Names the failure mode in operational language. +2. Matches only the hook events and tools that carry enough evidence to decide. +3. Uses the narrowest condition that catches the unsafe action. +4. Returns a reason that tells the agent what to do instead. +5. Uses `instruct` where the agent can safely correct course, and `deny` only where allowing the action is unacceptable or irreversible. + +Validate the source in the editor and fix every reported error. + +### 4. Test it, then publish + +Run **backtest** under the source before you publish: it replays the draft against calls your fleet already made and counts the working calls it would have interrupted. [Test a policy](/policies/test) covers that and the other checks. + +When it behaves, enter the policy identity and select **publish version**. Publishing mints an immutable version and deploys nothing: it sits unused until you [deploy it](/policies/deploy). From a terminal: + +```bash +fp policies publish checkout-guard ./checkout.policy.mjs --description "Block force-push" +``` + +`publish` parse-checks the source before sending it, so a syntax error surfaces here instead of on a machine at enforcement time. + +## Write it yourself + +A policy is JavaScript or TypeScript against the `failproofai` API: + +```ts +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "protect-production-paths", + description: "Block writes to production configuration", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); + if (path.split("/").includes("production")) { + return deny("Writes to production configuration require approval."); + } + return allow(); + }, +}); +``` + +This matches `production/config.yml`, `/srv/production/config.yml`, `/srv/production`, and `C:\\production\\config.yml` for both `Write` and `Edit`, but not `production-backup`: `production` has to be a whole path segment. The context also carries the event type, normalized payload, session metadata, parameters, and source CLI when available — see the [policy SDK](/reference/policy-sdk). + +To publish it as a version, paste the source into **compose** in **Admin → policy editor** and follow steps 3 and 4 above, or publish the file from a terminal with `fp policies publish`. -1. Name the failure mode in operational language. -2. Select the hook events and tools that contain enough evidence to decide. -3. Write the narrowest condition that matches unsafe behavior. -4. Return a reason that tells the agent or operator what to do next. -5. Add examples that should match and examples that must remain allowed. -6. Save a new version and request review. +To run it on a machine without Cloud, save it under `.failproofai/policies/` with a name ending in `policies.js`, `policies.mjs` or `policies.ts` — those load automatically at project and user scope — or install it by path: -Use `instruct` when the agent can safely correct course. Use `deny` when allowing the action would create unacceptable or irreversible risk. +```bash +failproofai policies --install --custom ./security.policies.ts --scope project +``` - - Policy versions are immutable deployment inputs. Editing a draft creates a new version; it should not rewrite the version already assigned to machines. - +Give every policy a name that is unique across convention, custom, pack, and Cloud-managed policies. diff --git a/docs/policies/failure-behavior.mdx b/docs/policies/failure-behavior.mdx index 15865cd3c..43f9058ca 100644 --- a/docs/policies/failure-behavior.mdx +++ b/docs/policies/failure-behavior.mdx @@ -61,7 +61,9 @@ It does not fire for: ### What to do ```bash -failproofai pack list +failproofai policies ``` -It names any installed pack that will not load, says why, and exits non-zero. Then either reinstall it (`failproofai pack add `) or remove it (`failproofai pack remove `) — removing it withdraws the expectation, and the deny stops with it. +The listing flags an installed pack whose install record or digest no longer checks out, and says why. It does not import the pack, so one that fails only once it loads — registering less than its manifest declares — lists as normal; the deny below is what names that one. Either way, reinstall it (`failproofai policies add `) or remove it (`failproofai policies remove `) — removing it withdraws the expectation, and the deny stops with it. + +The deny itself is attributed to `pack/failproofai-pack-unavailable`, which outranks the policies that did load, so a blocked tool call names the missing pack rather than whichever surviving guard happened to fire first. diff --git a/docs/policies/fleet.mdx b/docs/policies/fleet.mdx deleted file mode 100644 index e4f337092..000000000 --- a/docs/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Deploy policies to machines" -description: "Know which machines are enrolled, current, and enforcing the intended policy versions." -icon: "network" ---- - -Fleet coverage answers whether a policy exists where the risk exists. Track machines by stable ID and a human-readable label, then compare their assigned and reported deployment state. - -## Check coverage - - - - 1. Go to **Admin → enforcement** and review the enforcing and observing totals. - 2. Search for a machine by ID or label, or filter for machines missing a policy. - 3. Expand a row to compare assigned policies, reported deployment, last check-in, and history. - 4. Refresh after the machine's polling interval when an applied deployment remains pending. - - ![The Enforcement fleet showing policy coverage, machine deployment state, and observe and enforce assignments.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Use `fp events --agent-id --since 24h` to confirm the machine's agent activity reaches Cloud. - - - -Use coverage views to find: - -- Machines that never pulled the latest deployment -- Enrolled machines that stopped reporting activity -- A policy assigned to the wrong environment or cohort -- Version drift after an interrupted update - -Rename a machine without reconnecting it: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Check local state: - -```bash -failproofai config --status -``` - - - Use labels that identify workload and environment. Hostnames alone are often insufficient after autoscaling or machine replacement. - diff --git a/docs/policies/local-configuration.mdx b/docs/policies/local-configuration.mdx index 3af2f8b85..21ce91b1c 100644 --- a/docs/policies/local-configuration.mdx +++ b/docs/policies/local-configuration.mdx @@ -4,30 +4,25 @@ description: "Control policy scope, parameters, custom files, and machine-level icon: "file-cog" --- -Failproof AI separates policy selection from machine and daemon settings. This keeps repository policy choices reviewable while credentials and daemon state stay outside the repository. +Failproof AI keeps what a repository can commit — hook wiring, policy parameters, custom policies — apart from machine state such as credentials, installed packs, and the daemon. -## Choose a policy scope +## Choose a scope - - - Run `failproofai` without arguments to open the local policy dashboard. Choose the user, project, or local scope before enabling a policy so the change is written to the intended configuration file. +A scope decides where the hooks are wired, and which configuration file you write parameters and custom policy paths into: - - **User** applies across projects on this machine. - - **Project** belongs to the repository and can be committed. - - **Local** overrides one project for one user and should remain gitignored. +- **User** applies across projects on this machine. +- **Project** belongs to the repository and can be committed. +- **Local** overrides one project for one user and should remain gitignored. - - - ```bash - failproofai policy add block-rm-rf --scope user - failproofai policy add block-force-push --scope project - failproofai policy add warn-large-file-write --scope local - failproofai policies - ``` +```bash +failproofai policies --install --cli claude --scope project # wire hooks for this repository +failproofai policies --install --cli claude --scope user # or for every project on this machine +failproofai policies +``` - Not every harness supports local scope. The CLI rejects a scope that the selected harness cannot represent. - - +Not every harness supports local scope; the CLI rejects a scope the selected harness cannot represent. + +Which pack policies are on is **not** scoped. The switch is recorded with the installed pack, so `failproofai policies add ` turns a policy on for the whole machine, whatever `--scope` says. | Scope | Policy configuration file | | --- | --- | @@ -35,7 +30,7 @@ Failproof AI separates policy selection from machine and daemon settings. This k | Local | `/.failproofai/policies-config.local.json` | | User | `~/.failproofai/policies-config.json` | -Enabled policies are merged as a union. Policy parameters use the first scope that defines parameters for that policy, in project → local → user order. Explicit custom policy paths use the first scope that defines them. +Policy parameters use the first scope that defines parameters for that policy, in project → local → user order. Explicit custom policy paths use the first scope that defines them. ## Configure policy parameters @@ -45,11 +40,10 @@ Enabled policies are merged as a union. Policy parameters use the first scope th - Edit the selected scope's `policies-config.json`, then run `failproofai policies` to surface unknown policy names or parameter keys. + Edit the selected scope's `policies-config.json`, then run `failproofai policies`: it warns about a `policyParams` entry naming a policy no installed pack carries. It does not check the keys inside an entry, so check their spelling against the table below. ```json { - "enabledPolicies": ["block-rm-rf", "block-force-push"], "policyParams": { "block-rm-rf": { "allowPaths": ["/tmp/build-output"] @@ -64,6 +58,30 @@ Enabled policies are merged as a union. Policy parameters use the first scope th +### Parameters the Failproof AI policies accept + +Each policy validates its own parameter types. + +| Policy | Parameter | Type and default | +| --- | --- | --- | +| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; entries contain `regex` and `label` | +| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | +| `block-sudo` | `allowPatterns` | `string[]`, `[]` | +| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | +| Infrastructure blockers | `allowPatterns` | `string[]`, `[]` | +| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | +| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | +| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | +| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | +| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | +| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | +| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | +| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | + + + An allow pattern broadens what an agent may do. Test the exact tokenization and command variants on the target harness before deploying it across a fleet. + + ## Understand the machine files `~/.failproofai` contains separate files for separate trust boundaries: @@ -72,8 +90,8 @@ Enabled policies are merged as a union. Policy parameters use the first scope th | --- | --- | | `config.json` | Non-secret daemon, audit, and telemetry settings | | `credentials.json` | Cloud credentials; stored with owner-only permissions | -| `policies-config.json` | User-scope builtin selection, parameters, and explicit custom paths | -| `policies/` | User convention policies and Cloud-managed policy artifacts | +| `policies-config.json` | User-scope parameters and explicit custom policy paths | +| `policies/` | User convention policies, installed packs and which of their policies are on, and Cloud-managed policy artifacts | | `hook-activity/` | Local policy decision log | | `state/` | Daemon spool, health, pause, and runtime state | diff --git a/docs/policies/overview.mdx b/docs/policies/overview.mdx index f9a5a0d70..83b83b3a7 100644 --- a/docs/policies/overview.mdx +++ b/docs/policies/overview.mdx @@ -10,54 +10,45 @@ A policy evaluates an agent hook event and returns one of three decisions: - `instruct` gives the agent corrective guidance. - `deny` blocks the action with a reason. -## Use the three policy surfaces +## Where policies live - - - 1. Go to **Observe → policy** to filter and inspect policy decisions from sessions. - 2. Go to **Admin → policy editor** to compose, validate, publish, disable, or inspect immutable versions. - 3. Go to **Admin → enforcement** to assign versions and effects to machines. +| In the dashboard | What you do there | +| --- | --- | +| **Observe → policy** | Review decisions from real sessions: which policy matched, on which machine, and why | +| **Admin → policy editor** | Write a policy, backtest it against past traffic, publish an immutable version, and compare versions in **library** | +| **Admin → enforcement** | Put versions on machines, in observe or enforce mode | - Use the Policy page to understand what is already matching before authoring or changing enforcement. +The policy editor is where a failure becomes a rule. Describe the failure mode or paste policy source in **compose**, backtest the draft against traffic you already have, and publish a version: - ![The Policy page showing decision totals and local and Cloud-managed policy mappings.](/images/dashboard/policy-observe.png) +![The Policy editor compose view with policy identity, AI-assisted drafting, source validation, and publishing controls.](/images/dashboard/policy-editor.png) - The editor is where you turn a failure condition into source, validate it, and publish an immutable version. +On a machine, `failproofai policies` lists everything enforcing there. `fp policies` and `fp fleet` cover the editor and enforcement from a terminal — see the [Cloud CLI reference](/reference/cloud-cli). - ![The Policy editor used to compose and publish an immutable policy version.](/images/dashboard/policy-editor.png) +## Get a policy - Enforcement then assigns that published version and its observe or enforce effect to machines. - - ![The Enforcement fleet showing machine coverage and assigned policy versions.](/images/dashboard/enforcement-fleet.png) - - Verify decisions back on the Policy page after deployment so the authoring and fleet views are tied to real agent activity. - - - Use `failproofai` for local policy installation and validation: - - ```bash - failproofai policies - failproofai policy add block-rm-rf --scope project - failproofai config --status - ``` - - Use `fp` to find the Cloud sessions and events containing policy decisions. Cloud authoring and fleet deployment remain dashboard workflows. - - - -Policies have three distinct surfaces in Failproof AI: - -1. **Analyze decisions** in sessions, dashboards, and audits. -2. **Author versions** with builtin rules, code, or the policy editor. -3. **Deploy and enforce** versions across selected machines. - -Start from a confirmed failure mode. Define the smallest event and tool match that identifies it, test legitimate and unsafe examples, then observe before enforcing. +There are two ways to get one. - - Enable a reviewed rule for common secret, shell, Git, cloud, and workflow risks. + + Let Failproof AI draft one from an audit finding, or write the source yourself, then review and publish it in the editor. - - Express a workflow-specific decision in JavaScript or TypeScript. + + Plug in a Failproof AI policy pack for your use case, or a community pack from the policy hub, in one command. + +## Then ship it + + + + Backtest the draft against traffic you already have, and run it against an action it must stop and one it must allow — all before you publish. See [Test a policy](/policies/test). + + + Put the version on machines in **observe** mode, read its decisions, then enforce. See [Deploy a policy](/policies/deploy). + + + Every publish is a new, immutable version, so a rollout that blocks valid work is undone by redeploying the last good one. See [Versions and rollback](/policies/rollback). + + + +To share your policies with other teams, [publish them as a pack](/policies/publish-a-pack). For what happens when a policy cannot be evaluated at all, see [Failure behavior](/policies/failure-behavior). diff --git a/docs/policies/packs.mdx b/docs/policies/packs.mdx index 8f1db2d19..196a788df 100644 --- a/docs/policies/packs.mdx +++ b/docs/policies/packs.mdx @@ -1,41 +1,59 @@ --- -title: "Policy packs" -description: "Install a set of policies published as a GitHub release, and manage what it enforces." +title: "Use a policy pack" +description: "Plug in a Failproof AI policy pack for your use case, or a community pack from the policy hub, and choose what it enforces." icon: "package" --- -A pack is a set of policies published as a GitHub release. One command installs it, the release's own checksums are verified before anything runs, and the digest is recorded so the pack cannot change under your machine afterwards. +A pack is a set of policies published as a GitHub release. One command installs it: the release's checksums are verified before anything runs, and its digest is recorded so the pack cannot change under your machine afterwards. -## Install the Failproof AI policies +Browse every pack, and every policy in each, on the [policy hub](https://befailproof.ai/policy-hub/). There are two kinds: + +- **Failproof AI policy packs** — ready-made packs for predefined use cases: plug one in and it works. The [coding agent policy pack](https://befailproof.ai/policy-hub/failproofai/policies/) is available now, and packs for more use cases are coming soon. +- **Community policy packs** — policies developers have written for their own use cases and published for anyone to take. + +## Failproof AI policy packs + +### Coding agent policy pack ```bash -failproofai pack add core +failproofai policies add FailproofAI/policies ``` -That installs the set we publish, from the copy inside the package — so it needs no network and cannot fail behind a proxy. Take part of it: +The pack carries 38 policies and switches on the 10 its manifest marks safe to enable unattended; the rest are listed for you to choose from. Some of the most used, and whether a plain `policies add` switches them on: + +| Policy | What it does | On by default | +| --- | --- | --- | +| `block-push-master` | Blocks direct pushes to protected branches | Yes | +| `block-env-files` | Blocks reading and writing `.env` files | Yes | +| `protect-env-vars` | Blocks commands that dump environment variables | Yes | +| `block-sudo` | Blocks `sudo` unless an allow pattern matches | Yes | +| `block-curl-pipe-sh` | Blocks downloaded scripts piped straight into a shell | Yes | +| `sanitize-*` (five policies) | Report API keys, bearer tokens, JWTs, private keys, and connection strings found in tool output | Yes | +| `block-rm-rf` | Blocks catastrophic recursive deletes | No | +| `block-force-push` | Blocks force-pushes | No | +| `block-secrets-write` | Blocks writes to credential and secret-key files | No | +| `warn-destructive-sql` | Warns on `DROP`, `TRUNCATE`, and `DELETE` without `WHERE` | No | + +Switch on any that are off by name — `failproofai policies add block-rm-rf` — or take the whole pack with `--all`. See every policy in it, grouped by category: ```bash -failproofai pack add core --policy block-rm-rf # one, or a comma-separated few -failproofai pack add core --category dangerous-commands # a whole category -failproofai pack add core --all # everything in it +failproofai policies show FailproofAI/policies ``` -`failproofai pack list` names every category the pack offers. +## Community policy packs -## See what a pack contains, before installing it +Developers publish packs for the use cases they have met, and the [policy hub](https://befailproof.ai/policy-hub/) lists them. A community pack is published by its author, not audited by Failproof AI, so read what it carries before installing it: ```bash -failproofai pack list acme/support-agent +failproofai policies show acme/support-agent ``` -Lists every policy the pack carries, grouped by category, marking which ones its author switches on by default and which are opt-in. It reads **only the manifest** — the entry artifact is never downloaded and never imported, so looking at a stranger's pack cannot run a stranger's code. The manifest is still checked against the release's own `SHA256SUMS`, so what you are reading is what would install. - -`failproofai pack list` with no source lists the packs already installed here. +That lists every policy it carries, grouped by category, and marks which ones its author switches on by default. It reads **only the manifest** — the entry artifact is never downloaded or imported, so looking at a stranger's pack cannot run a stranger's code. The manifest is still checked against the release's own `SHA256SUMS`, so what you read is what would install. -## Install somebody else's pack +Then install it: ```bash -failproofai pack add acme/support-agent +failproofai policies add acme/support-agent ``` Any of these work — paste whichever you have: @@ -54,41 +72,32 @@ Naming no tag installs the newest release **and pins it**, then tells you which By default you get the pack's **own** defaults — the policies its author marked safe to switch on unattended — not everything it contains. ```bash -failproofai pack add acme/support-agent --category billing,git -failproofai pack add acme/support-agent --policy block-refunds -failproofai pack add acme/support-agent --all +failproofai policies add FailproofAI/policies --policy block-rm-rf # one, or a comma-separated few +failproofai policies add FailproofAI/policies --category dangerous-commands # a whole category +failproofai policies add FailproofAI/policies --all # everything in it ``` -`--category` and `--policy` combine as a union (`--only` is accepted as a synonym for `--policy`). Re-adding at a newer version keeps whatever you chose rather than switching the rest back on. +`--category` and `--policy` combine as a union (`--only` is accepted as a synonym for `--policy`). When the pack is already installed, the flags add to what you had, and re-adding it with no flag and no terminal — to upgrade, say — keeps your selection as it is. At a terminal with no flag, `add` opens the picker instead, pre-ticked with the author's defaults, and what you tick replaces your selection. ## Manage what is on ```bash -failproofai policies # every source in one list, packs included -failproofai pack list # packs only, grouped by category +failproofai policies # every source in one list, packs included +failproofai policies add block-rm-rf # switch one policy on failproofai policies --uninstall block-refunds # turn one pack policy off failproofai policies --install block-refunds # and back on -failproofai pack remove acme/support-agent +failproofai policies remove acme/support-agent # uninstall the pack ``` -A bare name means the **builtin** when one exists by that name. Name a pack's copy explicitly when you need to: +Switching a pack policy on or off applies to the whole machine: the switch is recorded with the installed pack, not in a project's configuration, whatever `--scope` says. + +A name with no slash is a policy; anything with one is a pack source. A bare name resolves to the installed pack that declares it. When two installed packs declare the same name, name the one you mean: ```bash failproofai policies --uninstall acme/support-agent:block-refunds ``` - -If a pack ships a policy whose name is also an **enabled builtin**, the builtin runs and the pack's copy is skipped — the same guard would otherwise be evaluated twice. Turn the builtin off to use the pack's copy instead. - - -## Where the Failproof AI policies come from - -`core` reads the copy vendored in the npm package. The same set is published as a GitHub release, which is what you install if you want a specific version: - -```bash -failproofai pack add core # from this package, no network -failproofai pack add FailproofAI/policies # the same set, from its GitHub release -``` +Scopes, parameters, and the files these commands write are covered in [local configuration](/policies/local-configuration). ## What integrity does and does not buy @@ -98,7 +107,7 @@ At install time the pack is also **imported once** and checked against its own m ## When a pack will not load -A pack this machine was told to enforce and cannot run **denies** the events its missing policies covered, rather than allowing them silently. See [Failure behavior](/policies/failure-behavior). `failproofai pack list` names any pack in that state and exits non-zero. +A pack this machine was told to enforce and cannot run **denies** the events its missing policies covered, rather than allowing them silently — as `pack/failproofai-pack-unavailable`, which outranks the policies that did load so the deny is attributed to the missing pack rather than to whichever guard happened to fire first. The exception is `UserPromptSubmit`, which instructs instead: denying there would lock you out of the agent you need in order to fix it. See [Failure behavior](/policies/failure-behavior). ## Offline and mirrors @@ -107,4 +116,4 @@ A pack this machine was told to enforce and cannot run **denies** the events its | `FAILPROOFAI_NO_DOWNLOAD=1` | Refuses to fetch; packs already installed keep enforcing | | `FAILPROOFAI_PACK_BASE_URL` | Points pack fetching at a mirror instead of `github.com` | -Publishing your own pack: see [Publish a pack](/policies/publish-a-pack). +To share your own policies this way, see [Publish a policy pack](/policies/publish-a-pack). diff --git a/docs/policies/publish-a-pack.mdx b/docs/policies/publish-a-pack.mdx index 54a977e10..0ca827f4b 100644 --- a/docs/policies/publish-a-pack.mdx +++ b/docs/policies/publish-a-pack.mdx @@ -1,14 +1,22 @@ --- -title: "Publish a pack" +title: "Publish a policy pack" description: "Ship your own policies as a GitHub release that anyone can install." icon: "upload" --- -A pack is three files attached to a GitHub release. `failproofai pack build` writes all three from a policy file you already have. +A pack is three files attached to a GitHub release. `failproofai publish` writes all three from the policy files in front of it, creates the release, and uploads them. ## 1. Write the policies -One file, using the same API as any custom policy. Two extra fields matter for a pack: +Start from something that already works rather than a template with blanks: + +```bash +failproofai publish --init +``` + +That asks what the pack is called, writes `.mjs`, and stops — no network, no git, nothing published. The file it writes is one policy that already blocks `git push --force`. It refuses to overwrite a file that exists. + +Policies use the same API as any custom policy. Two extra fields matter for a pack: ```js import { customPolicies, deny, allow } from "failproofai"; @@ -17,7 +25,7 @@ customPolicies.add({ name: "block-refunds", description: "Refunds above the approved limit need a human", category: "Billing", // groups it, and is what --category selects on - defaultEnabled: true, // switched on by a plain `pack add` + defaultEnabled: true, // switched on by a plain `policies add` match: { events: ["PreToolUse"], tools: ["Bash"] }, fn: async (ctx) => String(ctx.toolInput?.command ?? "").includes("refund") @@ -26,53 +34,80 @@ customPolicies.add({ }); ``` -`defaultEnabled` defaults to **false** when you omit it. A plain `failproofai pack add` switches on only what you marked — installing a stranger's every policy unattended is not a decision the installer should make for its user. +`defaultEnabled` defaults to **false** when you omit it. A plain `failproofai policies add` switches on only what you marked — installing a stranger's every policy unattended is not a decision the installer should make for its user. + +Write as many files as you like; one per category reads well. Every file in the directory that registers policies is bundled into the single artifact a pack has to be. -The entry must be **one self-contained file**. Only the entry is digest-pinned, so a pack that imports local files could not honestly claim the digest covers what runs. Bundle first (`esbuild`, `bun build`, `rollup`) and build the pack from the bundle — `pack build` refuses a local import rather than shipping a promise it cannot keep. + Bundling needs **bun**. Without it, keep to one self-contained file. Either way the published entry must not import local files at install time: only the entry is digest-pinned, so a pack that reached for siblings could not honestly claim the digest covers what runs — and `publish` refuses one rather than shipping a promise it cannot keep. -## 2. Build the release assets +## 2. Try it here first + +Before anyone else can see it, enforce the file on this machine: ```bash -failproofai pack build ./policies.mjs \ - --id acme/support-agent \ - --version 1.0.0 \ - --out ./dist-pack +failproofai policies -i -c ./.mjs ``` -It writes three files, and validates every policy with the **loader's own rules** first — so a pack that could never install fails here, where you can fix it: +Any path, any filename. Ask your agent to do the thing you blocked and watch it get refused. Nothing is published and nobody else is affected. [Test a policy](/policies/test) covers the rest: the legitimate case it must allow, and the inputs that break it. + +## 3. Publish it + +```bash +failproofai publish +``` + +It works out where to publish, what to bundle and what version to call it, and only asks when nothing in the repository tells it. In order, stopping before it creates a release if anything is wrong: + +1. Finds the policy files here by **content** — those that import `failproofai` and call `customPolicies.add` — rather than by filename, so it finds `guards.mjs` and ignores an unrelated `policies.mjs`. It does not descend into subdirectories, so a test fixture is never swept up by accident. +2. Reads the repo from `git remote get-url origin`, in the **file's** directory rather than yours, and decides the version. +3. Finds your credential: `GITHUB_TOKEN`, `GH_TOKEN`, or `gh auth login`. It needs release-write and nothing else, and is never printed. +4. Creates the repository if it does not exist. This happens before the build, so a pack refused in the next step can leave a new repository behind with no release in it. +5. Builds the three assets, validating them with the **loader's own rules** — the same code that decides what may install on a stranger's machine — so a pack that could never install fails here, where you can still fix it. +6. Creates or reuses the release and uploads, replacing assets of the same name. | File | What it is | | --- | --- | | `failproofai-pack.json` | The manifest: id, version, effect, and one entry per policy | -| `failproofai-pack.mjs` | Your entry, verbatim | +| `failproofai-pack.mjs` | Your bundled entry | | `SHA256SUMS` | ` ` for the other two | -Refused at build time: an id that is not `publisher/name`, a policy name containing `/`, a policy declaring `alwaysOn`, a missing `description`, `category` or `match`, an entry that registers nothing, and an entry that imports local files. +The asset names are fixed — they are what a consumer's CLI constructs its URLs from, with no API call and no discovery. -## 3. Attach them to a release +Refused at build time: an id that is not `publisher/name`, a policy name containing `/`, a policy declaring `alwaysOn`, a missing `description`, `category` or `match`, an entry that registers nothing, and an entry that imports local files. -Tag the release with the same version you built, and attach all three files as release assets: +Override anything it decided: ```bash -gh release create 1.0.0 \ - ./dist-pack/failproofai-pack.json \ - ./dist-pack/failproofai-pack.mjs \ - ./dist-pack/SHA256SUMS +failproofai publish \ + --repo acme/support-agent \ + --version 1.0.0 \ + --effect observe \ + --dry-run ``` -Anyone can now install it: +`--id` sets the pack id when it should differ from the repo, `--tag` sets the release's tag, `--notes` replaces the generated release notes — which is where `policies show --releases` reads each release's counts and commit from — `--out` chooses where the assets are written (default `dist-pack`), and `--dry-run` builds them without publishing and needs no credential. -```bash -failproofai pack add acme/support-agent -``` +Anyone can now install it with `failproofai policies add acme/support-agent`. See [policy packs](/policies/packs) for pinning a version and taking only part of one. -The asset names are fixed — they are what a consumer's CLI constructs its URLs from, with no API call and no discovery. +### List it on the policy hub + +Add the `failproofai-policies` topic to the repository on GitHub. There is no submission form and no approval queue: the [policy hub](https://befailproof.ai/policy-hub/)'s crawler picks the repository up on its next pass. The topic only puts it up for consideration — what lists it is a release whose manifest verifies against its own `SHA256SUMS` and parses under the same rules the CLI uses, which is exactly what `failproofai publish` produces. + +## How the version is decided + +The version is the **commit you are publishing from** — its short sha, twelve characters: `a1b2c3d4e5f6`. There is nothing to pick and nothing to increment, and the version names exactly where the bytes came from, so publishing the same source twice gives the same version. + +It is read from the tree in front of you, never from the repository's releases, so a fresh clone and an air-gapped machine compute the same answer without asking GitHub what happened before. + +Because the version names a commit, that commit has to exist. At a terminal, `publish` makes it for you: it initialises a repository when there is none, and commits changed policy files before it builds. It **refuses** instead — naming `--version` as the way out — when it runs without a terminal (a commit made on a CI runner would exist nowhere else), when files other than the policies are uncommitted, or in a checkout that has no commits yet. A tag on `HEAD` wins over the sha — someone who tagged `v1.2.0` has said what this release is. + +A sha carries no ordering of its own, so use `failproofai policies show / --releases` to see which release came first — newest at the top. ## Shipping a new version -Build with the new `--version`, tag a new release, attach the three assets again. Consumers run the same `pack add` and keep whatever subset they had chosen; a policy they turned off stays off across the upgrade. +Commit the change and run `failproofai publish` again — the new commit is the new version. Consumers run the same `failproofai policies add`. Without a terminal, or with a selection flag, they keep the subset they had chosen and a policy they turned off stays off; at a terminal with no flag, the picker opens pre-ticked with your defaults and their answer replaces their selection. Changing a policy's **name** is a breaking change: a machine that had turned it off is turning off a name that no longer exists, and the new name arrives at whatever `defaultEnabled` says. @@ -82,10 +117,12 @@ Changing a policy's **name** is a breaking change: a machine that had turned it Publish from a repository whose write access you control, and treat a pack release like publishing a package. +The repository must also be **public**. Installs are anonymous HTTPS with no credential to offer, so an existing private repo is refused before anything is built or uploaded, and one `publish` creates is public for the same reason. `--allow-private` overrides that for somebody handing the three assets over another way, and says plainly that no `policies add` can reach them. Only the release matters: installs read `releases/download//` and never touch your git tree. + ## Observe before you enforce -A manifest may declare `"effect": "observe"`. Those policies run and their verdicts are **recorded and discarded** — nothing is blocked. It is the way to measure a new rule against real traffic before it can interrupt anyone's work. +A manifest may declare `"effect": "observe"` — `failproofai publish --effect observe` is what sets it. Those policies run and their verdicts are **recorded and discarded** — nothing is blocked. It is the way to measure a new rule against real traffic before it can interrupt anyone's work. ```json -{ "id": "acme/support-agent", "version": "1.1.0", "effect": "observe", "policies": [ ... ] } +{ "id": "acme/support-agent", "version": "a1b2c3d4e5f6", "effect": "observe", "policies": [ ... ] } ``` diff --git a/docs/policies/rollback.mdx b/docs/policies/rollback.mdx index 4dda49283..9601d4098 100644 --- a/docs/policies/rollback.mdx +++ b/docs/policies/rollback.mdx @@ -1,10 +1,24 @@ --- -title: "Rollback" -description: "Restore a known policy deployment when a rollout disrupts valid agent work." +title: "Versions and rollback" +description: "Every publish is an immutable version, so a rollout that disrupts valid agent work is undone by redeploying the last good one." icon: "rotate-ccw" --- -Rollback changes the deployed version or removes a policy assignment; it does not erase the decision history that explains the incident. +A published policy version never changes. Editing a policy and publishing again mints a new version; it never rewrites the one already on machines. That is what makes rollback safe: the last good version is still there, byte for byte, and rolling back does not erase the decision history that explains what went wrong. + +## Find a version + + + + Go to **Admin → policy editor** and open **library** to compare a policy's versions or disable one. + + + ```bash + fp policies list # every policy version + fp policies show # one version, with its source + ``` + + ## Roll back a machine @@ -12,21 +26,41 @@ Rollback changes the deployed version or removes a policy assignment; it does no 1. Go to **Admin → enforcement**, expand the affected machine, and identify its last known-good policy set. 2. Select **edit**, restore those versions and effects, and apply the new deployment. - 3. Wait for machine check-in, then verify the reported deployment. + 3. Wait for the machine's check-in, then verify the reported deployment. 4. Open **Observe → policy** and the affected sessions to confirm valid work is no longer blocked. - - Cloud deployment rollback is a dashboard workflow. Use local status to confirm that the corrected deployment has reached the machine: + Every deployment to a machine is a numbered generation. List them, then reinstate one: ```bash - failproofai config --status + fp fleet history + fp fleet rollback ``` - `failproofai config --pause` pauses builtin, custom, and convention policies for one local session. It does not pause Cloud-managed policies, so it is not a workaround for a bad Cloud deployment. + `rollback` mints a new generation carrying the old set rather than rewinding the counter, so history stays append-only, and it refuses a generation that names a policy since disabled or deleted. It needs a signed-in session with `policies:write`. `fp fleet diff ` shows what was intended against what the machine applied — it reads as `behind` until the machine next polls — and on the machine itself, `failproofai policies` lists the deployment it is running. +## Take one policy off every machine + +```bash +fp policies disable # remove it from every deployment carrying it +fp policies enable # add it back +``` + +Each mints a new generation on every deployment it touches. Rolling one of those generations back is not how you undo a `disable`, though — `rollback` refuses a generation that names a disabled policy, and every generation from before the disable names this one. `fp policies enable` is the way back, and it mints its own generation in turn. + +## Roll back a pack + +A pack is pinned to the release you installed, so rolling it back means installing an earlier one: + +```bash +failproofai policies show FailproofAI/policies --releases # every version it has published, and which one is here +failproofai policies add FailproofAI/policies@a1b2c3d4e5f6 # pin that one +``` + +Without a terminal, or with `--policy`, `--category` or `--all`, re-adding keeps the subset you had chosen. At a terminal with none of those, it opens the picker pre-ticked with the author's defaults, and what you tick replaces your selection — so re-tick what you had. + ## When to roll back - A policy blocks an expected production action. @@ -34,8 +68,8 @@ Rollback changes the deployed version or removes a policy assignment; it does no - A policy depends on fields that an integration does not provide. - A new version changes behavior outside the intended failure mode. -After rollback, open the affected sessions and identify the condition that caused the false positive. Create a new version, test both the unsafe and legitimate cases, then repeat the observe phase. +After rolling back, open the affected sessions and find the condition behind the false positive. Publish a new version, [test](/policies/test) both the unsafe and the legitimate case, and observe it again before enforcing. - Pausing enforcement can be appropriate during an incident, but it widens exposure for every active policy in that scope. Prefer rolling back the specific policy version when possible. + `failproofai config --pause` suspends local policies for one session and never Cloud-managed ones, so it is no way out of a bad Cloud deployment. A pause also widens exposure for every policy in its scope; prefer rolling back the one version that misbehaves. diff --git a/docs/policies/test.mdx b/docs/policies/test.mdx new file mode 100644 index 000000000..e41746d76 --- /dev/null +++ b/docs/policies/test.mdx @@ -0,0 +1,60 @@ +--- +title: "Test a policy" +description: "Backtest a draft against traffic you already have, and prove it stops what it should and allows what it must, before any machine enforces it." +icon: "flask-conical" +--- + +Test every policy two ways: against the traffic your agents already produced, and against a legitimate action it must let through. A policy that has only seen the unsafe case has not been tested. + +## Backtest the draft + + + + The policy editor replays a draft against calls your fleet already made, before you publish it. + + 1. Open the draft in **Admin → policy editor**. The editor confirms it parses as JavaScript. + 2. In **backtest**, pick the agents and the time window to replay — **every agent** and **30d** by default — and leave the last filter on **everything** unless you want to narrow it. + 3. Select **run backtest**. + + ![The backtest panel under a draft that parses as JavaScript, with its three filters and the run backtest action, above publish version.](/images/dashboard/policy-backtest.png) + + The result is what the draft would have done to those calls — including how many **working** calls it would have interrupted. Those are false positives found before any agent meets them: tighten the draft and run it again until that number is one you can accept. + + + Backtesting is a dashboard feature. From a terminal, run the policy against events you describe instead, below. + + + +## Run it against an event you describe + +`fp policies test` runs a policy file on your machine against a synthetic event and checks the decision. Nothing is published and nothing reaches Cloud: + +```bash +fp policies test ./checkout.policy.mjs --command "git push --force" --expect deny +fp policies test ./checkout.policy.mjs --command "git push" --expect allow +``` + +Shape the event with `--event`, `--tool`, `--command` and `--file`. The policy's own `match` filter still applies, so a policy that does not cover the event you described reports `skipped` rather than a decision — usually a sign its `match` is narrower than you meant. + +## Run it on one machine + +Next, enforce it for real on your own machine, against your own agent: + +```bash +failproofai policies --install --custom ./checkout.policy.mjs --scope project +failproofai policies +``` + +The first command validates and installs the file; the second confirms it loaded, alongside everything else enforcing here. Ask the agent to do what the policy stops and watch it get refused, then do the legitimate version and watch it go through. Nobody else is affected. + +On a machine connected to Cloud, check both decisions under **Observe → policy**: filter by the policy name, then open each linked session to confirm the tool input it matched and the reason it returned. + +## Test what breaks + +The install refuses a missing file, a syntax error, an unresolved import, a top-level exception, or a module that times out while loading — so re-run it after every change to the file or anything it imports. At enforcement time the same broken file is logged and **skipped** so every other policy keeps running: treat a load warning in production logs as lost enforcement. Convention files load without the install command, so keep an explicit `failproofai policies --install --custom ` step in CI — it is what fails the build on a broken policy. + +Then feed it what agents actually send, not only the input you expect: missing fields, alternate tool names such as `Write` and `Edit`, Windows paths, malformed input. Return an intentional `allow`, `instruct` or `deny` on every path, keep the function deterministic, and bound any external call with a short timeout. + +## Then publish it and observe it + +A backtest shows what the policy would have done to the traffic you had; it cannot show what traffic you have not seen yet will do. Select **publish version** in the editor (or run `fp policies publish`), then [deploy it](/policies/deploy) in **observe** mode first — its verdicts are recorded and nothing is blocked — and enforce once its matches separate unsafe actions from valid ones. diff --git a/docs/pt-br/policies/builtin-catalog.mdx b/docs/pt-br/policies/builtin-catalog.mdx deleted file mode 100644 index 80270ba76..000000000 --- a/docs/pt-br/policies/builtin-catalog.mdx +++ /dev/null @@ -1,132 +0,0 @@ ---- -title: "Catálogo de políticas builtin" -description: "Revise cada política builtin do Failproof AI, seu gatilho, estado recomendado e parâmetros configuráveis." -icon: "list-checks" ---- - -O pacote instalado é a fonte de verdade para a disponibilidade de políticas. Execute `failproofai policies` após cada atualização, pois as entradas do catálogo e o comportamento podem mudar com a versão do pacote. - -## Linha de base recomendada - -A seleção recomendada pela configuração guiada atualmente habilita sanitizadores de segredos, proteções de ambiente, autoproteção, proteções contra comandos catastróficos e segurança de branches protegidas: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` está **sempre ativo**. Ele é listado acima apenas para -fins de completude, mas se registra em toda avaliação independentemente de aparecer ou não -no seu conjunto habilitado — não pode ser desabilitado ou pausado. Uma proteção contra o -agente desativar a aplicação de regras não seria uma proteção se o próprio agente pudesse desativá-la. - -O conjunto recomendado é deliberadamente mais restrito do que **Tudo**. Políticas de infraestrutura e de fluxo de trabalho podem interromper trabalhos válidos e devem ser habilitadas apenas para os repositórios e máquinas que delas necessitam. - -## Segredos e ambiente - -| Política | Gatilho | Resultado | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | Redaciona JWTs da saída da ferramenta antes que o modelo os veja. | -| `sanitize-api-keys` | `PostToolUse` | Redaciona chaves comuns de OpenAI, Anthropic, GitHub, AWS, Stripe e Google. | -| `sanitize-connection-strings` | `PostToolUse` | Redaciona strings de conexão de banco de dados que contenham credenciais. | -| `sanitize-private-key-content` | `PostToolUse` | Redaciona corpos de chaves privadas PEM. | -| `sanitize-bearer-tokens` | `PostToolUse` | Redaciona tokens bearer de autorização. | -| `protect-env-vars` | `PreToolUse` em ferramentas de shell | Bloqueia comandos que expõem variáveis de ambiente. | -| `block-env-files` | `PreToolUse` | Bloqueia leituras e escritas de arquivos `.env`. | -| `block-read-outside-cwd` | `PreToolUse` em ferramentas de leitura, glob, grep ou shell | Mantém leituras dentro do diretório de trabalho da sessão. | -| `block-secrets-write` | `PreToolUse` em ferramentas de escrita | Bloqueia escritas em nomes de arquivos comuns de chaves secretas e credenciais. | - -## Comandos perigosos e infraestrutura - -| Política | Gatilho | Resultado | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | Bloqueia `sudo` a menos que um padrão de permissão corresponda. | -| `block-curl-pipe-sh` | `PreToolUse` | Bloqueia scripts baixados e redirecionados diretamente para um shell. | -| `block-rm-rf` | `PreToolUse` | Bloqueia padrões catastróficos de exclusão recursiva. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Sempre ativo, não pode ser desabilitado.** Bloqueia toda invocação da CLI do Failproof AI, autopausas e desinstalações via gerenciador de pacotes. | -| `block-kubectl` | `PreToolUse` | Controla comandos Kubernetes. | -| `block-terraform` | `PreToolUse` | Controla comandos Terraform e OpenTofu. | -| `block-aws-cli` | `PreToolUse` | Controla comandos da AWS CLI. | -| `block-gcloud` | `PreToolUse` | Controla comandos da Google Cloud CLI. | -| `block-az-cli` | `PreToolUse` | Controla comandos da Azure CLI. | -| `block-helm` | `PreToolUse` | Controla comandos do Helm. | -| `block-gh-pipeline` | `PreToolUse` | Controla operações mutantes de workflow, run, merge, release, cache e secret da GitHub CLI. | - -## Segurança de Git e banco de dados - -| Política | Gatilho | Resultado | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Bloqueia pushes diretos para branches protegidas configuradas. | -| `block-force-push` | `PreToolUse` | Bloqueia force-pushes; `--force-with-lease` permanece permitido pela implementação atual. | -| `block-work-on-main` | `PreToolUse` | Bloqueia commits e merges em branches protegidas. | -| `warn-git-amend` | `PreToolUse` | Avisa antes de reescrever um commit com `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | Avisa antes de descartar ou limpar stashes permanentemente. | -| `warn-all-files-staged` | `PreToolUse` | Avisa em uso amplo de `git add -A`, `git add .` ou `git add --all`. | -| `warn-destructive-sql` | `PreToolUse` | Avisa em `DROP`, `TRUNCATE` e `DELETE` sem `WHERE` por meio de clientes de banco de dados reconhecidos. | -| `warn-schema-alteration` | `PreToolUse` | Avisa em operações reconhecidas de `ALTER TABLE` com colunas e renomeações. | - -## Pacotes, comportamento do sistema e loops de agente - -| Política | Gatilho | Resultado | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Avisa antes de publicar em registros de pacotes. | -| `warn-global-package-install` | `PreToolUse` | Avisa antes de instalar pacotes globalmente. | -| `prefer-package-manager` | `PreToolUse` | Instrui o agente a usar um gerenciador de pacotes permitido. | -| `warn-large-file-write` | `PreToolUse` em ferramentas de escrita | Avisa acima do limite de tamanho de arquivo configurado. | -| `warn-background-process` | `PreToolUse` | Avisa em padrões de processos em segundo plano desconectados ou de longa duração. | -| `warn-repeated-tool-calls` | `PreToolUse` | Avisa após três ou mais chamadas de ferramenta idênticas. | - -## Fluxo de trabalho de fim de tarefa - -Essas políticas requerem um harness que emita um evento `Stop` compatível. - -| Política | Resultado | -| --- | --- | -| `require-commit-before-stop` | Recusa a conclusão enquanto houver trabalho rastreado sem commit. | -| `require-push-before-stop` | Recusa a conclusão enquanto houver commits apenas locais. | -| `require-pr-before-stop` | Exige um pull request para o branch atual. | -| `require-no-conflicts-before-stop` | Exige um merge limpo com o branch base configurado. | -| `require-ci-green-before-stop` | Exige que as verificações de CI do HEAD atual sejam concluídas com sucesso. | - -## Referência de parâmetros - -Configure os parâmetros no objeto `policyParams` do escopo selecionado. Os tipos são validados por cada política. - -| Política | Parâmetro | Tipo e padrão | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; as entradas contêm `regex` e `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Bloqueadores de infraestrutura | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - Um padrão de permissão amplia o que um agente pode fazer. Teste a tokenização exata e as variantes de comando no harness de destino antes de implantá-lo em uma frota. - \ No newline at end of file diff --git a/docs/pt-br/policies/builtin.mdx b/docs/pt-br/policies/builtin.mdx deleted file mode 100644 index a339ad116..000000000 --- a/docs/pt-br/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Políticas integradas" -description: "Ative proteções mantidas para modos comuns de falha de agentes." -icon: "library" ---- - -As políticas integradas cobrem tratamento de segredos, arquivos de ambiente, comandos shell destrutivos, branches protegidas, ferramentas de nuvem e infraestrutura, publicação de pacotes, chamadas repetidas e verificações de fluxo de trabalho ao final de tarefas. - -## Ativar e verificar uma política integrada - - - - 1. Instale a política em uma máquina conectada usando a CLI local. - 2. Execute uma ação de teste segura no agente instrumentado. - 3. Vá em **Observe → policy** e filtre pelo nome da política, ambiente da máquina ou decisão. - 4. Abra a sessão vinculada para confirmar a entrada da ferramenta correspondente e o motivo retornado. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - Remova com `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -Liste as políticas disponíveis na versão instalada: - -```bash -failproofai policies -``` - -Ative uma política para um projeto: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Ative várias políticas para harnesses selecionados: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Algumas políticas aceitam parâmetros ou estão marcadas como beta. Revise a descrição, o escopo de correspondência e o comportamento padrão antes de aplicar. Uma política que protege um fluxo de trabalho pode bloquear operações válidas em outro. - - - Revise todas as 40 políticas atuais, seus gatilhos, linha de base recomendada e parâmetros. - - - - Prefira o escopo de projeto para expectativas específicas do repositório e o escopo de usuário para requisitos de segurança em toda a máquina. - \ No newline at end of file diff --git a/docs/pt-br/policies/custom.mdx b/docs/pt-br/policies/custom.mdx deleted file mode 100644 index f8f8dac66..000000000 --- a/docs/pt-br/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Políticas personalizadas" -description: "Escreva uma política para um modo de falha exclusivo do seu fluxo de trabalho de agente." -icon: "shield-plus" ---- - -Crie um arquivo com a extensão `policies.js`, `policies.mjs` ou `policies.ts` dentro de `.failproofai/policies/`. Arquivos de convenção são carregados automaticamente nos escopos de projeto e de usuário. - -## Teste a política antes de publicar na Cloud - - - - 1. Instale a política personalizada em uma máquina de teste e acione tanto uma ação correspondente quanto uma não correspondente legítima. - 2. Vá em **Observe → policy** e compare as duas decisões. - 3. Abra cada sessão vinculada e verifique se o payload do evento contém evidências suficientes para a regra. - 4. Quando o comportamento estiver correto, mova o código revisado para **Admin → policy editor** e publique uma versão. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - Arquivos de convenção em `.failproofai/policies/` são carregados sem `--custom`. Mantenha um comando de instalação explícito no CI quando a validação precisar falhar em caso de módulo corrompido. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -Isso corresponde a `production/config.yml`, `/srv/production/config.yml`, `/srv/production` e `C:\\production\\config.yml` tanto para `Write` quanto para `Edit`. Não corresponde a nomes como `production-backup`, pois `production` deve ser um segmento de caminho completo. - -Valide e instale um arquivo explícito: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -O contexto da política inclui o tipo de evento, payload normalizado, nome e entrada da ferramenta, metadados da sessão, parâmetros e o CLI de origem quando disponível. - -## Teste os caminhos de falha - -Execute a validação após alterar o arquivo de entrada ou qualquer módulo local que ele importe: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -O caminho do CLI com --strict falha para arquivos ausentes, erros de sintaxe, importações não resolvidas, exceções de nível superior e timeouts de carregamento de módulo. No momento da aplicação, um arquivo personalizado corrompido é registrado em log e ignorado para que as políticas integradas possam continuar. Trate qualquer aviso de carregamento como uma perda de aplicação esperada e configure alertas para isso nos logs de produção. - -Use nomes globalmente únicos entre políticas explícitas, de convenção e gerenciadas pela Cloud. Mantenha as funções de política determinísticas, limite chamadas externas com timeouts curtos e retorne explicitamente allow, instruct ou deny em todos os caminhos. - - - Uma política personalizada é código de aplicação. Teste campos ausentes, nomes de ferramentas alternativos e entradas malformadas — não apenas a correspondência esperada. - \ No newline at end of file diff --git a/docs/pt-br/policies/fleet.mdx b/docs/pt-br/policies/fleet.mdx deleted file mode 100644 index 3b937d8d8..000000000 --- a/docs/pt-br/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Implantar políticas nas máquinas" -description: "Saiba quais máquinas estão registradas, atualizadas e aplicando as versões de política pretendidas." -icon: "network" ---- - -A cobertura de frota responde se uma política existe onde o risco existe. Rastreie máquinas por ID estável e um rótulo legível por humanos, depois compare o estado de implantação atribuído e reportado. - -## Verificar cobertura - - - - 1. Vá para **Admin → enforcement** e revise os totais de aplicação e observação. - 2. Pesquise uma máquina por ID ou rótulo, ou filtre por máquinas sem política. - 3. Expanda uma linha para comparar políticas atribuídas, implantação reportada, último check-in e histórico. - 4. Atualize após o intervalo de polling da máquina quando uma implantação aplicada permanecer pendente. - - ![O frota de Enforcement exibindo cobertura de políticas, estado de implantação das máquinas e atribuições de observação e aplicação.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Use `fp events --agent-id --since 24h` para confirmar que a atividade do agente da máquina chega ao Cloud. - - - -Use as visualizações de cobertura para encontrar: - -- Máquinas que nunca obtiveram a implantação mais recente -- Máquinas registradas que pararam de reportar atividade -- Uma política atribuída ao ambiente ou coorte errado -- Desvio de versão após uma atualização interrompida - -Renomeie uma máquina sem reconectá-la: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Verifique o estado local: - -```bash -failproofai config --status -``` - - - Use rótulos que identifiquem a carga de trabalho e o ambiente. Nomes de host por si só frequentemente são insuficientes após dimensionamento automático ou substituição de máquinas. - \ No newline at end of file diff --git a/docs/reference/cloud-cli.mdx b/docs/reference/cloud-cli.mdx index 946200cc5..404acc524 100644 --- a/docs/reference/cloud-cli.mdx +++ b/docs/reference/cloud-cli.mdx @@ -331,7 +331,7 @@ Cloud-managed policy versions. **Session-only** — every command here exits `2` | `fp policies enable POLICY_ID` | Add it back to every deployment it was removed from, minting a new generation on each. | `--yes`, `-y` | | `fp policies disable POLICY_ID` | Remove it from every deployment carrying it, minting a new generation on each. | `--yes`, `-y` | | `fp policies delete POLICY_ID` | Delete a policy version. | `--yes`, `-y` | -| `fp policies test PATH` | Run a policy locally against a synthetic context. Applies each policy's `match` filter, so one that does not cover the given event/tool is reported `skipped` rather than run. | `--event`; `--tool`; `--command`; `--file-path`; `--expect` | +| `fp policies test PATH` | Run a policy locally against a synthetic context. Applies each policy's `match` filter, so one that does not cover the given event/tool is reported `skipped` rather than run. | `--event`; `--tool`; `--command`; `--file`; `--expect` | | `fp policies compose PROMPT` | Draft a policy with the assistant. Needs `policies:write`. | — | ### Fleet @@ -345,7 +345,7 @@ Which machines run which policies. **Session-only**, same reason as above. | `fp fleet deploy MACHINE_ID` | **Replaces the machine's whole policy set.** Prints the plan and asks only on an interactive terminal without `--json`. | `--add`; `--remove`; `--set`; `--create`; `--yes`, `-y` | | `fp fleet diff MACHINE_ID` | Compare a machine against another deployment. | — | | `fp fleet history MACHINE_ID` | Past deployments for a machine. | — | -| `fp fleet rollback MACHINE_ID` | Restore a previous deployment. | `--yes`, `-y` | +| `fp fleet rollback MACHINE_ID GENERATION` | Reinstate a past generation's policy set, as a new generation. | `--yes`, `-y` | | `fp fleet rename MACHINE_ID` | Give a machine a readable name. | required `--name` | ### Guardrails diff --git a/docs/reference/custom-agents.mdx b/docs/reference/custom-agents.mdx index 938a3b5b9..ab9b7245d 100644 --- a/docs/reference/custom-agents.mdx +++ b/docs/reference/custom-agents.mdx @@ -37,10 +37,16 @@ The package is installed as `failproofai-sdk` and imported in Python as `failpro ![A custom Python agent session reconstructed as an execution graph and ordered event trace.](/images/dashboard/session-detail.png) + Read the `events:add` key into the shell. `read -s` takes it at a prompt that does not echo, so it never appears in a command or in shell history: + + ```bash + read -rs FAILPROOFAI_CLOUD_TOKEN && export FAILPROOFAI_CLOUD_TOKEN + ``` + + Then set the machine up and check that it connected: + ```bash - failproofai config \ - --connect https://app.befailproof.ai \ - --token + failproofai config failproofai config --status ``` diff --git a/docs/reference/evaluator-sdk.mdx b/docs/reference/evaluator-sdk.mdx index 1c3da5d4d..054c7ab2c 100644 --- a/docs/reference/evaluator-sdk.mdx +++ b/docs/reference/evaluator-sdk.mdx @@ -1,190 +1,118 @@ --- title: "Evaluator SDK" -description: "Build a service that scores Failproof AI sessions synchronously or asynchronously." +description: "Run your own evaluation worker, for LLM judges and anything else hosted Python cannot do." icon: "gauge" --- -An evaluator receives a completed agent session and returns the quality signals you care about: numeric scores, an explanation for each score, and an optional summary. Failproof AI stores these results beside the trace and charts them across agents and environments. +The Evaluator SDK runs evaluations on your own infrastructure. Your worker registers its evaluations with Failproof AI, claims sessions as they finish, scores them, and submits the results, all over outbound HTTPS: nothing connects in to it. Use it for what [hosted Python](/evaluations/write) cannot do — LLM judges, model calls, packages, secrets, and network access. Its results appear beside hosted ones on the [evaluations page](/sessions/evaluations), tagged **customer**. -## Set up an evaluator +It ships in `failproofai-sdk`, under `failproofai_sdk.evaluator`; importing the tracing SDK does not load it. - - - Install the SDK and the server used to run it. - - ```bash - pip install failproofai-sdk uvicorn - ``` - - - - Create `evaluator.py`. This example checks whether a session contains any failed tool calls. - - ```python - import os - from failproofai.evaluator import Evaluator, EvalResponse - - app = Evaluator(token=os.environ.get("EVALUATOR_TOKEN")) - - @app.config - def config(): - return {"inactivity_timeout_secs": 1800} - - @app.evaluator - def evaluate(req): - tool_errors = sum( - 1 for item in req.events - if item.event_type == "tool_result" and item.payload.get("error") - ) - return EvalResponse( - scores={"tool_reliability": 1.0 if tool_errors == 0 else 0.0}, - reasoning={"tool_reliability": f"{tool_errors} tool errors"}, - ) - ``` - - - - Set a shared token, start the evaluator, and confirm its health endpoint responds. - - ```bash - export EVALUATOR_TOKEN= - uvicorn evaluator:app --host 0.0.0.0 --port 8080 - ``` - - In another terminal: - - ```bash - curl http://127.0.0.1:8080/health - ``` - - - -## Connect the evaluator to Failproof AI +```bash +pip install failproofai-sdk +``` -1. Deploy the evaluator at an HTTPS URL reachable by Failproof AI Cloud. -2. Configure `EVALUATOR_ENDPOINT` with that URL and set `EVALUATOR_TOKEN` to the same token used by the evaluator. For managed Cloud, contact [support@befailproof.ai](mailto:support@befailproof.ai) to configure the connection. -3. Run an evaluation and confirm its scores appear in Failproof AI. +## Write evaluations - - - Open a completed session under **Observe → Sessions** and select **Run evaluation** if it was not evaluated automatically. Review the status, scores, reasoning, and summary in the session's **Evaluation** panel. +```python +from failproofai_sdk.evaluator import ConditionResult, EvalResult, Evaluator, Metric, Score + +app = Evaluator(name="customer-production", version="2026.08.1") + + +@app.eval( + "tool_efficiency", + version="1.0.0", + labels=["tools", "deterministic"], + when=lambda session: ConditionResult(session.count("tool_use") > 0, "no_tool_calls"), +) +def tool_efficiency(session): + calls = session.events_of_type("tool_use") + distinct = {e.payload.get("tool_name") for e in calls if e.payload.get("tool_name")} + value = len(distinct) / len(calls) + return EvalResult( + score=Score(value, passed=value >= 0.7), + metrics={"tool_call_count": Metric(len(calls), unit="events")}, + reasoning=f"{len(distinct)} distinct tools across {len(calls)} calls", + ) - Use **Observe → Evaluations** to compare scores across agents or environments. Use **Observe → Metrics** for latency, cost, token, and other numeric measurements. - Start with one session to confirm that the evaluator returned the expected score keys and useful reasoning for that specific run. +@app.eval( + "answer_relevance", + version="judge-v1", + labels=["llm_judge", "relevance"], + when=lambda session: ConditionResult( + session.count("human_input") > 0 and session.count("model_response") > 0, + "no_exchange", + ), + timeout_seconds=30, +) +async def answer_relevance(session): + question = session.events_of_type("human_input")[-1].payload.get("response") + answer = session.events_of_type("model_response")[-1].payload.get("content") + value, reasoning = await ask_judge(question, answer) # your LLM call: a 0-1 score and why + return EvalResult(score=Score(value, passed=value >= 0.7), reasoning=reasoning) + + +if __name__ == "__main__": + app.run_from_env() +``` - ![A session detail view showing evaluation scores and reasoning beside its trace.](/images/dashboard/session-detail.png) +- `@app.eval(key, version=...)` registers an evaluation. The key is what its results chart under; change the version whenever the logic changes, and each result keeps the version that produced it. One worker holds up to 100 evaluations. +- `result_kind` is `"score"` unless you say otherwise. For a `"metric"` or `"assertion"` evaluation, name one `metrics` or `assertions` entry after the key: that entry is its result. +- `when` decides whether a session applies. Return `ConditionResult(False, "")` to skip one, and the reason is recorded. +- An evaluation can be a plain function or `async`, and `timeout_seconds` bounds it. +- Payload keys — `tool_name`, `response`, and `content` above — are whatever your agents send, so read them off a real session. - Once individual results look correct, use the evaluation dashboard to compare those scores over time and across agents or environments. +## Run the worker - ![A quality dashboard charting evaluator scores over time.](/images/dashboard/dashboard-quality.png) +Put a key with the `evaluations:run` permission, created under **Administration → Keys**, in `FAILPROOFAI_EVALUATOR_TOKEN` — set it from your secret store rather than typing it into a command — and start the worker: - A healthy chart should use stable score names; changing a key creates a separate series. - - - ```bash - fp evals --since 1h --score tool_reliability:0..1 - fp evals --since 24h --aggregate - ``` - - +```bash +FAILPROOFAI_EVALUATOR_URL=https://app.befailproof.ai python evaluator.py +``` -For a self-hosted Cloud instance, automatic evaluation is disabled until `EVALUATOR_ENDPOINT` is set on the server process. Restart the server after changing evaluator environment variables. +Without the `__main__` block, `python -m failproofai_sdk.evaluator evaluator:app` does the same. -The service exposes `GET /health`, `GET /config`, `POST /evaluate`, and optionally `GET /evaluate/{job_id}`. Return `JobPending` for asynchronous work and register `@app.job_lookup` so Failproof AI can poll it. +| Variable | Default | Purpose | +| --- | --- | --- | +| `FAILPROOFAI_EVALUATOR_URL` | required | Where Failproof AI is: `https://app.befailproof.ai` for Cloud. HTTPS unless it points at loopback | +| `FAILPROOFAI_EVALUATOR_TOKEN` | required | A key with `evaluations:run` | +| `FAILPROOFAI_EVALUATOR_WORKER_ID` | `-` | Names this worker | +| `FAILPROOFAI_EVALUATOR_CONCURRENCY` | `1` | Sessions this worker scores at once | +| `FAILPROOFAI_EVALUATOR_REQUEST_TIMEOUT_SECONDS` | `30` | Timeout for each request to Failproof AI | +| `FAILPROOFAI_EVALUATOR_DRAIN_TIMEOUT_SECONDS` | `60` | How long a stopping worker waits for runs in flight | +| `FAILPROOFAI_EVALUATOR_ALLOW_INSECURE_HTTP` | `false` | Allow plain HTTP to a URL that is not loopback — see the warning below | +| `FAILPROOFAI_EVALUATOR_MODULE` | none | The `module:attribute` for `python -m failproofai_sdk.evaluator` | -When a token is configured, all routes except health require the same bearer token that Failproof AI sends as `EVALUATOR_TOKEN`. + + `FAILPROOFAI_EVALUATOR_ALLOW_INSECURE_HTTP` sends everything in cleartext. The worker carries `FAILPROOFAI_EVALUATOR_TOKEN` as an `Authorization: Bearer` header on every request, and the transcripts it fetches are the sessions themselves — so anyone on the path reads both, and the token they read runs evaluations until you rotate it. Use it only on an isolated development network. Everywhere else the URL must be HTTPS; loopback needs no flag. + -## SDK types +## Result types | Type | Fields | | --- | --- | -| `AgentEvent` | `id`, `ts`, `event_type`, `payload` | -| `EvalRequest` | `schema_version`, `session_id`, `agent_id`, `environment`, `started_at`, `ended_at`, `events` | -| `EvalResponse` | `scores`, `reasoning`, `summary` | -| `JobPending` | `job_id`, `next_poll_secs` | -| `EvaluatorConfig` | `inactivity_timeout_secs`, `default_poll_interval_secs` | - -## Decorators and routes +| `Score` | `value` (0 to 1), `passed`, `unit` (default `ratio`), `display_value`, `description` | +| `Metric` | `value`, `unit`, `display_value`, `description` | +| `Assertion` | `passed`, `description` | +| `EvalResult` | `score`, `metrics`, `assertions`, `reasoning`, `summary`, `labels` | +| `ConditionResult` | `applicable`, `reason_code` | -| Decorator | Route | Required | -| --- | --- | --- | -| `@app.evaluator` | `POST /evaluate` | Yes | -| `@app.job_lookup` | `GET /evaluate/{job_id}` | When returning `JobPending` | -| `@app.config` | `GET /config` | No | - -The SDK caps evaluation request bodies at 25 MiB. Unknown request fields are ignored so services remain compatible as the event contract grows. +An `EvalResult` carries at least one score, metric, or assertion, and at most 25, each under a unique key. -## Return asynchronous work +## The session -Use `JobPending` when evaluation cannot finish inside one request. The job ID is opaque to Failproof AI and must remain resolvable by your service until the result is collected or the server timeout expires. - -```python -from failproofai.evaluator import EvalRequest, EvalResponse, Evaluator, JobPending - -app = Evaluator(token="shared-secret") - -@app.evaluator -def start(req: EvalRequest) -> JobPending: - job_id = enqueue(req) - return JobPending(job_id=job_id, next_poll_secs=30) - -@app.job_lookup -def lookup(job_id: str): - result = get_result(job_id) - if result is None: - return JobPending(job_id=job_id, next_poll_secs=30) - return EvalResponse( - scores=result.scores, - reasoning=result.reasoning, - summary=result.summary, - ) -``` +| Field or method | Gives you | +| --- | --- | +| `session_id`, `agent_id`, `environment` | The session's identity | +| `started_at`, `ended_at` | When it started and ended | +| `event_count`, `events` | The full, ordered transcript | +| `count(event_type)` | How many events of that type it holds | +| `events_of_type(event_type)` | Those events, in order | -Polling cadence is selected in this order: `JobPending.next_poll_secs`, `EvaluatorConfig.default_poll_interval_secs`, then the server's `EVALUATOR_POLLING_INTERVAL_SECS`. Values are clamped between 1 second and 1 hour. The server's default wall-clock polling cap is one hour. +Each event carries `id`, `ts`, `event_type`, and `payload`. -## Request and response fields +## The legacy evaluator -| Field | Type | Notes | -| --- | --- | --- | -| `EvalRequest.schema_version` | `str` | Currently `"1"`. | -| `session_id`, `agent_id`, `environment` | `str` | Session identity and environment. | -| `started_at` | `datetime` | Timestamp of the first event. | -| `ended_at` | `datetime \| None` | Present when the session emitted an end event. | -| `events` | `list[AgentEvent]` | Full ordered event stream. | -| `AgentEvent.id` | `int` | Backend event row identifier. | -| `AgentEvent.ts` | `datetime` | Event timestamp. | -| `AgentEvent.event_type` | `str` | Event family such as `tool_use`. | -| `AgentEvent.payload` | `dict[str, Any]` | Complete event payload. | -| `EvalResponse.scores` | `dict[str, float] \| None` | Numeric dimensions charted in evaluations. | -| `EvalResponse.reasoning` | `dict[str, str] \| None` | Per-score explanations; keys should mirror `scores`. | -| `EvalResponse.summary` | `str \| None` | Overall evaluation narrative. | - -## Server operator settings - -Automatic evaluation is deployment-wide and remains disabled when `EVALUATOR_ENDPOINT` is absent. - -| Variable | Default | Purpose | -| --- | --- | --- | -| `EVALUATOR_ENDPOINT` | unset | Base URL of the evaluator service. | -| `EVALUATOR_TOKEN` | unset | Bearer token shared with `Evaluator(token=...)`. | -| `EVALUATOR_WORKERS` | `2` | Concurrent dispatcher workers. | -| `EVALUATOR_CLAIM_BATCH` | `4` | Sessions claimed per dispatcher pass. | -| `EVALUATOR_POLLING_INTERVAL_SECS` | `10` | Fallback async polling cadence. | -| `EVALUATOR_REQUEST_TIMEOUT_MS` | `30000` | Per-request evaluator timeout. | -| `EVALUATOR_MAX_ATTEMPTS` | `5` | Delivery attempts before terminal failure. | -| `EVALUATOR_CONFIG_REFRESH_SECS` | `300` | Refresh cadence for `/config`. | -| `EVALUATOR_MAX_POLL_DURATION_SECS` | `3600` | Maximum wall-clock async polling time. | - -The server can also constrain which organizations use the deployment-global evaluator. Treat endpoint, token, retry, and organization-gate changes as operator configuration and restart or roll the server after changing them. - -## Security and operations - -- Put the evaluator behind HTTPS when traffic crosses a trusted network boundary. -- Configure a non-empty bearer token and keep it identical on both services. -- Do not log the token or full sensitive prompts from request payloads. -- Make synchronous handlers idempotent; retries may repeat a request. -- Persist asynchronous job state outside process memory in production. -- Return stable score keys. Renaming a key creates a new chart series rather than changing the old one. - -The SDK emits structured lifecycle logs such as `eval received`, `eval responded`, `job lookup`, `config returned`, `auth rejected`, and handler exceptions. It does not configure logging handlers; use the host application's logging configuration. +The earlier Evaluator SDK — an HTTP service Failproof AI called at `EVALUATOR_ENDPOINT`, answering `/evaluate` and polled through `JobPending` — is retired. Build new evaluators on this worker; operators of a self-hosted instance running a legacy service can keep it through the transition. diff --git a/docs/reference/failproof-cli.mdx b/docs/reference/failproof-cli.mdx index 49242ed3d..c4a234cb9 100644 --- a/docs/reference/failproof-cli.mdx +++ b/docs/reference/failproof-cli.mdx @@ -6,37 +6,53 @@ icon: "terminal" Install the local CLI with `npm install -g failproofai`. Run it with no arguments to open the local policy dashboard. -The package requires Node.js 20.9 or newer. Bun 1.3 or newer is supported for development and source installs. `failproofai configure` and `failproofai setup` are aliases for `failproofai config`; `failproofai p` is an alias for `failproofai policies`. +The package requires Node.js 20.9 or newer. Bun 1.3 or newer is supported for development and source installs. `failproofai configure` and `failproofai setup` are aliases for `failproofai config`. `failproofai policy`, `failproofai pack` and `failproofai p` are all spellings of `failproofai policies` — packs and single policies were three commands for one idea and are now one. The older spellings still work, with two exceptions: `pack list ` is now `policies show `, and `pack build` is now `publish`. ## Set up a machine +Install the CLI, then read the machine key into the shell. `read -s` takes it at a prompt that does not echo, so it never appears in a command: + ```bash npm install -g failproofai -failproofai config \ - --connect https://app.befailproof.ai \ - --token \ - --machine-label checkout-prod-01 -failproofai policies --install +read -rs FAILPROOFAI_CLOUD_TOKEN && export FAILPROOFAI_CLOUD_TOKEN +``` + +Then set the machine up and choose what it enforces: + +```bash +failproofai config +failproofai policies add FailproofAI/policies failproofai config --status ``` +`failproofai config` is the whole of setup: it installs the `failproofaid` service (root once, via `sudo -n` — never an interactive password prompt), wires hooks into every agent CLI it finds, and connects to Cloud when a key is available. With no terminal — CI, a container, an agent driving it — it applies rather than asking, and exits 1 if anything it was asked to do did not happen. + +It chooses **no** policies. That is the second command's job, and without it a freshly configured machine enforces nothing but the always-on guard. + +Prefer the environment variable over `--token`: a command-line argument is readable from `ps` by every user on the box. That is all the variable protects against — a key typed into any command, `export` included, still lands in shell history, which is why it is read in with `read -s` above. In CI, set it from the secret store and keep shell tracing (`set -x`) off, or the trace prints it. + + + `--connect ` enrols a machine that is **already set up**. It returns as soon as enrolment succeeds — it does not install the daemon and does not wire any hooks. Use plain `failproofai config` (or `failproofai config --token `) on a machine that has not been set up yet, or it will read as connected while collecting and enforcing nothing. + + Run `failproofai` without arguments to open the local policy dashboard. | Command | Outcome | | --- | --- | -| `failproofai config` | Run interactive machine setup | -| `failproofai config --connect --token ` | Connect Cloud ingestion and policy delivery | +| `failproofai config` | Set the machine up: agents, daemon, and Cloud when a key is present | +| `failproofai config --token ` | Set up and connect in one pass, asking nothing | +| `failproofai config --connect ` | Enrol a machine that is **already** set up — no daemon, no hooks | | `failproofai config --status` | Show connection, daemon, delivery, and pause state | | `failproofai policies` | List builtin, custom, convention, pack, and Cloud-managed policies | -| `failproofai policies --install` | Install hooks and enable policies | -| `failproofai policy add ` | Enable one policy — a builtin, or `:` from an installed pack | -| `failproofai policy remove ` | Disable one policy, same naming | +| `failproofai policies --install` | Wire hooks into your agent CLIs. Enables no policy on its own | +| `failproofai policies add ` | Enable one policy — a builtin, or `:` from an installed pack | +| `failproofai policies remove ` | Disable one policy, same naming | | `failproofai policies --uninstall` | Disable policies or remove harness hooks | -| `failproofai pack list` | List installed policy packs and every policy each one carries | -| `failproofai pack add ` | Install a policy pack from a GitHub release; no tag takes the newest and pins it | -| `failproofai pack add --bundled` | Install the builtin policies as a pack, from this package, with no network | -| `failproofai pack build ` | Build the three release assets for a pack of your own | -| `failproofai pack remove ` | Deactivate an installed pack | +| `failproofai policies show /` | What a pack carries, read from its manifest, before you take it | +| `failproofai policies show / --releases` | Every version it has published, and which one is here | +| `failproofai policies add ` | Install a policy pack from a GitHub release; no tag takes the newest and pins it | +| `failproofai publish` | Ship your own policies as a pack; `--init` writes one to start from | +| `failproofai policies remove ` | Uninstall a pack | | `failproofai audit` | Scan local agent history and open the local audit view | | `failproofai audit --schedule [days] --email
` | Schedule recurring local scans and email their findings | | `failproofai audit --status` | Show the report address, interval, and next scheduled scan | @@ -56,9 +72,11 @@ Run `failproofai` without arguments to open the local policy dashboard. | Flag | Use | | --- | --- | -| `--connect --token ` | Connect non-interactively | +| `--token ` | Set up and connect non-interactively; also read from `FAILPROOFAI_CLOUD_TOKEN` | +| `--url ` | Connect somewhere other than `app.befailproof.ai`; also read from `FAILPROOFAI_CLOUD_URL` | +| `--connect ` | Enrol only, on a machine already set up. Skips the daemon and every hook | | `--machine-id ` | Set the stable machine ID | -| `--machine-label ` | Set or change the dashboard label | +| `--machine-label ` | Rename a machine that is **already connected**. On its own it never runs setup, so give it after `failproofai config`, not during | | `--no-transcripts` | Send decisions without transcript content | | `--disconnect` | Stop Cloud policy pulls and event delivery | | `--status` | Show current machine state | @@ -73,7 +91,7 @@ Local pauses suspend builtin, custom, convention, and pack policies for one sess | Flag | Use | | --- | --- | -| `--install`, `-i` | Enable policies and install harness hooks | +| `--install`, `-i` | Install harness hooks. Names after it enable those policies; with none, no policy changes | | `--uninstall`, `-u` | Disable policies or remove hooks | | `--cli ` | Target one or more supported harnesses | | `--scope user\|project\|local\|all` | Choose the configuration scope; `all` is for uninstall | @@ -116,6 +134,8 @@ Use configuration files for persistent machine behavior. Environment variables a | Variable | Use | | --- | --- | +| `FAILPROOFAI_CLOUD_TOKEN` | The Cloud key, instead of `--token`. Prefer this: an argument is readable from `ps` by every user. Set it with `read -s` or from a CI secret store, never by typing the key into a command, which lands in shell history either way | +| `FAILPROOFAI_CLOUD_URL` | The Cloud URL, instead of `--url`. The same variable the daemon reads | | `FAILPROOFAI_HOME` | Relocate the complete `~/.failproofai` layout | | `FAILPROOFAI_LOG_LEVEL` | Set local logging verbosity | | `FAILPROOFAI_HOOK_LOG_FILE` | Write hook diagnostics to a selected file | diff --git a/docs/reference/harnesses.mdx b/docs/reference/harnesses.mdx index ceafd3c9c..09e095854 100644 --- a/docs/reference/harnesses.mdx +++ b/docs/reference/harnesses.mdx @@ -65,15 +65,21 @@ Capabilities are version-sensitive. Re-test after upgrading an agent CLI, especi ![The Policy page used to verify policy decisions from a newly connected harness.](/images/dashboard/policy-observe.png) - Install hooks for every detected harness: + Read the machine key into the shell. `read -s` takes it at a prompt that does not echo, so it never appears in a command or in shell history: ```bash - failproofai config \ - --connect https://app.befailproof.ai \ - --token - failproofai policies --install + read -rs FAILPROOFAI_CLOUD_TOKEN && export FAILPROOFAI_CLOUD_TOKEN ``` + Then set the machine up — this wires hooks for every detected harness, installs the daemon, and connects to Cloud: + + ```bash + failproofai config + failproofai policies add FailproofAI/policies + ``` + + Setup enables no policy on its own, which is what the second command is for. + Or target named harnesses and a configuration scope: ```bash diff --git a/docs/reference/overview.mdx b/docs/reference/overview.mdx index a39656a7e..c98399bf8 100644 --- a/docs/reference/overview.mdx +++ b/docs/reference/overview.mdx @@ -58,16 +58,19 @@ The generated [HTTP API reference](/reference/http-api) covers the public `/v1` Open one of these sessions before considering the integration complete; the trace should contain the model, tool, error, and policy evidence your audits need. - Create a machine key, connect the Failproof daemon, and verify the first session. + Create a machine key, then read the secret it prints into the shell. `read -s` takes it at a prompt that does not echo, so it never appears in a command or in shell history: ```bash fp keys create agent-production \ --add events:add \ --add policies:pull + read -rs FAILPROOFAI_CLOUD_TOKEN && export FAILPROOFAI_CLOUD_TOKEN + ``` + + Connect the Failproof daemon and verify the first session: - failproofai config \ - --connect https://app.befailproof.ai \ - --token + ```bash + failproofai config failproofai flush --wait fp sessions --since 1h --env production diff --git a/docs/reference/policy-sdk.mdx b/docs/reference/policy-sdk.mdx index 78aad8063..f31e027e8 100644 --- a/docs/reference/policy-sdk.mdx +++ b/docs/reference/policy-sdk.mdx @@ -6,7 +6,7 @@ icon: "shield-plus" Custom policies turn a failure pattern from your traces or audits into a decision that runs while an agent works. A policy can allow an action, give the agent guidance, or deny the action before it causes another incident. -Use a custom policy when the behavior depends on your tools, paths, commands, environments, or operating rules. Check the [built-in policy catalog](/policies/builtin-catalog) first so you do not recreate an existing control. +Use a custom policy when the behavior depends on your tools, paths, commands, environments, or operating rules. Check the [Failproof AI policy pack](/policies/packs) first so you do not recreate an existing control. ## Author a custom policy diff --git a/docs/ru/audits/recipes.mdx b/docs/ru/audits/recipes.mdx index 5bfc2f4e9..cd4f0f0ad 100644 --- a/docs/ru/audits/recipes.mdx +++ b/docs/ru/audits/recipes.mdx @@ -1,5 +1,4 @@ --- ---- title: "Рецепты аудитов" description: "Начальные цели для типичных расследований сбоев агентов." icon: "book-open-check" diff --git a/docs/ru/audits/setup.mdx b/docs/ru/audits/setup.mdx index 499cc9937..623c926a5 100644 --- a/docs/ru/audits/setup.mdx +++ b/docs/ru/audits/setup.mdx @@ -1,5 +1,4 @@ --- ---- title: "Настройка аудита" description: "Определите цель аудита, популяцию сессий и контекст доказательств." icon: "sliders-horizontal" diff --git a/docs/ru/policies/builtin-catalog.mdx b/docs/ru/policies/builtin-catalog.mdx deleted file mode 100644 index 728daf173..000000000 --- a/docs/ru/policies/builtin-catalog.mdx +++ /dev/null @@ -1,132 +0,0 @@ ---- -title: "Каталог встроенных политик" -description: "Просмотрите каждую встроенную политику Failproof AI, её триггер, рекомендуемое состояние и настраиваемые параметры." -icon: "list-checks" ---- - -Установленный пакет является источником истины для доступности политик. Запустите `failproofai policies` после каждого обновления, так как записи каталога и поведение могут меняться в зависимости от версии пакета. - -## Рекомендуемая базовая конфигурация - -Рекомендуемый выбор в интерактивной установке в настоящее время включает санитайзеры секретов, защиту окружения, самозащиту, защиту от опасных команд и безопасность защищённых веток: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` **всегда включена**. Она указана выше для -полноты, но регистрируется при каждой проверке независимо от того, включена ли она в вашем наборе -активных политик, и не может быть отключена или приостановлена — защита, которую -агент может отключить, не является защитой. - -Рекомендуемый набор намеренно ограничен в сравнении с **Всеми**. Политики инфраструктуры и рабочего процесса могут прерывать допустимые операции и должны быть включены только для репозиториев и машин, которым они необходимы. - -## Секреты и окружение - -| Политика | Триггер | Результат | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | Удаляет JWT из вывода инструмента перед тем, как его увидит модель. | -| `sanitize-api-keys` | `PostToolUse` | Удаляет распространённые ключи OpenAI, Anthropic, GitHub, AWS, Stripe и Google. | -| `sanitize-connection-strings` | `PostToolUse` | Удаляет строки подключения к базам данных, содержащие учётные данные. | -| `sanitize-private-key-content` | `PostToolUse` | Удаляет содержимое PEM приватных ключей. | -| `sanitize-bearer-tokens` | `PostToolUse` | Удаляет токены авторизации в заголовке Authorization. | -| `protect-env-vars` | `PreToolUse` на shell-инструментах | Блокирует команды, которые выводят переменные окружения. | -| `block-env-files` | `PreToolUse` | Блокирует чтение и запись файлов `.env`. | -| `block-read-outside-cwd` | `PreToolUse` на инструментах чтения, glob, grep или shell | Ограничивает чтение директорией работы сеанса. | -| `block-secrets-write` | `PreToolUse` на инструментах записи | Блокирует запись в распространённые файлы с ключами секретов и учётными данными. | - -## Опасные команды и инфраструктура - -| Политика | Триггер | Результат | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | Блокирует `sudo`, если не совпадает разрешающий шаблон. | -| `block-curl-pipe-sh` | `PreToolUse` | Блокирует загруженные скрипты, передаваемые напрямую в shell. | -| `block-rm-rf` | `PreToolUse` | Блокирует опасные шаблоны рекурсивного удаления. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Всегда включена, не может быть отключена.** Блокирует каждый вызов CLI Failproof AI, самопауз и удаление пакета. | -| `block-kubectl` | `PreToolUse` | Ограничивает команды Kubernetes. | -| `block-terraform` | `PreToolUse` | Ограничивает команды Terraform и OpenTofu. | -| `block-aws-cli` | `PreToolUse` | Ограничивает команды AWS CLI. | -| `block-gcloud` | `PreToolUse` | Ограничивает команды Google Cloud CLI. | -| `block-az-cli` | `PreToolUse` | Ограничивает команды Azure CLI. | -| `block-helm` | `PreToolUse` | Ограничивает команды Helm. | -| `block-gh-pipeline` | `PreToolUse` | Ограничивает операции GitHub CLI, изменяющие workflow, run, merge, release, cache и secrets. | - -## Безопасность Git и базы данных - -| Политика | Триггер | Результат | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Блокирует прямые push в настроенные защищённые ветки. | -| `block-force-push` | `PreToolUse` | Блокирует force-push; `--force-with-lease` остаётся разрешённым в текущей реализации. | -| `block-work-on-main` | `PreToolUse` | Блокирует коммиты и merge на защищённых ветках. | -| `warn-git-amend` | `PreToolUse` | Предупреждает перед переписыванием коммита с `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | Предупреждает перед необратимым удалением или очисткой stash. | -| `warn-all-files-staged` | `PreToolUse` | Предупреждает при широком `git add -A`, `git add .` или `git add --all`. | -| `warn-destructive-sql` | `PreToolUse` | Предупреждает о `DROP`, `TRUNCATE` и `DELETE` без `WHERE` через известные клиенты БД. | -| `warn-schema-alteration` | `PreToolUse` | Предупреждает о известных операциях `ALTER TABLE` с колонками и переименованием. | - -## Пакеты, поведение системы и циклы агента - -| Политика | Триггер | Результат | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Предупреждает перед публикацией в реестры пакетов. | -| `warn-global-package-install` | `PreToolUse` | Предупреждает перед глобальной установкой пакетов. | -| `prefer-package-manager` | `PreToolUse` | Указывает агенту использовать разрешённый менеджер пакетов. | -| `warn-large-file-write` | `PreToolUse` на инструментах записи | Предупреждает при превышении настроенного порога размера файла. | -| `warn-background-process` | `PreToolUse` | Предупреждает о фоновых процессах в отдельном сеансе или долгоживущих шаблонах. | -| `warn-repeated-tool-calls` | `PreToolUse` | Предупреждает после трёх или более идентичных вызовов инструмента. | - -## Рабочий процесс завершения задачи - -Эти политики требуют обработчика, который генерирует совместимое событие `Stop`. - -| Политика | Результат | -| --- | --- | -| `require-commit-before-stop` | Отказывает завершение, пока остаётся отслеживаемая работа без коммита. | -| `require-push-before-stop` | Отказывает завершение, пока коммиты остаются только локальными. | -| `require-pr-before-stop` | Требует pull request для текущей ветки. | -| `require-no-conflicts-before-stop` | Требует чистый merge с настроенной базовой веткой. | -| `require-ci-green-before-stop` | Требует успешного завершения проверок CI для текущего HEAD. | - -## Справочник параметров - -Настраивайте параметры в объекте `policyParams` выбранной области. Типы проверяются каждой политикой. - -| Политика | Параметр | Тип и значение по умолчанию | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; записи содержат `regex` и `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Ограничители инфраструктуры | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - Разрешающий шаблон расширяет допустимые действия агента. Протестируйте точную токенизацию и варианты команд на целевом обработчике перед развёртыванием на множестве машин. - \ No newline at end of file diff --git a/docs/ru/policies/builtin.mdx b/docs/ru/policies/builtin.mdx deleted file mode 100644 index b084f1827..000000000 --- a/docs/ru/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Встроенные политики" -description: "Включите поддерживаемые гарантии безопасности для типичных режимов отказа агентов." -icon: "library" ---- - -Встроенные политики охватывают обработку секретов, файлы окружения, деструктивные команды оболочки, защищённые ветки, облачные и инфраструктурные инструменты, публикацию пакетов, повторяющиеся вызовы и проверки рабочего процесса конца задачи. - -## Включение и проверка встроенной политики - - - - 1. Установите политику на подключённой машине с помощью локального CLI. - 2. Выполните безопасное тестовое действие в инструментированном агенте. - 3. Перейдите в **Observe → policy** и отфильтруйте по имени политики, среде машины или решению. - 4. Откройте связанную сессию, чтобы подтвердить совпадающий ввод инструмента и возвращённую причину. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - Удалите её с помощью `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -Получите список доступных политик в установленной версии: - -```bash -failproofai policies -``` - -Включите одну политику для проекта: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Включите несколько политик для выбранных интеграций: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Некоторые политики принимают параметры или помечены как бета. Перед развёртыванием ознакомьтесь с описанием, областью действия и поведением по умолчанию. Политика, защищающая один рабочий процесс, может блокировать допустимые операции в другом. - - - Ознакомьтесь со всеми 40 текущими политиками, их триггерами, рекомендуемой базовой конфигурацией и параметрами. - - - - Предпочитайте область проекта для зависящих от репозитория ожиданий и область пользователя для требований безопасности на уровне машины. - \ No newline at end of file diff --git a/docs/ru/policies/custom.mdx b/docs/ru/policies/custom.mdx deleted file mode 100644 index ce8946bd7..000000000 --- a/docs/ru/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Пользовательские политики" -description: "Напишите политику для режима отказа, уникального для вашего рабочего процесса агента." -icon: "shield-plus" ---- - -Создайте файл с расширением `policies.js`, `policies.mjs` или `policies.ts` в папке `.failproofai/policies/`. Файлы соглашений загружаются автоматически на уровне проекта и пользователя. - -## Протестируйте политику перед публикацией в Cloud - - - - 1. Установите пользовательскую политику на одну тестовую машину и запустите как совпадающее действие, так и легитимное несовпадение. - 2. Перейдите в **Observe → policy** и сравните два решения. - 3. Откройте каждую связанную сессию и проверьте, содержит ли полезная нагрузка события достаточно доказательств для правила. - 4. Когда поведение будет правильным, переместите проверенный исходный код в **Admin → policy editor** и опубликуйте версию. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - Файлы соглашений в `.failproofai/policies/` загружаются без флага `--custom`. Сохраняйте явную команду установки в CI, когда проверка должна завершиться ошибкой при наличии неработающего модуля. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -Это совпадает с `production/config.yml`, `/srv/production/config.yml`, `/srv/production` и `C:\\production\\config.yml` как для `Write`, так и для `Edit`. Это не совпадает с именами вроде `production-backup`, потому что `production` должен быть полным сегментом пути. - -Проверьте и установите явный файл: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -Контекст политики включает тип события, нормализованную полезную нагрузку, имя и входные данные инструмента, метаданные сессии, параметры и исходный CLI, если доступен. - -## Протестируйте пути отказа - -Запустите проверку после изменения файла входа или любого локального модуля, который он импортирует: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -Строгий путь CLI завершается ошибкой для отсутствующих файлов, синтаксических ошибок, неразрешённых импортов, исключений верхнего уровня и истечения времени загрузки модуля. Во время применения нарушение пользовательского файла регистрируется и пропускается, чтобы встроенные политики могли продолжить работу. Рассматривайте любое предупреждение о загрузке как потерю ожидаемого применения и установите на него оповещение в журналах производства. - -Используйте глобально уникальные имена во всех явных, соглашениях и управляемых Cloud политиках. Обеспечьте детерминированность функций политик, ограничьте внешние вызовы короткими тайм-аутами и возвращайте намеренный `allow`, `instruct` или `deny` на каждом пути выполнения. - - - Пользовательская политика — это код применения. Тестируйте отсутствующие поля, альтернативные имена инструментов и неправильно сформированные входные данные — не только ожидаемое совпадение. - \ No newline at end of file diff --git a/docs/ru/policies/fleet.mdx b/docs/ru/policies/fleet.mdx deleted file mode 100644 index 187c087de..000000000 --- a/docs/ru/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Развертывание политик на машинах" -description: "Узнайте, какие машины зарегистрированы, актуальны и применяют нужные версии политик." -icon: "network" ---- - -Покрытие парка отвечает на вопрос, существует ли политика там, где есть риск. Отслеживайте машины по стабильному ID и понятной для человека метке, затем сравните назначенное и сообщаемое состояние развертывания. - -## Проверка покрытия - - - - 1. Перейдите в **Admin → enforcement** и ознакомьтесь с общим количеством применяемых и мониторируемых политик. - 2. Выполните поиск машины по ID или метке, или отфильтруйте машины без политики. - 3. Разверните строку, чтобы сравнить назначенные политики, сообщаемое развертывание, время последней проверки и историю. - 4. Обновите страницу после интервала опроса машины, если применённое развертывание остаётся ожидающим. - - ![Парк Enforcement, показывающий покрытие политик, состояние развертывания машины и назначения в режиме наблюдения и применения.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Используйте `fp events --agent-id --since 24h` для проверки того, что активность агента машины попадает в Cloud. - - - -Используйте представления покрытия для поиска: - -- Машин, которые никогда не загружали последнее развертывание -- Зарегистрированных машин, которые перестали сообщать о своей активности -- Политики, назначенной неправильной среде или когорте -- Расхождения версий после прерванного обновления - -Переименуйте машину без её переподключения: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Проверьте локальное состояние: - -```bash -failproofai config --status -``` - - - Используйте метки, которые идентифицируют рабочую нагрузку и среду. Только имён хостов часто недостаточно после автоскалирования или замены машины. - \ No newline at end of file diff --git a/docs/ru/policies/publish-a-pack.mdx b/docs/ru/policies/publish-a-pack.mdx index 92fffcbdd..5298b4508 100644 --- a/docs/ru/policies/publish-a-pack.mdx +++ b/docs/ru/policies/publish-a-pack.mdx @@ -1,5 +1,4 @@ --- ---- title: "Опубликовать пак" description: "Распространяйте собственные политики как релиз на GitHub, который сможет установить любой." icon: "upload" diff --git a/docs/ru/sessions/assistant.mdx b/docs/ru/sessions/assistant.mdx index 54fe9e1bb..8831ed5b1 100644 --- a/docs/ru/sessions/assistant.mdx +++ b/docs/ru/sessions/assistant.mdx @@ -1,5 +1,4 @@ --- ---- title: "Failproof Assistant" description: "Анализируйте и управляйте Failproof AI на естественном языке: от вопросов и запросов до панелей управления и аудитов." icon: "message-square-text" diff --git a/docs/ru/start/integrations/crewai.mdx b/docs/ru/start/integrations/crewai.mdx index 67004ea67..378cdce16 100644 --- a/docs/ru/start/integrations/crewai.mdx +++ b/docs/ru/start/integrations/crewai.mdx @@ -1,5 +1,4 @@ --- ---- title: "CrewAI" sidebarTitle: "CrewAI" description: "Оркестрируйте команды, потоки, агентов по ролям, инструменты, память и обратную связь от пользователей." diff --git a/docs/sessions/evaluations.mdx b/docs/sessions/evaluations.mdx index cbeb69c66..beaf52bc4 100644 --- a/docs/sessions/evaluations.mdx +++ b/docs/sessions/evaluations.mdx @@ -1,25 +1,25 @@ --- -title: "Online evaluations" -description: "Score live and completed sessions for quality, compliance, cost, and latency." +title: "Read evaluation results" +description: "Chart evaluation scores over time, compare agents and environments, see why a session scored low, and ask the assistant." icon: "gauge" --- -Online evaluations apply consistent judgments to agent sessions. Use them for signals that should be measured continuously rather than investigated only during an audit. +Results from every evaluation, hosted or from your own worker, land in the same places. -## Review evaluation quality +## Compare scores over time - 1. Go to **Observe → Evaluations**. - 2. Add a series and choose the agent, environment, evaluation score, statistic, and curve. - 3. Add series to compare environments, agents, or score keys. - 4. Select a result to open matching sessions or share the filtered view. Use **Observe → Metrics** for latency, tokens, cost, and other magnitude values. + Go to **Observe → evaluations**. - ![A quality dashboard showing average evaluation scores and trends over time.](/images/dashboard/dashboard-quality.png) + - **Recent runs** lists each evaluation as it lands: whether it came from a hosted (**managed**) or your own (**customer**) evaluator, the agent and session, the evaluation and its version, its status, and its score or metrics. + - **Score over time** plots what you ask for. Select **add series** and choose an agent, an environment, an evaluation, and a statistic: avg, min, max, p50, p75, p90, p95, p99, stddev, or mode. Each series is one line; give it its own **curve** to draw it on a separate chart. - Open a session from the drill-down to inspect its per-score reasoning: + ![The evaluations page: recent runs tagged customer, a score over time chart with reference lines at 0.5 and 0.8, and one series averaging finished_clean across all agents and environments.](/images/dashboard/evaluations-chart.png) - ![A session detail view showing evaluation scores and reasoning beside the complete trace.](/images/dashboard/session-detail.png) + One time range and one bin size apply to every series. A fine bin finds an incident; a coarse one shows a trend, and can hide the spikes you are looking for. A bucket where nothing was scored is a gap in the line, never a zero, and reference lines mark 0.5 and 0.8. + + Every part of the view lives in the URL: **share** copies it, and whoever opens it sees exactly the comparison you built. ```bash @@ -32,21 +32,25 @@ Online evaluations apply consistent judgments to agent sessions. Use them for si -An evaluator receives the session identity, environment, timestamps, and ordered events. It can return numeric score keys with optional reasoning and a summary. Long-running evaluators can return a pending job and be polled later. +Plot **avg** and **p90** for the same evaluation to see whether a good average is hiding a bad tail, or the same evaluation for two agents, or for production and staging, to compare them on one axis. Costs, latencies, and token counts, which carry units, chart under **Observe → metrics**, one chart per unit. + +## See why a session scored low + +Open a session from **Observe → sessions**; the grid carries each session's scores and filters by score range. The session's right rail leads with the evaluation summary, then a bar per score with the evaluator's reasoning under it. + +![A session detail view showing evaluation scores and reasoning beside the complete trace.](/images/dashboard/session-detail.png) + +## Ask the assistant + +Ask about evaluation data in plain English: "tell me about some of the recent evaluations", or which agents' scores are slipping. The [assistant](/sessions/assistant) reads and analyzes the results and answers with tables you can follow up on, and a question worth keeping can become a [query](/sessions/queries) or a [dashboard](/sessions/dashboards). -## Good evaluation targets +![The evaluations page beside the assistant, which answers "tell me about some of the recent evaluations" with a summary of totals, statuses, and scores.](/images/dashboard/evaluations-assistant.png) -- Task completion or correctness -- Groundedness and hallucination risk -- Tool selection and tool efficiency -- Policy or process compliance -- Cost and latency budgets -- Required human escalation +## Watch and act -## From score to response +- **Dashboards**, under **Analyze → dashboards**, trend the scores you feature, per agent and environment, for the whole organization. -Show scores in dashboards to track trends. Create alerts for thresholds or compound conditions. When a score declines across a population, run an audit to investigate why; when the cause is a repeatable action, deploy a policy. + ![A quality dashboard showing average evaluation scores and trends over time.](/images/dashboard/dashboard-quality.png) - - Implement synchronous or asynchronous evaluation with the Python evaluator SDK. - +- **Alerts** notify you when a score crosses a threshold. See [alerts](/audits/alerts). +- When a score declines across many sessions, [run an audit](/audits/run) to find out why; when the cause is a repeatable action, [write a policy](/policies/editor). diff --git a/docs/start/quickstart.mdx b/docs/start/quickstart.mdx index 31f47f690..6d98a26d7 100644 --- a/docs/start/quickstart.mdx +++ b/docs/start/quickstart.mdx @@ -30,10 +30,10 @@ This quickstart gets one machine reporting sessions, runs an audit, and deploys 1. Open the [Failproof AI dashboard](https://app.befailproof.ai) and create an account or sign in with your work email. 2. Go to **Administration → Keys** and create a key with `events:add` and `policies:pull`. -3. Copy the one-time secret and store it on the target machine: +3. Copy the one-time secret, then read it into a shell on the target machine. `read -s` takes it at a prompt that does not echo, so it never appears in a command: ```bash -export FAILPROOFAI_KEY="" +read -rs FAILPROOFAI_KEY && export FAILPROOFAI_KEY ``` ## Install @@ -42,11 +42,17 @@ export FAILPROOFAI_KEY="" ```bash npm install -g failproofai - failproofai config --connect https://app.befailproof.ai --token "$FAILPROOFAI_KEY" + FAILPROOFAI_CLOUD_TOKEN="$FAILPROOFAI_KEY" failproofai config ``` + That one command is the whole of setup: it installs the local daemon (root once), wires hooks into every agent CLI it finds, and connects this machine to Cloud. Passing the key through the environment rather than `--token` keeps it out of `ps`, where every user on the machine can read a command's arguments. It does not keep it out of shell history — reading it with `read -s` is what does that. In CI, inject it as a masked secret and keep shell tracing (`set -x`) off, or the trace prints it. + Session transcripts are sent by default. Add `--no-transcripts` to report hook activity and policy decisions without transcript content. + + Do not reach for `failproofai config --connect ` here. That flag enrols a machine that is **already** set up and returns straight after — no daemon, no hooks — so the machine would appear in Cloud while collecting and enforcing nothing. + + If this machine already has agent history, preview and import the last seven days, then wait for delivery to finish. Skip this step on a new machine. ```bash @@ -57,10 +63,8 @@ export FAILPROOFAI_KEY="" Open **Sessions** in Failproof AI and select an imported session. - - This attaches Failproof AI to your harness and installs the 39 built-in policies. Use them to see local policy decisions and try enforcement before Failproof AI audits your sessions and writes policies for your agents. - - Let the installer detect your harness, or name one explicitly. Every one of the 12 is a valid `--cli` value — `claude`, `codex`, `copilot`, `cursor`, `opencode`, `pi`, `hermes`, `openclaw`, `factory`, `devin`, `antigravity`, `goose`. + + The previous step already wired every agent CLI it detected. Re-run it for one harness explicitly when you need to, or to add a harness installed afterwards. Every one of the 12 is a valid `--cli` value — `claude`, `codex`, `copilot`, `cursor`, `opencode`, `pi`, `hermes`, `openclaw`, `factory`, `devin`, `antigravity`, `goose`. ```bash failproofai policies --install --cli claude --scope user # a coding CLI @@ -69,6 +73,19 @@ export FAILPROOFAI_KEY="" Blocking a tool call before it runs is verified on all 12. Turn-end gates are verified on 8 — see [enforcement capability](/reference/harnesses#enforcement-capability) for the per-harness matrix. + + Wiring hooks enables no policy. Setup deliberately picks none — that decision is yours — so take a pack: + + ```bash + failproofai policies add FailproofAI/policies + ``` + + The pack is fetched from its GitHub release, checksum-verified, and pinned to the exact tag it resolved. It carries 38 policies and switches on the 10 its manifest marks as safe to enable unattended. Use them to see local policy decisions and try enforcement before Failproof AI audits your sessions and writes policies for your agents. + + Read any pack before taking it with `failproofai policies show /`, and see [policy packs](/policies/packs) for taking only part of one. + + Until this runs, the only thing enforcing is `block-failproofai-commands` — the always-on guard that stops an agent switching Failproof AI off. `failproofai policies` lists what is on. + Follow [Run your first failure check](/start/first-audit). Use a concrete goal such as “find sessions where the agent retried a failing tool without changing its approach.” diff --git a/docs/start/setup.mdx b/docs/start/setup.mdx index b88165d66..8a3e2e211 100644 --- a/docs/start/setup.mdx +++ b/docs/start/setup.mdx @@ -6,7 +6,7 @@ icon: "waypoints" - Install hooks and policies on a machine. Use this when you need immediate guardrails without sending session data to Cloud. + Set a machine up with no Cloud key and take a policy pack. Use this when you need immediate guardrails without sending session data to Cloud. Add centralized sessions, audits, online evaluations, dashboards, alerts, and fleet policy deployment. @@ -16,6 +16,10 @@ icon: "waypoints" +## Enforce locally + +Run `failproofai config` without a key, then take a pack with `failproofai policies add FailproofAI/policies`. In a terminal, pick **Not now — stay local** when setup asks to connect to Cloud; with no terminal and no `FAILPROOFAI_CLOUD_TOKEN`, it stays local on its own. The daemon and hooks enforce on the machine, and no session data is sent to Cloud. To connect later, follow the steps below. + ## Recommended production path 1. Connect a non-production machine with transcript capture enabled. @@ -48,16 +52,33 @@ icon: "waypoints" Continue only after both the machine and its first event are visible. + Read the one-time secret into the shell. `read -s` takes it at a prompt that does not echo, so it never appears in a command or in shell history: + ```bash - failproofai config --connect https://app.befailproof.ai \ - --token "$FAILPROOFAI_KEY" \ - --machine-label checkout-runner-01 + read -rs FAILPROOFAI_CLOUD_TOKEN && export FAILPROOFAI_CLOUD_TOKEN + ``` + + Then set the machine up, choose its policies, and name it: - failproofai policies --install --cli claude --scope user + ```bash + failproofai config + + failproofai policies add FailproofAI/policies + failproofai config --machine-label checkout-runner-01 failproofai config --status ``` + `failproofai config` does the whole setup — daemon, hooks for every agent CLI it finds, and the Cloud connection — then chooses no policies, which is what the second command is for. + + The label comes **after** connecting, not during: `failproofai config --machine-label ` renames a machine that is already connected, and on one that is not it does nothing but say so. + Add `--no-transcripts` when transcript content must remain local. + + In CI, set `FAILPROOFAI_CLOUD_TOKEN` from the secret store instead of `read -s`, and keep shell tracing (`set -x`) off, or the trace prints the key. + + + On a machine that is **already** set up, `failproofai config --connect ` enrols it and nothing else. Do not use that form for a first install: it returns before the daemon or any hook is in place, leaving a machine that shows up in Cloud but collects and enforces nothing. + diff --git a/docs/tr/admin/keys-and-permissions.mdx b/docs/tr/admin/keys-and-permissions.mdx index 1250cd0b1..08d6f7da6 100644 --- a/docs/tr/admin/keys-and-permissions.mdx +++ b/docs/tr/admin/keys-and-permissions.mdx @@ -1,5 +1,4 @@ --- ---- title: "Anahtarlar ve izinler" description: "Makineler, otomasyon ve operatörler için kapsamlı API anahtarları oluşturun." icon: "key-round" diff --git a/docs/tr/admin/usage.mdx b/docs/tr/admin/usage.mdx index be7adcaa7..76a728155 100644 --- a/docs/tr/admin/usage.mdx +++ b/docs/tr/admin/usage.mdx @@ -1,5 +1,4 @@ --- ---- title: "Kullanım" description: "Kuruluş tüketimini ve etkin faturalama dönemini inceleyin." icon: "chart-no-axes-combined" diff --git a/docs/tr/admin/users-and-organizations.mdx b/docs/tr/admin/users-and-organizations.mdx index de032c3c2..f082e362f 100644 --- a/docs/tr/admin/users-and-organizations.mdx +++ b/docs/tr/admin/users-and-organizations.mdx @@ -1,5 +1,4 @@ --- ---- title: "Kullanıcılar ve kuruluşlar" description: "Üyelik kontrolü ve her kuruluşun verilerini ve işlemlerini kapsamlı tutun." icon: "users" diff --git a/docs/tr/audits/alerts.mdx b/docs/tr/audits/alerts.mdx index ea9bc133d..bc735942a 100644 --- a/docs/tr/audits/alerts.mdx +++ b/docs/tr/audits/alerts.mdx @@ -1,5 +1,4 @@ --- ---- title: "Uyarılar" description: "Tekrarlayan sorunları algılayın ve olayı uygun müdahalelere yönlendirin." icon: "bell-ring" diff --git a/docs/tr/audits/overview.mdx b/docs/tr/audits/overview.mdx index 34eaa75dd..5b696362e 100644 --- a/docs/tr/audits/overview.mdx +++ b/docs/tr/audits/overview.mdx @@ -1,5 +1,4 @@ --- ---- title: "Denetimler" description: "Tanı izleri tek başına ortaya çıkarmayacak başarısızlıkları bulmak için belirlenmiş bir oturum popülasyonunu gözden geçirin." icon: "scan-search" diff --git a/docs/tr/audits/recipes.mdx b/docs/tr/audits/recipes.mdx index 9252fa273..7e7293bc0 100644 --- a/docs/tr/audits/recipes.mdx +++ b/docs/tr/audits/recipes.mdx @@ -1,5 +1,4 @@ --- ---- title: "Denetim reçeteleri" description: "Yaygın aracı arıza soruşturmaları için başlangıç hedefleri." icon: "book-open-check" diff --git a/docs/tr/policies/builtin-catalog.mdx b/docs/tr/policies/builtin-catalog.mdx deleted file mode 100644 index dbf1755e1..000000000 --- a/docs/tr/policies/builtin-catalog.mdx +++ /dev/null @@ -1,130 +0,0 @@ ---- ---- -title: "Dahili politika kataloğu" -description: "Her bir Failproof AI dahili politikasını, tetikleyicisini, önerilen durumunu ve yapılandırılabilir parametrelerini gözden geçirin." -icon: "list-checks" ---- - -Yüklü paket, politika kullanılabilirliğinin kaynağıdır. Her yükseltmeden sonra `failproofai policies` çalıştırın çünkü katalog girdileri ve davranış paket sürümüyle değişebilir. - -## Önerilen temel yapı - -Kılavuzlu kurulumun önerilen seçimi şu anda gizli dizi sterilizatörleri, ortam korumaları, kendi kendini koruma, felaket komut korumaları ve korumalı dal güvenliğini etkinleştirir: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` **her zaman açıktır**. Bütünlük açısından yukarıda listelenmiştir, ancak etkinleştirilmiş setinizde yer alıp almadığına bakılmaksızın her değerlendirmede kaydedilir ve devre dışı bırakılamaz veya duraklatalamaz — aracının uygulamayı kapatabildiği bir koruma, koruma değildir. - -Önerilen yapı, **Her Şey**'den kasıtlı olarak daha dar kapsamlıdır. Altyapı ve iş akışı politikaları geçerli çalışmayı kesintiye uğratabilir ve bunları ihtiyaç duyan depolar ve makinelerde etkinleştirilmelidir. - -## Gizli diziler ve ortam - -| Politika | Tetikleyici | Sonuç | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | Araç çıktısından JWT'leri model görmeden önce düşür. | -| `sanitize-api-keys` | `PostToolUse` | Yaygın OpenAI, Anthropic, GitHub, AWS, Stripe ve Google anahtarlarını düşür. | -| `sanitize-connection-strings` | `PostToolUse` | Kimlik bilgileri içeren veritabanı bağlantı dizelerini düşür. | -| `sanitize-private-key-content` | `PostToolUse` | PEM özel anahtar gövdelerini düşür. | -| `sanitize-bearer-tokens` | `PostToolUse` | Yetkilendirme taşıyıcı jetonlarını düşür. | -| `protect-env-vars` | `PreToolUse` kabuk araçlarında | Ortam değişkenlerini döken komutları engelle. | -| `block-env-files` | `PreToolUse` | `.env` dosyalarının okunmasını ve yazılmasını engelle. | -| `block-read-outside-cwd` | `PreToolUse` okuma, glob, grep veya kabuk araçlarında | Okumaları oturum çalışma dizini içinde tut. | -| `block-secrets-write` | `PreToolUse` yazma araçlarında | Yaygın gizli anahtar ve kimlik bilgisi dosya adlarına yazmaları engelle. | - -## Tehlikeli komutlar ve altyapı - -| Politika | Tetikleyici | Sonuç | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | İzin deseni eşleşmediği sürece `sudo`'yu engelle. | -| `block-curl-pipe-sh` | `PreToolUse` | İndirilen komutları doğrudan kabuğa aktaran betikleri engelle. | -| `block-rm-rf` | `PreToolUse` | Felaket niteliğinde özyinelemeli silme kalıplarını engelle. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Her zaman açık, devre dışı bırakılamaz.** Her Failproof AI CLI çağırmasını, kendi kendini duraklat işlemini ve paket yöneticisi kaldırma işlemini engelle. | -| `block-kubectl` | `PreToolUse` | Kubernetes komutlarını kısıtla. | -| `block-terraform` | `PreToolUse` | Terraform ve OpenTofu komutlarını kısıtla. | -| `block-aws-cli` | `PreToolUse` | AWS CLI komutlarını kısıtla. | -| `block-gcloud` | `PreToolUse` | Google Cloud CLI komutlarını kısıtla. | -| `block-az-cli` | `PreToolUse` | Azure CLI komutlarını kısıtla. | -| `block-helm` | `PreToolUse` | Helm komutlarını kısıtla. | -| `block-gh-pipeline` | `PreToolUse` | Değiştirici GitHub CLI iş akışı, çalıştırma, birleştirme, sürüm, önbellek ve gizli işlemleri kısıtla. | - -## Git ve veritabanı güvenliği - -| Politika | Tetikleyici | Sonuç | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Yapılandırılmış korumalı dallara doğrudan göndermeleri engelle. | -| `block-force-push` | `PreToolUse` | Zorla gönderi işlemlerini engelle; `--force-with-lease` mevcut uygulamada izinli kalır. | -| `block-work-on-main` | `PreToolUse` | Korumalı dallardaki işlemeleri ve birleştirmeleri engelle. | -| `warn-git-amend` | `PreToolUse` | `--amend` ile bir işlemeyi yeniden yazmadan önce uyar. | -| `warn-git-stash-drop` | `PreToolUse` | Depo durumunu kalıcı olarak bırakmadan veya temizlemeden önce uyar. | -| `warn-all-files-staged` | `PreToolUse` | Geniş `git add -A`, `git add .` veya `git add --all` için uyar. | -| `warn-destructive-sql` | `PreToolUse` | Tanınan veritabanı istemcileri aracılığıyla `WHERE` olmadan `DROP`, `TRUNCATE` ve `DELETE` işlemleri için uyar. | -| `warn-schema-alteration` | `PreToolUse` | Tanınan `ALTER TABLE` sütun ve yeniden adlandırma işlemleri için uyar. | - -## Paketler, sistem davranışı ve aracı döngüleri - -| Politika | Tetikleyici | Sonuç | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Paket kayıtlarına yayınlamadan önce uyar. | -| `warn-global-package-install` | `PreToolUse` | Genel paket kurulumundan önce uyar. | -| `prefer-package-manager` | `PreToolUse` | Aracıyı izin verilen bir paket yöneticisini kullanmaya yönlendir. | -| `warn-large-file-write` | `PreToolUse` yazma araçlarında | Yapılandırılmış dosya boyutu eşiğinin üzerinde uyar. | -| `warn-background-process` | `PreToolUse` | Ayrılmış veya uzun ömürlü arka plan süreci kalıplarını uyar. | -| `warn-repeated-tool-calls` | `PreToolUse` | Üç veya daha fazla özdeş araç çağrısından sonra uyar. | - -## Görev sonu iş akışı - -Bu politikalar, uyumlu bir `Stop` etkinliği yayan bir aparat gerektirir. - -| Politika | Sonuç | -| --- | --- | -| `require-commit-before-stop` | İzlenen çalışma işlenmemiş kalırken tamamlamayı reddet. | -| `require-push-before-stop` | İşlemeler yalnızca yerel kalırken tamamlamayı reddet. | -| `require-pr-before-stop` | Geçerli dal için bir çekme isteği gerektirir. | -| `require-no-conflicts-before-stop` | Yapılandırılmış temel dal karşısında temiz bir birleştirme gerektirir. | -| `require-ci-green-before-stop` | Geçerli-HEAD CI kontrollerinin başarıyla tamamlanmasını gerektirir. | - -## Parametre referansı - -Seçilen kapsamın `policyParams` nesnesi altında parametreleri yapılandırın. Türler her politika tarafından doğrulanır. - -| Politika | Parametre | Tür ve varsayılan | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; girdiler `regex` ve `label` içerir | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Altyapı engelleyicileri | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - İzin deseni, aracının yapabileceği işlemleri genişletir. Hedef apar üzerinde tam jetonlaştırma ve komut varyantlarını dağıtmadan önce test edin. - \ No newline at end of file diff --git a/docs/tr/policies/builtin.mdx b/docs/tr/policies/builtin.mdx deleted file mode 100644 index 393e95176..000000000 --- a/docs/tr/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Yerleşik politikalar" -description: "Aracı hatalarının yaygın modları için bakımı yapılan koruma mekanizmalarını etkinleştirin." -icon: "library" ---- - -Yerleşik politikalar, gizli bilgi işleme, ortam dosyaları, yıkıcı kabuk komutları, korumalı dallar, bulut ve altyapı araçları, paket yayınlama, tekrarlanan çağrılar ve görev sonlandırması iş akışı denetimlerini kapsar. - -## Yerleşik bir politikayı etkinleştirme ve doğrulama - - - - 1. Politikayı bağlı bir makinede yerel CLI ile kurun. - 2. İşleme alınan aracıda güvenli bir test işlemi çalıştırın. - 3. **Observe → policy** bölümüne gidin ve politika adı, makine ortamı veya karar ile filtreleyin. - 4. Bağlantılı oturumu açarak eşleşen araç girişini ve döndürülen nedeni doğrulayın. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - `failproofai policy remove block-rm-rf --cli claude --scope project` ile kaldırın. - - - -Yüklü sürümünüzde bulunan politikaları listeleyin: - -```bash -failproofai policies -``` - -Bir proje için bir politikayı etkinleştirin: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Seçilen araçlar için birden fazla politikayı etkinleştirin: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Bazı politikalar parametreleri kabul eder veya beta olarak işaretlenmiş olabilir. Çıkıştan önce açıklamayı, eşleşme kapsamını ve varsayılan davranışı gözden geçirin. Bir iş akışını koruyan bir politika, başka bir iş akışında geçerli işlemleri engelleyebilir. - - - Mevcut 40 politikanın tümünü, tetikleyicilerini, önerilen temeli ve parametrelerini gözden geçirin. - - - - Depoya özgü beklentiler için proje kapsamını ve makine genelindeki güvenlik gereksinimleri için kullanıcı kapsamını tercih edin. - \ No newline at end of file diff --git a/docs/tr/policies/custom.mdx b/docs/tr/policies/custom.mdx deleted file mode 100644 index 74753111b..000000000 --- a/docs/tr/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Özel politikalar" -description: "Agent iş akışınıza özgü bir hata moduna yönelik bir politika yazın." -icon: "shield-plus" ---- - -`.failproofai/policies/` altında `policies.js`, `policies.mjs` veya `policies.ts` ile biten bir dosya oluşturun. Kural dosyaları proje ve kullanıcı kapsamında otomatik olarak yüklenir. - -## Politikayı Cloud yayını öncesinde test edin - - - - 1. Özel politikayı bir test makinesine yükleyin ve hem eşleşen bir işlemi hem de geçerli bir eşleşmeyen işlemi tetikleyin. - 2. **Observe → policy** öğesine gidin ve iki kararı karşılaştırın. - 3. Her bağlantılı oturumu açın ve olay yükünün kural için yeterli kanıt içerdiğini doğrulayın. - 4. Davranış doğru olduğunda, gözden geçirilen kaynağı **Admin → policy editor** öğesine taşıyın ve bir sürüm yayımlayın. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - `.failproofai/policies/` altındaki kural dosyaları `--custom` olmadan yüklenir. Doğrulama bozuk bir modülde başarısız olmalıysa CI'da açık bir yükleme komutu tutun. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -Bu, hem `Write` hem de `Edit` için `production/config.yml`, `/srv/production/config.yml`, `/srv/production` ve `C:\\production\\config.yml` ile eşleşir. `production-backup` gibi isimlerle eşleşmez çünkü `production` tam bir yol segmenti olmalıdır. - -Açık bir dosyayı doğrulayın ve yükleyin: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -Politika bağlamı olay türünü, normalleştirilmiş yükü, araç adını ve girdisini, oturum meta verilerini, parametreleri ve kullanılabilir olduğunda kaynak CLI'yi içerir. - -## Başarısızlık yollarını test edin - -Giriş dosyasını veya içeri aktardığı herhangi bir yerel modülü değiştirdikten sonra doğrulamayı çalıştırın: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -Katı CLI yolu eksik dosyalar, söz dizimi hataları, çözülmemiş içeri aktarmalar, üst düzey istisnalar ve modül yükleme zaman aşımları için başarısız olur. Zorlama sırasında, bozuk bir özel dosya günlüğe kaydedilir ve atlanır, böylece yerleşik politikalar devam edebilir. Herhangi bir yükleme uyarısını beklenen zorlama kaybı olarak değerlendirin ve bunu üretim günlüklerinde uyarı olarak ayarlayın. - -Açık, kural ve Cloud tarafından yönetilen politikalar arasında küresel olarak benzersiz adlar kullanın. Politika işlevlerini deterministic tutun, harici çağrıları kısa zaman aşımlarıyla sınırlayın ve her yolda ıchraç bir `allow`, `instruct` veya `deny` döndürün. - - - Özel bir politika, zorlama kodudur. Eksik alanları, alternatif araç adlarını ve hatalı girdiyi test edin — yalnızca beklenen eşleşmeyi değil. - \ No newline at end of file diff --git a/docs/tr/policies/fleet.mdx b/docs/tr/policies/fleet.mdx deleted file mode 100644 index d02d251cd..000000000 --- a/docs/tr/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Politikaları makinelere dağıtma" -description: "Hangi makinelerin kaydedildiğini, güncel olduğunu ve hedeflenen politika sürümlerini uyguladığını bilin." -icon: "network" ---- - -Fleet kapsamı, bir politikanın risk bulunan yerde var olup olmadığını cevaplar. Makineleri sabit kimlik ve insan tarafından okunabilir etiketle takip edin, ardından atanan ve bildirilen dağıtım durumlarını karşılaştırın. - -## Kapsamı kontrol etme - - - - 1. **Admin → enforcement** kısmına gidin ve uygulanan ve gözlenen toplam değerleri inceleyin. - 2. Bir makineyi kimlik veya etiketle arayın ya da politikası olmayan makineleri filtreleyip bulun. - 3. Atanan politikaları, bildirilen dağıtımı, son check-in zamanını ve geçmişi karşılaştırmak için bir satırı genişletin. - 4. Uygulanan dağıtım beklemede kalırsa, makinenin polling aralığından sonra yenileyin. - - ![Politika kapsamını, makine dağıtım durumunu ve gözlem ile uygulama atamalarını gösteren Enforcement fleet.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Makinenin agent aktivitesinin Cloud'a ulaştığını doğrulamak için `fp events --agent-id --since 24h` kullanın. - - - -Kapsamı görüntülemek için şunları bulun: - -- En son dağıtımı hiç çekmeyen makineler -- Aktivite raporlamayı durduran kayıtlı makineler -- Yanlış ortama veya kohort'a atanan politika -- İnkıta uğramış güncellemenin ardından sürüm kayması - -Makineyi yeniden bağlanmadan yeniden adlandırma: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Yerel durumu kontrol etme: - -```bash -failproofai config --status -``` - - - İş yükü ve ortamı tanımlayan etiketler kullanın. Otomatik ölçeklendirme veya makine değişiminden sonra tek başına ana bilgisayar adları genellikle yeterli değildir. - \ No newline at end of file diff --git a/docs/tr/reference/cloud-cli.mdx b/docs/tr/reference/cloud-cli.mdx index d68139bce..fa491b25b 100644 --- a/docs/tr/reference/cloud-cli.mdx +++ b/docs/tr/reference/cloud-cli.mdx @@ -1,5 +1,4 @@ --- ---- title: "Failproof Cloud CLI" description: "Failproof AI Cloud'u fp ile sorgulamak ve yönetmek için kapsamlı referans." icon: "cloud-cog" diff --git a/docs/tr/reference/evaluator-sdk.mdx b/docs/tr/reference/evaluator-sdk.mdx index d1567ed6b..ff1efbee2 100644 --- a/docs/tr/reference/evaluator-sdk.mdx +++ b/docs/tr/reference/evaluator-sdk.mdx @@ -1,5 +1,4 @@ --- ---- title: "Evaluator SDK" description: "Failproof AI oturumlarını senkron veya asenkron olarak puanlayan bir hizmet oluşturun." icon: "gauge" diff --git a/docs/tr/reference/harnesses.mdx b/docs/tr/reference/harnesses.mdx index 28694dd3b..440450f50 100644 --- a/docs/tr/reference/harnesses.mdx +++ b/docs/tr/reference/harnesses.mdx @@ -1,5 +1,4 @@ --- ---- title: "Agent araçları" description: "12 desteklenen agent aracında oturumları yakalayın ve politikaları uygulatın." icon: "plug-zap" diff --git a/docs/tr/reference/policy-sdk.mdx b/docs/tr/reference/policy-sdk.mdx index b297dc823..ef8e6fbf5 100644 --- a/docs/tr/reference/policy-sdk.mdx +++ b/docs/tr/reference/policy-sdk.mdx @@ -1,5 +1,4 @@ --- ---- title: "Özel politikalar" description: "Aracılarınıza özgü hatalar için JavaScript veya TypeScript politikaları yazın, test edin ve dağıtın." icon: "shield-plus" diff --git a/docs/tr/reference/troubleshooting.mdx b/docs/tr/reference/troubleshooting.mdx index 45a27e39a..b16640cdd 100644 --- a/docs/tr/reference/troubleshooting.mdx +++ b/docs/tr/reference/troubleshooting.mdx @@ -1,5 +1,4 @@ --- ---- title: "Sorun Giderme" description: "Eksik oturumları, eksik politikaları, başarısız teslimatı ve engellenen aracı işlemlerini tanılayın." icon: "wrench" diff --git a/docs/tr/sessions/overview.mdx b/docs/tr/sessions/overview.mdx index 29db18368..71d77f27b 100644 --- a/docs/tr/sessions/overview.mdx +++ b/docs/tr/sessions/overview.mdx @@ -1,5 +1,4 @@ --- ---- title: "Oturumlar" description: "Bir aracı çalışmasının tam kaydıyla başlayın." icon: "workflow" diff --git a/docs/tr/sessions/policy-decisions.mdx b/docs/tr/sessions/policy-decisions.mdx index e815d420e..d77fd8f4a 100644 --- a/docs/tr/sessions/policy-decisions.mdx +++ b/docs/tr/sessions/policy-decisions.mdx @@ -1,5 +1,4 @@ --- ---- title: "İlke kararları" description: "Hangi ilkelerin değerlendirildiğini, engellediğini, yönlendirdiğini veya izin verdiğini görün." icon: "shield-check" diff --git a/docs/tr/start/concepts.mdx b/docs/tr/start/concepts.mdx index 2a34a09d6..34f238916 100644 --- a/docs/tr/start/concepts.mdx +++ b/docs/tr/start/concepts.mdx @@ -1,5 +1,4 @@ --- ---- title: "Temel kavramlar" description: "Failproof AI genelinde kullanılan kavramların küçük bir seti." icon: "boxes" diff --git a/docs/tr/start/quickstarts/langchain.mdx b/docs/tr/start/quickstarts/langchain.mdx index 43a51a2a2..07e30b83a 100644 --- a/docs/tr/start/quickstarts/langchain.mdx +++ b/docs/tr/start/quickstarts/langchain.mdx @@ -1,5 +1,4 @@ --- ---- title: "LangChain ve LangGraph" description: "Kurun, enstrüman edin ve ilk izinizi görün." icon: "/images/frameworks/langchain.svg" diff --git a/docs/vi/admin/settings-and-security.mdx b/docs/vi/admin/settings-and-security.mdx index 5993dc912..ee694e80a 100644 --- a/docs/vi/admin/settings-and-security.mdx +++ b/docs/vi/admin/settings-and-security.mdx @@ -1,5 +1,4 @@ --- ---- title: "Cài đặt và bảo mật" description: "Định cấu hình các giá trị hoạt động cụ thể và đưa ra những lựa chọn có chủ đích về dữ liệu của agent." icon: "lock-keyhole" diff --git a/docs/vi/audits/agent-contracts.mdx b/docs/vi/audits/agent-contracts.mdx index 299a5ec56..4f7344995 100644 --- a/docs/vi/audits/agent-contracts.mdx +++ b/docs/vi/audits/agent-contracts.mdx @@ -1,5 +1,4 @@ --- ---- title: "Ngữ cảnh agent" description: "Cho các audit biết mỗi agent phải làm gì, tạo ra cái gì, và không được phép làm gì." icon: "bot" diff --git a/docs/vi/audits/local-audit.mdx b/docs/vi/audits/local-audit.mdx index a8db5db00..2021a704a 100644 --- a/docs/vi/audits/local-audit.mdx +++ b/docs/vi/audits/local-audit.mdx @@ -1,5 +1,4 @@ --- ---- title: "Kiểm toán lịch sử agent cục bộ" description: "Quét lịch sử CLI agent được hỗ trợ ngoại tuyến và xem xét hành vi rủi ro hoặc lãng phí cục bộ." icon: "laptop-minimal-check" diff --git a/docs/vi/audits/run.mdx b/docs/vi/audits/run.mdx index fd774facc..d893a96ea 100644 --- a/docs/vi/audits/run.mdx +++ b/docs/vi/audits/run.mdx @@ -1,5 +1,4 @@ --- ---- title: "Chạy và xem xét một audit" description: "Chạy một audit, xác minh phạm vi bao phủ của nó, và kiểm tra các phát hiện kết quả." icon: "play" diff --git a/docs/vi/audits/setup.mdx b/docs/vi/audits/setup.mdx index 11557ee9e..c61be6e3d 100644 --- a/docs/vi/audits/setup.mdx +++ b/docs/vi/audits/setup.mdx @@ -1,5 +1,4 @@ --- ---- title: "Thiết lập một audit" description: "Xác định mục tiêu audit, dân số phiên, và ngữ cảnh bằng chứng." icon: "sliders-horizontal" diff --git a/docs/vi/index.mdx b/docs/vi/index.mdx index 045fcc0b6..46cf81ccc 100644 --- a/docs/vi/index.mdx +++ b/docs/vi/index.mdx @@ -1,5 +1,4 @@ --- ---- title: "Làm cho agent của bạn không thể thất bại" description: "Khả năng quan sát và thực thi cho mọi công cụ mà agent chạy — CLI mã hóa, cổng trò chuyện, trợ lý tự lưu trữ và agent được đo lường của riêng bạn." icon: "shield-check" diff --git a/docs/vi/policies/builtin-catalog.mdx b/docs/vi/policies/builtin-catalog.mdx deleted file mode 100644 index e01c0908c..000000000 --- a/docs/vi/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "Danh mục chính sách tích hợp" -description: "Xem xét mọi chính sách Failproof AI tích hợp, kích hoạt, trạng thái được khuyến nghị, và các tham số có thể cấu hình." -icon: "list-checks" ---- - -Gói đã cài đặt là nguồn dữ liệu chính thức về tính khả dụng chính sách. Chạy `failproofai policies` sau mỗi lần nâng cấp vì các mục danh mục và hành vi có thể thay đổi theo phiên bản gói. - -## Cấu hình cơ sở được khuyến nghị - -Lựa chọn được khuyến nghị của hướng dẫn thiết lập hiện tại cho phép các công cụ vệ sinh bí mật, bảo vệ môi trường, tự bảo vệ, bảo vệ lệnh thảm họa, và an toàn nhánh được bảo vệ: - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` **luôn bật**. Nó được liệt kê ở trên để hoàn thiện, nhưng nó đăng ký trong mỗi lần đánh giá cho dù nó có xuất hiện trong tập hợp được bật của bạn hay không, và nó không thể bị tắt hay tạm dừng — đây là một bảo vệ chống lại việc agent tắt thực thi mà chính agent có thể tắt không phải là một bảo vệ. - -Được khuyến nghị ý định hẹp hơn **Everything**. Các chính sách cơ sở hạ tầng và quy trình làm việc có thể làm gián đoạn công việc hợp lệ và nên được bật cho các kho lưu trữ và máy cần chúng. - -## Bí mật và môi trường - -| Chính sách | Kích hoạt | Kết quả | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | Xóa JWTs khỏi kết quả công cụ trước khi mô hình nhìn thấy chúng. | -| `sanitize-api-keys` | `PostToolUse` | Xóa các khóa OpenAI, Anthropic, GitHub, AWS, Stripe và Google thông thường. | -| `sanitize-connection-strings` | `PostToolUse` | Xóa chuỗi kết nối cơ sở dữ liệu chứa thông tin xác thực. | -| `sanitize-private-key-content` | `PostToolUse` | Xóa các phần nội dung khóa riêng tư PEM. | -| `sanitize-bearer-tokens` | `PostToolUse` | Xóa các mã thông báo xác thực bearer. | -| `protect-env-vars` | `PreToolUse` trên các công cụ shell | Chặn các lệnh xả các biến môi trường. | -| `block-env-files` | `PreToolUse` | Chặn việc đọc và ghi các tệp `.env`. | -| `block-read-outside-cwd` | `PreToolUse` trên các công cụ đọc, glob, grep, hoặc shell | Giữ các lần đọc bên trong thư mục làm việc của phiên. | -| `block-secrets-write` | `PreToolUse` trên các công cụ ghi | Chặn ghi vào các tên tệp khóa bí mật và thông tin xác thực thông thường. | - -## Các lệnh nguy hiểm và cơ sở hạ tầng - -| Chính sách | Kích hoạt | Kết quả | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`, `PermissionRequest` | Chặn `sudo` trừ khi một mẫu allow khớp. | -| `block-curl-pipe-sh` | `PreToolUse` | Chặn các tập lệnh đã tải xuống được piped trực tiếp đến shell. | -| `block-rm-rf` | `PreToolUse` | Chặn các mẫu xóa đệ quy thảm họa. | -| `block-failproofai-commands` | `PreToolUse`, `PermissionRequest` | **Luôn bật, không thể tắt.** Chặn mọi lệnh gọi CLI Failproof AI, tự tạm dừng, và gỡ cài đặt trình quản lý gói. | -| `block-kubectl` | `PreToolUse` | Cổng các lệnh Kubernetes. | -| `block-terraform` | `PreToolUse` | Cổng các lệnh Terraform và OpenTofu. | -| `block-aws-cli` | `PreToolUse` | Cổng các lệnh AWS CLI. | -| `block-gcloud` | `PreToolUse` | Cổng các lệnh Google Cloud CLI. | -| `block-az-cli` | `PreToolUse` | Cổng các lệnh Azure CLI. | -| `block-helm` | `PreToolUse` | Cổng các lệnh Helm. | -| `block-gh-pipeline` | `PreToolUse` | Cổng các hoạt động luồng công việc, chạy, hợp nhất, phát hành, bộ nhớ cache và bí mật thay đổi của GitHub CLI. | - -## An toàn Git và cơ sở dữ liệu - -| Chính sách | Kích hoạt | Kết quả | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | Chặn việc push trực tiếp đến các nhánh được bảo vệ đã cấu hình. | -| `block-force-push` | `PreToolUse` | Chặn force-pushes; `--force-with-lease` vẫn được phép bởi triển khai hiện tại. | -| `block-work-on-main` | `PreToolUse` | Chặn commits và merges trên các nhánh được bảo vệ. | -| `warn-git-amend` | `PreToolUse` | Cảnh báo trước khi viết lại một commit với `--amend`. | -| `warn-git-stash-drop` | `PreToolUse` | Cảnh báo trước khi vĩnh viễn bỏ hoặc xóa các stashes. | -| `warn-all-files-staged` | `PreToolUse` | Cảnh báo trên rộng `git add -A`, `git add .`, hoặc `git add --all`. | -| `warn-destructive-sql` | `PreToolUse` | Cảnh báo trên `DROP`, `TRUNCATE`, và `DELETE` không có `WHERE` thông qua các máy khách cơ sở dữ liệu được công nhận. | -| `warn-schema-alteration` | `PreToolUse` | Cảnh báo trên các hoạt động cột `ALTER TABLE` và đổi tên được công nhận. | - -## Gói, hành vi hệ thống, và vòng lặp agent - -| Chính sách | Kích hoạt | Kết quả | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | Cảnh báo trước khi xuất bản sang các kho lưu trữ gói. | -| `warn-global-package-install` | `PreToolUse` | Cảnh báo trước khi cài đặt gói toàn cầu. | -| `prefer-package-manager` | `PreToolUse` | Hướng dẫn agent sử dụng một trình quản lý gói được phép. | -| `warn-large-file-write` | `PreToolUse` trên các công cụ ghi | Cảnh báo vượt quá ngưỡng kích thước tệp đã cấu hình. | -| `warn-background-process` | `PreToolUse` | Cảnh báo trên các mẫu quy trình nền tách rời hoặc có thời gian sống lâu dài. | -| `warn-repeated-tool-calls` | `PreToolUse` | Cảnh báo sau ba hoặc nhiều lệnh gọi công cụ giống hệt nhau. | - -## Luồng công việc kết thúc tác vụ - -Những chính sách này yêu cầu một harness phát ra một sự kiện `Stop` tương thích. - -| Chính sách | Kết quả | -| --- | --- | -| `require-commit-before-stop` | Từ chối hoàn thành trong khi công việc được theo dõi vẫn chưa được commit. | -| `require-push-before-stop` | Từ chối hoàn thành trong khi các commits vẫn chỉ ở local. | -| `require-pr-before-stop` | Yêu cầu một pull request cho nhánh hiện tại. | -| `require-no-conflicts-before-stop` | Yêu cầu một hợp nhất sạch lại nhánh cơ sở đã cấu hình. | -| `require-ci-green-before-stop` | Yêu cầu các kiểm tra CI HEAD hiện tại hoàn thành thành công. | - -## Tham khảo tham số - -Cấu hình các tham số dưới đối tượng `policyParams` của phạm vi đã chọn. Các loại được xác thực bởi mỗi chính sách. - -| Chính sách | Tham số | Loại và mặc định | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`, `[]`; các mục nhập chứa `regex` và `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`, `[]` | -| `block-sudo` | `allowPatterns` | `string[]`, `[]` | -| `block-rm-rf` | `allowPaths` | `string[]`, `[]` | -| Các bộ chặn cơ sở hạ tầng | `allowPatterns` | `string[]`, `[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`, `[]` | -| `block-push-master` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`, `["main", "master"]` | -| `prefer-package-manager` | `allowed`, `blocked` | `string[]`, `[]` | -| `warn-large-file-write` | `thresholdKb` | `number`, `1024` | -| `require-push-before-stop` | `remote`, `baseBranch` | `string`, `"origin"`; `string`, `"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`, `"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`, `"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - Một mẫu allow mở rộng những gì một agent có thể làm. Kiểm tra tokenization chính xác và các biến thể lệnh trên harness đích trước khi triển khai nó trên toàn bộ đội. - \ No newline at end of file diff --git a/docs/vi/policies/builtin.mdx b/docs/vi/policies/builtin.mdx deleted file mode 100644 index afc039347..000000000 --- a/docs/vi/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "Các chính sách tích hợp sẵn" -description: "Kích hoạt các biện pháp bảo vệ được duy trì cho các chế độ lỗi agent thường gặp." -icon: "library" ---- - -Các chính sách tích hợp sẵn bao gồm xử lý bí mật, tệp môi trường, lệnh shell phá hủy, nhánh được bảo vệ, công cụ đám mây và cơ sở hạ tầng, xuất bản gói, các cuộc gọi lặp lại, và kiểm tra quy trình làm việc khi kết thúc nhiệm vụ. - -## Bật và xác minh một chính sách tích hợp sẵn - - - - 1. Cài đặt chính sách trên một máy được kết nối với CLI cục bộ. - 2. Chạy một hành động kiểm tra an toàn trong agent được đo lường. - 3. Đi tới **Observe → policy** và lọc theo tên chính sách, môi trường máy, hoặc quyết định. - 4. Mở phiên được liên kết để xác nhận đầu vào công cụ khớp và lý do được trả về. - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - Xóa nó bằng `failproofai policy remove block-rm-rf --cli claude --scope project`. - - - -Liệt kê các chính sách có sẵn trong phiên bản đã cài đặt của bạn: - -```bash -failproofai policies -``` - -Bật một chính sách cho một dự án: - -```bash -failproofai policy add block-rm-rf --scope project -``` - -Bật nhiều chính sách cho các harness được chọn: - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -Một số chính sách chấp nhận các tham số hoặc được đánh dấu là phiên bản beta. Xem lại mô tả, phạm vi khớp, và hành vi mặc định trước khi triển khai. Một chính sách bảo vệ một quy trình làm việc có thể chặn các hoạt động hợp lệ trong quy trình khác. - - - Xem lại tất cả 40 chính sách hiện tại, các trình kích hoạt của chúng, giá trị cơ bản được đề xuất, và các tham số. - - - - Ưu tiên phạm vi dự án cho các kỳ vọng cụ thể của kho lưu trữ và phạm vi người dùng cho các yêu cầu bảo mật toàn máy. - \ No newline at end of file diff --git a/docs/vi/policies/custom.mdx b/docs/vi/policies/custom.mdx deleted file mode 100644 index 89779ff56..000000000 --- a/docs/vi/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Custom policies" -description: "Viết một policy cho một chế độ lỗi độc đáo đối với quy trình làm việc của agent của bạn." -icon: "shield-plus" ---- - -Tạo một file kết thúc bằng `policies.js`, `policies.mjs`, hoặc `policies.ts` trong `.failproofai/policies/`. Các file convention sẽ tải tự động ở phạm vi project và user. - -## Kiểm tra policy trước khi xuất bản lên Cloud - - - - 1. Cài đặt custom policy trên một máy test và kích hoạt cả một hành động phù hợp và một hành động hợp lệ không phù hợp. - 2. Đi tới **Observe → policy** và so sánh hai quyết định. - 3. Mở từng phiên được liên kết và xác minh rằng payload sự kiện chứa đủ bằng chứng cho quy tắc. - 4. Khi hành vi là chính xác, di chuyển mã nguồn được xem xét vào **Admin → policy editor** và xuất bản một phiên bản. - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - Các file convention trong `.failproofai/policies/` sẽ tải mà không cần `--custom`. Giữ một lệnh cài đặt rõ ràng trong CI khi xác thực nên thất bại trên một module bị hỏng. - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -Cái này khớp với `production/config.yml`, `/srv/production/config.yml`, `/srv/production`, và `C:\\production\\config.yml` cho cả `Write` và `Edit`. Nó không khớp với các tên như `production-backup` vì `production` phải là một phần đường dẫn hoàn chỉnh. - -Xác thực và cài đặt một file rõ ràng: - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -Policy context bao gồm loại sự kiện, payload bình thường hóa, tên công cụ và input, metadata phiên, tham số, và CLI nguồn khi có sẵn. - -## Kiểm tra các đường dẫn lỗi - -Chạy xác thực sau khi thay đổi file entry hoặc bất kỳ module cục bộ nào mà nó import: - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -Đường dẫn CLI strict thất bại cho các file bị thiếu, lỗi cú pháp, import không được phân giải, ngoại lệ top-level, và timeout tải module. Tại thời điểm thực thi, một file custom bị hỏng được ghi log và bỏ qua để các policy dựng sẵn có thể tiếp tục. Coi bất kỳ cảnh báo tải nào là mất mát của việc thực thi dự kiến và cảnh báo về nó trong log sản xuất. - -Sử dụng các tên duy nhất trên toàn cầu trên các policy rõ ràng, convention, và được quản lý bởi Cloud. Giữ các hàm policy xác định, liên kết các cuộc gọi bên ngoài với timeout ngắn, và trả về một `allow`, `instruct`, hoặc `deny` có chủ ý trên mọi đường dẫn. - - - Một custom policy là code thực thi. Kiểm tra các field bị thiếu, các tên công cụ thay thế, và input không đúp định dạng—không chỉ khớp dự kiến. - \ No newline at end of file diff --git a/docs/vi/policies/failure-behavior.mdx b/docs/vi/policies/failure-behavior.mdx index 8c75d7c90..3a64aa423 100644 --- a/docs/vi/policies/failure-behavior.mdx +++ b/docs/vi/policies/failure-behavior.mdx @@ -1,5 +1,4 @@ --- ---- title: "Hành vi khi gặp sự cố" description: "Hiểu điều gì xảy ra khi đánh giá chính sách hoặc daemon cục bộ không khả dụng." icon: "shield-alert" diff --git a/docs/vi/policies/fleet.mdx b/docs/vi/policies/fleet.mdx deleted file mode 100644 index c7e0d142a..000000000 --- a/docs/vi/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Triển khai chính sách tới các máy" -description: "Biết những máy nào đã đăng ký, hiện tại và thực thi các phiên bản chính sách dự định." -icon: "network" ---- - -Phạm vi bao phủ của nhóm máy trả lời câu hỏi liệu một chính sách có tồn tại ở nơi rủi ro tồn tại hay không. Theo dõi các máy bằng ID ổn định và nhãn dễ đọc, sau đó so sánh trạng thái triển khai được gán và được báo cáo của chúng. - -## Kiểm tra phạm vi bao phủ - - - - 1. Đi tới **Admin → enforcement** và xem xét tổng số thực thi và quan sát. - 2. Tìm kiếm một máy theo ID hoặc nhãn, hoặc lọc các máy thiếu chính sách. - 3. Mở rộng một hàng để so sánh các chính sách được gán, triển khai được báo cáo, lần kiểm tra cuối cùng và lịch sử. - 4. Làm mới sau khoảng thời gian bỏ phiếu của máy khi triển khai được áp dụng vẫn đang chờ xử lý. - - ![Bảng điều khiển Fleet Enforcement hiển thị phạm vi bao phủ chính sách, trạng thái triển khai máy và các bài tập quan sát và thực thi.](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - Sử dụng `fp events --agent-id --since 24h` để xác nhận hoạt động tác nhân của máy đạt tới Cloud. - - - -Sử dụng các chế độ xem phạm vi bao phủ để tìm: - -- Các máy chưa bao giờ lấy triển khai mới nhất -- Các máy đã đăng ký không còn báo cáo hoạt động -- Một chính sách được gán cho môi trường hoặc cohort sai -- Phiên bản không khớp sau khi cập nhật bị gián đoạn - -Đổi tên một máy mà không cần kết nối lại: - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -Kiểm tra trạng thái cục bộ: - -```bash -failproofai config --status -``` - - - Sử dụng các nhãn xác định khối lượng công việc và môi trường. Chỉ tên máy chủ thôi thường là không đủ sau khi tự động mở rộng hoặc thay thế máy. - \ No newline at end of file diff --git a/docs/vi/sessions/evaluations.mdx b/docs/vi/sessions/evaluations.mdx index 731123b9b..821a1cd15 100644 --- a/docs/vi/sessions/evaluations.mdx +++ b/docs/vi/sessions/evaluations.mdx @@ -1,5 +1,4 @@ --- ---- title: "Đánh giá trực tuyến" description: "Chấm điểm các phiên làm việc trực tiếp và đã hoàn thành để đánh giá chất lượng, tuân thủ, chi phí và độ trễ." icon: "gauge" diff --git a/docs/vi/start/integrations.mdx b/docs/vi/start/integrations.mdx index 134d131a9..bdc9ce35c 100644 --- a/docs/vi/start/integrations.mdx +++ b/docs/vi/start/integrations.mdx @@ -1,5 +1,4 @@ --- ---- title: "Thiết bị cho agent của bạn" sidebarTitle: "Frameworks" description: "Kết nối bất kỳ framework agent được hỗ trợ nào với Failproof AI chỉ bằng một lệnh gọi." diff --git a/docs/vi/start/setup.mdx b/docs/vi/start/setup.mdx index 4223bb0e6..76290fdf3 100644 --- a/docs/vi/start/setup.mdx +++ b/docs/vi/start/setup.mdx @@ -1,5 +1,4 @@ --- ---- title: "Chọn cách thiết lập của bạn" description: "Chọn thực thi cục bộ, Failproof AI Cloud, hoặc triển khai doanh nghiệp." icon: "waypoints" diff --git a/docs/zh/policies/builtin-catalog.mdx b/docs/zh/policies/builtin-catalog.mdx deleted file mode 100644 index e7118f49c..000000000 --- a/docs/zh/policies/builtin-catalog.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "内置策略目录" -description: "查阅每条 Failproof AI 内置策略的触发条件、推荐状态及可配置参数。" -icon: "list-checks" ---- - -已安装的软件包是策略可用性的权威来源。每次升级后请运行 `failproofai policies`，因为目录条目和行为可能随包版本发生变化。 - -## 推荐基准配置 - -引导式安装的推荐选项当前会启用密钥脱敏器、环境保护、自我保护、灾难性命令防护以及受保护分支安全策略： - -```text -sanitize-jwt sanitize-api-keys -sanitize-connection-strings sanitize-private-key-content -sanitize-bearer-tokens protect-env-vars -block-env-files block-secrets-write -block-failproofai-commands block-sudo -block-curl-pipe-sh block-rm-rf -block-push-master block-force-push -``` - -`block-failproofai-commands` **始终开启**。上方列出该策略仅为完整性说明，无论其是否出现在你的启用列表中，它都会在每次评估时注册，且无法被禁用或暂停——一个能被代理关闭的执行开关，本身就不构成真正的防护。 - -推荐配置有意比**全量**配置范围更窄。基础设施和工作流策略可能会中断正常工作，应仅在需要的仓库和机器上启用。 - -## 密钥与环境 - -| 策略 | 触发条件 | 结果 | -| --- | --- | --- | -| `sanitize-jwt` | `PostToolUse` | 在模型读取工具输出之前，对其中的 JWT 进行脱敏处理。 | -| `sanitize-api-keys` | `PostToolUse` | 脱敏常见的 OpenAI、Anthropic、GitHub、AWS、Stripe 和 Google 密钥。 | -| `sanitize-connection-strings` | `PostToolUse` | 脱敏包含凭证的数据库连接字符串。 | -| `sanitize-private-key-content` | `PostToolUse` | 脱敏 PEM 格式私钥内容。 | -| `sanitize-bearer-tokens` | `PostToolUse` | 脱敏授权 Bearer Token。 | -| `protect-env-vars` | Shell 工具上的 `PreToolUse` | 阻止转储环境变量的命令。 | -| `block-env-files` | `PreToolUse` | 阻止读取和写入 `.env` 文件。 | -| `block-read-outside-cwd` | 读取、glob、grep 或 Shell 工具上的 `PreToolUse` | 将读取操作限制在会话工作目录内。 | -| `block-secrets-write` | 写入工具上的 `PreToolUse` | 阻止向常见密钥及凭证文件名写入内容。 | - -## 危险命令与基础设施 - -| 策略 | 触发条件 | 结果 | -| --- | --- | --- | -| `block-sudo` | `PreToolUse`、`PermissionRequest` | 阻止 `sudo`，除非匹配到允许模式。 | -| `block-curl-pipe-sh` | `PreToolUse` | 阻止将下载的脚本直接通过管道传递给 Shell 执行。 | -| `block-rm-rf` | `PreToolUse` | 阻止灾难性递归删除操作。 | -| `block-failproofai-commands` | `PreToolUse`、`PermissionRequest` | **始终开启，无法禁用。** 阻止所有 Failproof AI CLI 调用、自我暂停及包管理器卸载操作。 | -| `block-kubectl` | `PreToolUse` | 管控 Kubernetes 命令。 | -| `block-terraform` | `PreToolUse` | 管控 Terraform 和 OpenTofu 命令。 | -| `block-aws-cli` | `PreToolUse` | 管控 AWS CLI 命令。 | -| `block-gcloud` | `PreToolUse` | 管控 Google Cloud CLI 命令。 | -| `block-az-cli` | `PreToolUse` | 管控 Azure CLI 命令。 | -| `block-helm` | `PreToolUse` | 管控 Helm 命令。 | -| `block-gh-pipeline` | `PreToolUse` | 管控具有变更性质的 GitHub CLI 工作流、运行、合并、发布、缓存及密钥操作。 | - -## Git 与数据库安全 - -| 策略 | 触发条件 | 结果 | -| --- | --- | --- | -| `block-push-master` | `PreToolUse` | 阻止直接推送到已配置的受保护分支。 | -| `block-force-push` | `PreToolUse` | 阻止强制推送；当前实现中 `--force-with-lease` 仍被允许。 | -| `block-work-on-main` | `PreToolUse` | 阻止在受保护分支上提交和合并。 | -| `warn-git-amend` | `PreToolUse` | 在使用 `--amend` 重写提交前发出警告。 | -| `warn-git-stash-drop` | `PreToolUse` | 在永久丢弃或清除暂存区前发出警告。 | -| `warn-all-files-staged` | `PreToolUse` | 对 `git add -A`、`git add .` 或 `git add --all` 等大范围暂存操作发出警告。 | -| `warn-destructive-sql` | `PreToolUse` | 对通过已识别数据库客户端执行的不带 `WHERE` 的 `DROP`、`TRUNCATE` 和 `DELETE` 操作发出警告。 | -| `warn-schema-alteration` | `PreToolUse` | 对已识别的 `ALTER TABLE` 列修改和重命名操作发出警告。 | - -## 包管理、系统行为与代理循环 - -| 策略 | 触发条件 | 结果 | -| --- | --- | --- | -| `warn-package-publish` | `PreToolUse` | 在向包注册表发布前发出警告。 | -| `warn-global-package-install` | `PreToolUse` | 在全局安装包之前发出警告。 | -| `prefer-package-manager` | `PreToolUse` | 指示代理使用允许的包管理器。 | -| `warn-large-file-write` | 写入工具上的 `PreToolUse` | 当文件大小超过配置阈值时发出警告。 | -| `warn-background-process` | `PreToolUse` | 对后台分离或长期运行的进程模式发出警告。 | -| `warn-repeated-tool-calls` | `PreToolUse` | 在出现三次或以上相同工具调用后发出警告。 | - -## 任务结束工作流 - -这些策略需要一个能发出兼容 `Stop` 事件的执行环境。 - -| 策略 | 结果 | -| --- | --- | -| `require-commit-before-stop` | 在有已跟踪的未提交工作时，拒绝完成任务。 | -| `require-push-before-stop` | 在有仅存于本地的提交时，拒绝完成任务。 | -| `require-pr-before-stop` | 要求为当前分支创建 Pull Request。 | -| `require-no-conflicts-before-stop` | 要求与配置的基础分支干净合并，无冲突。 | -| `require-ci-green-before-stop` | 要求当前 HEAD 的 CI 检查全部通过。 | - -## 参数参考 - -在对应作用域的 `policyParams` 对象下配置参数。各策略会对参数类型进行验证。 - -| 策略 | 参数 | 类型与默认值 | -| --- | --- | --- | -| `sanitize-api-keys` | `additionalPatterns` | `pattern[]`，`[]`；条目包含 `regex` 和 `label` | -| `block-read-outside-cwd` | `allowPaths` | `string[]`，`[]` | -| `block-sudo` | `allowPatterns` | `string[]`，`[]` | -| `block-rm-rf` | `allowPaths` | `string[]`，`[]` | -| 基础设施拦截器 | `allowPatterns` | `string[]`，`[]` | -| `block-secrets-write` | `additionalPatterns` | `string[]`，`[]` | -| `block-push-master` | `protectedBranches` | `string[]`，`["main", "master"]` | -| `block-work-on-main` | `protectedBranches` | `string[]`，`["main", "master"]` | -| `prefer-package-manager` | `allowed`、`blocked` | `string[]`，`[]` | -| `warn-large-file-write` | `thresholdKb` | `number`，`1024` | -| `require-push-before-stop` | `remote`、`baseBranch` | `string`，`"origin"`；`string`，`"main"` | -| `require-pr-before-stop` | `baseBranch` | `string`，`"main"` | -| `require-no-conflicts-before-stop` | `baseBranch` | `string`，`"main"` | - -```json -{ - "enabledPolicies": ["block-sudo", "block-push-master"], - "policyParams": { - "block-sudo": { - "allowPatterns": ["sudo systemctl status"] - }, - "block-push-master": { - "protectedBranches": ["main", "release"] - } - } -} -``` - - - 允许模式会扩大代理可执行的操作范围。在将其部署到整个集群之前，请先在目标执行环境中测试确切的令牌化规则和命令变体。 - \ No newline at end of file diff --git a/docs/zh/policies/builtin.mdx b/docs/zh/policies/builtin.mdx deleted file mode 100644 index 89b618529..000000000 --- a/docs/zh/policies/builtin.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: "内置策略" -description: "为常见的 Agent 故障模式启用可维护的安全防护。" -icon: "library" ---- - -内置策略涵盖密钥处理、环境文件、破坏性 Shell 命令、受保护分支、云和基础设施工具、软件包发布、重复调用以及任务结束工作流检查。 - -## 启用并验证内置策略 - - - - 1. 通过本地 CLI 在已连接的机器上安装策略。 - 2. 在已插桩的 Agent 中运行一个安全的测试操作。 - 3. 前往 **Observe → policy**，按策略名称、机器环境或决策进行筛选。 - 4. 打开关联的会话，确认匹配的工具输入和返回的原因。 - - - - ```bash - failproofai policies - failproofai policy add block-rm-rf --cli claude --scope project - failproofai config --status - ``` - - 通过 `failproofai policy remove block-rm-rf --cli claude --scope project` 将其移除。 - - - -列出当前已安装版本中可用的策略： - -```bash -failproofai policies -``` - -为某个项目启用一条策略： - -```bash -failproofai policy add block-rm-rf --scope project -``` - -为指定的 Harness 启用多条策略： - -```bash -failproofai policies --install block-sudo block-force-push \ - --cli claude codex --scope project -``` - -部分策略支持参数配置或标记为 Beta 版本。在正式部署前，请仔细阅读其描述、匹配范围和默认行为。保护某个工作流的策略，在另一个工作流中可能会阻止合法操作。 - - - 查看全部 40 条现有策略、其触发条件、推荐基准配置及可用参数。 - - - - 对于仓库特定的需求，建议使用项目范围；对于全机器范围的安全要求，建议使用用户范围。 - \ No newline at end of file diff --git a/docs/zh/policies/custom.mdx b/docs/zh/policies/custom.mdx deleted file mode 100644 index c00346ca9..000000000 --- a/docs/zh/policies/custom.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "自定义策略" -description: "为您的智能体工作流中特有的故障模式编写策略。" -icon: "shield-plus" ---- - -在 `.failproofai/policies/` 目录下创建以 `policies.js`、`policies.mjs` 或 `policies.ts` 结尾的文件。约定文件会在项目和用户作用域下自动加载。 - -## 发布到云端之前测试策略 - - - - 1. 在一台测试机器上安装自定义策略，并分别触发一个匹配的操作和一个合法的非匹配操作。 - 2. 前往 **Observe → policy**，对比两次决策结果。 - 3. 打开各自关联的会话，验证事件载荷中包含足够支撑规则判断的证据。 - 4. 确认行为正确后，将已审查的源代码移入 **Admin → policy editor** 并发布一个版本。 - - - - ```bash - failproofai policies --install --custom ./security.policies.ts \ - --cli claude --scope project - failproofai policies - ``` - - `.failproofai/policies/` 下的约定文件无需 `--custom` 即可加载。当模块损坏时需要让验证失败，请在 CI 中保留显式的安装命令。 - - - -```ts -import { customPolicies, allow, deny } from "failproofai"; - -customPolicies.add({ - name: "protect-production-paths", - description: "Block writes to production configuration", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); - const path = String(ctx.toolInput?.file_path ?? "").replaceAll("\\", "/"); - if (path.split("/").includes("production")) { - return deny("Writes to production configuration require approval."); - } - return allow(); - }, -}); -``` - -该策略会匹配 `production/config.yml`、`/srv/production/config.yml`、`/srv/production` 以及 `C:\\production\\config.yml`，对 `Write` 和 `Edit` 均生效。它不会匹配 `production-backup` 之类的名称，因为 `production` 必须是完整的路径段。 - -验证并安装指定文件： - -```bash -failproofai policies --install --custom ./security.policies.ts -``` - -策略上下文包含事件类型、规范化载荷、工具名称和输入、会话元数据、参数，以及可用时的来源 CLI 信息。 - -## 测试失败路径 - -修改入口文件或其导入的任何本地模块后，请运行验证： - -```bash -failproofai policies --install --custom ./security.policies.ts --scope project -``` - -严格的 CLI 路径在以下情况下会失败：文件缺失、语法错误、未解析的导入、顶层异常以及模块加载超时。在策略执行时，损坏的自定义文件会被记录日志并跳过，以便内置策略可以继续运行。请将任何加载警告视为预期执行能力的丧失，并在生产日志中为其设置告警。 - -在显式策略、约定策略和云端托管策略之间使用全局唯一的名称。保持策略函数的确定性，为外部调用设置较短的超时时间，并在每条执行路径上都返回明确的 allow、instruct 或 deny。 - - - 自定义策略是强制执行代码。请测试缺失字段、不同工具名称和格式错误的输入——而不仅仅是预期的匹配情况。 - \ No newline at end of file diff --git a/docs/zh/policies/fleet.mdx b/docs/zh/policies/fleet.mdx deleted file mode 100644 index fa0417002..000000000 --- a/docs/zh/policies/fleet.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "将策略部署到机器" -description: "了解哪些机器已注册、处于最新状态，并正在执行预期的策略版本。" -icon: "network" ---- - -机队覆盖率回答的是：风险所在之处是否存在对应的策略。通过稳定 ID 和易读标签追踪机器，然后比较其已分配与已上报的部署状态。 - -## 检查覆盖率 - - - - 1. 前往 **Admin → enforcement**，查看"执行中"和"观察中"的总数。 - 2. 按 ID 或标签搜索机器，或筛选缺少策略的机器。 - 3. 展开某一行，比较已分配策略、已上报部署情况、最近签入时间及历史记录。 - 4. 当已应用的部署仍处于待定状态时，等待该机器的轮询间隔后刷新。 - - ![显示策略覆盖率、机器部署状态以及观察和执行分配情况的 Enforcement 机队视图。](/images/dashboard/enforcement-fleet.png) - - - ```bash - failproofai config --status - failproofai config --machine-label checkout-runner-03 - failproofai flush --wait - ``` - - 使用 `fp events --agent-id --since 24h` 确认该机器的代理活动已到达 Cloud。 - - - -使用覆盖率视图可以发现： - -- 从未拉取最新部署的机器 -- 已注册但停止上报活动的机器 -- 分配到错误环境或批次的策略 -- 更新中断后产生的版本漂移 - -在不重新连接的情况下重命名机器： - -```bash -failproofai config --machine-label checkout-runner-03 -``` - -检查本地状态： - -```bash -failproofai config --status -``` - - - 使用能够标识工作负载和环境的标签。在自动扩缩容或机器替换后，仅凭主机名往往是不够的。 - \ No newline at end of file diff --git a/scripts/translate-docs/validate-translation.ts b/scripts/translate-docs/validate-translation.ts index 632343028..56be21b53 100644 --- a/scripts/translate-docs/validate-translation.ts +++ b/scripts/translate-docs/validate-translation.ts @@ -11,6 +11,11 @@ * - Frontmatter YAML (class A): the model re-emits an inner `"` unescaped into * a double-quoted `title:`/`description:` value, breaking the YAML. This is * the class that failed run 29575781632; `findFrontmatterError` catches it. + * - Empty block (class A2): the model emits a second opening `---`, so the + * block Mintlify reads closes before any key is inside it and the whole + * frontmatter renders as body text. YAML calls a leading `---` a + * document-start marker and parses it clean, so class A is blind to this; + * `mintlifyFrontmatterBlock` is the second view that is not. * - Key parity (class B): the model drops the frontmatter block entirely, or * renames a key. A dropped block is still *valid YAML* (mintlify tolerates * it, deriving the title from the slug), so only comparing against the @@ -52,6 +57,34 @@ function frontmatterKeys(page: string): string[] | null { } } +/** + * The frontmatter block as MINTLIFY reads it: opened by a first line that is + * exactly `---`, closed by the FIRST line after it that is `---` — trailing + * spaces or tabs allowed, as `FRONTMATTER_RE` allows them, but never leading + * ones: an indented `---` is content, not a delimiter. `null` when the page + * does not open with a delimiter at all. + * + * Deliberately NOT reusing `FRONTMATTER_RE`. That matcher exists to feed + * `YAML.parse`, and YAML reads a bare `---` as a document-START marker rather + * than as a terminator — so a page beginning `---\n---\ntitle: …` parses there + * as a perfectly clean `{title: …}` while Mintlify closes the block on line 2 + * and renders every key as body text. The reader gets a literal `title: "…"` at + * the top of the page and no title on it. + * + * That is not hypothetical: 102 published pages across seven locales shipped + * exactly that way, invisible to every other check in this file, because both + * of them asked YAML. The two views have to be compared, never shared. + */ +function mintlifyFrontmatterBlock(page: string): string | null { + const lines = page.split(/\r?\n/); + if (lines[0] !== "---") return null; + const close = lines.findIndex( + (line, i) => i > 0 && /^---[ \t]*$/.test(line), + ); + if (close < 0) return null; + return lines.slice(1, close).join("\n"); +} + /** * A ±2-line window around `line` (1-based), the failing line prefixed `> ` and * its neighbours ` `. Empty string when `line` is undefined. @@ -93,6 +126,21 @@ export async function findTranslationError( ); } + // Class A2: a block that is EMPTY as Mintlify delimits it. A stray opening + // delimiter (`---\n---\ntitle: …`) is the shape that does this, and it is + // invisible to everything above — see `mintlifyFrontmatterBlock`. Checked for + // every source shape, like the YAML above: the model adding this to a + // frontmatter-less page breaks that page just as thoroughly. + const mintlifyBlock = mintlifyFrontmatterBlock(rendered); + if (mintlifyBlock !== null && mintlifyBlock.trim() === "") { + return ( + "The YAML frontmatter block is empty. The page opens with `---` and the " + + "very next line is `---` again, which closes the block before any key is " + + "in it — every `title:`/`description:`/`icon:` line below then renders as " + + "body text and the page has no title. Emit exactly one opening `---`." + ); + } + const sourceKeys = frontmatterKeys(source); if (sourceKeys) { // The source has frontmatter, so the translation must carry the same keys.