ci: pin semgrep registry rules as snapshots with scheduled update PRs

[feliperalmeida]

Problem

Our semgrep workflows pass live registry configs (--config p/owasp-top-ten, etc.), which resolve at scan time. When the registry adds or modifies a rule, CI starts failing org-wide on unrelated PRs with no code change on our side — worst in PostHog/posthog, where engineers get blocked. The --exclude-rule lists in our workflows are the accumulated manual responses to this. Registry changes also break CI a second way: rules that fail to parse/analyze under --strict (see the comment in semgrep.yml), which no finding-level mitigation can catch.

Approach: pin the rules, update deliberately

• .semgrep/registry/ now vendors the resolved YAML of every registry pack used org-wide (~1,270 rules, 3 MB), fetched anonymously from semgrep.dev/c/<id> — exactly what CI fetched before, but frozen. sources.json maps snapshot files to registry ids.
• Scan workflows (semgrep.yml, semgrep-package-managers.yml, ci-security.yml) point at the snapshots instead of p/.... Like flags-boards.json, snapshots load from main at runtime, so callers never re-pin.
• semgrep-registry-update.yml (weekday cron) re-fetches the packs, diffs per rule id, dry-runs only the added/changed rules against PostHog/posthog to measure blast radius (finding counts + analysis errors, never failing), opens/updates a single rolling snapshot-bump PR with the full report, and notifies Slack. Merging that PR is the moment new rules start being enforced — occurrences can be fixed (or exclusions added) first.
• .github/scripts/semgrep_registry.py implements sync/diff/report; unit-tested via the existing semgrep-tests.yml discover step.

A registry-side change can no longer break CI on unrelated PRs — including the parse-error failure mode, since a rule that errors under --strict fails the snapshot PR's own CI before it can reach main.

Verification

• Parity scan: live p/ packs vs. snapshots on this repo produce identical findings (the snapshot fetch is anonymous, same as CI's).
• sync is idempotent (re-run produces zero diffs); all snapshots pass semgrep --validate.
• 10 unit tests pass, including inside the pinned semgrep container image; semgrep --test .semgrep/rules/ 4/4.
• Both workflow-equivalent scans run clean over the repo including the new workflow (custom shell-injection rules included); zizmor findings on the new/touched files fixed (scoped app-token permissions, persist-credentials: false on read-only checkouts).

Setup required before the updater is useful

• SEMGREP_REGISTRY_BOT_APP_ID / SEMGREP_REGISTRY_BOT_PRIVATE_KEY secrets — GitHub App with Contents + Pull requests write on this repo (App token so the bot's PRs trigger CI).
• SEMGREP_REGISTRY_SLACK_BOT_TOKEN secret + SEMGREP_REGISTRY_SLACK_CHANNEL_ID repo variable (the Slack step skips itself until set).

Follow-up

• PostHog/posthog PR switching the eight semgrep jobs in ci-security.yaml to these snapshots (must land after this merges — snapshots are fetched from main at runtime). All packs posthog needs are already vendored here.
• Known behavior: while a snapshot PR sits unmerged, each weekday run force-pushes an equivalent commit and re-notifies Slack — a deliberate nag, easy to guard against if unwanted.
• Note for review: this vendors Semgrep/Trail of Bits registry rules (public-tier only) into a public repo; provenance and licensing are noted in .semgrep/registry/README.md.

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

[Piccirello]

This currently only checks against PostHog/posthog. In the past when we've had failures from new rules they often effected multiple repos. Maybe PostHog/posthog is a good sentinel for all of those? (meaning it's unlikely another repo will fail if PostHog/posthog isn't also failing) Otherwise we may want a list of "critical" repos to include in this scan, which would probably include a bunch of SDK repos.