Skip to content

Add deterministic weekly Socket Firewall review - #8

Merged
peakematt merged 22 commits into
mainfrom
feature/help-724-rollout-verifier-foundation
Sep 16, 2026
Merged

peakematt merged 22 commits into
mainfrom
feature/help-724-rollout-verifier-foundation

Conversation

@peakematt

@peakematt peakematt commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a version-controlled script that an engineer or agent can run locally to review Socket Firewall coverage across WorkOS repositories.

This gives us a repeatable way to find gaps without manually checking every repository or revisiting the same accepted exceptions each week.

The script:

  • Reads CI configuration to identify dependency installs needing attention.
  • Separates new or changed findings from known issues and accepted exceptions.
  • Keeps review decisions in a private state file for use on subsequent runs.

It is read-only: running it does not modify repositories or create a schedule.

Run locally

npm run --silent review:weekly -- --state /path/to/private/review-state.json

Reuse the same state file between runs. The README covers setup, state initialization, and the output format.

Related: HELP-724

Requested by matt.peake@workos.com in TARS

@linear-code

linear-code Bot commented Sep 3, 2026

Copy link
Copy Markdown

HELP-724

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Devin Review

Comment thread package.json Outdated
Comment thread tools/rollout/rollout.test.mjs
@greptile-apps

greptile-apps Bot commented Sep 3, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

The PR appears safe to merge.

Summary

The PR adds a deterministic weekly Socket Firewall audit and durable review ledger while expanding the underlying repository classifier, source resolution, reporting, documentation, and offline verification.

  • Adds stable repository and case fingerprints with explicit record, forget, and replay workflows.
  • Preserves last-good decisions across incomplete scans and serializes owner-only state updates.
  • Updates GitHub API retry handling to recognize both parenthesized and colon-form rate-limit diagnostics.
  • Extends CI with dependency installation and offline verifier tests.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Weekly review command] --> B[Capture REST and GraphQL inventory]
  B --> C[Audit immutable repository snapshots]
  C --> D{Scan complete?}
  D -->|No| E[Fail without advancing ledger]
  D -->|Yes| F[Classify jobs and fingerprint cases]
  F --> G[Apply durable recorded decisions]
  G --> H[needsProtection]
  G --> I[needsReview]
  G --> J[known]
  G --> K[unobserved]
  G --> L[Atomic owner-only ledger and report writes]
Loading

Reviews (7) · Last reviewed commit: "fix(audit): keep accepted Rush cases sta..."

Comment thread tools/rollout/github.mjs Outdated
@peakematt peakematt changed the title Add rollout verifier foundation Add read-only Socket Firewall gap detector Sep 14, 2026
@workos-tars

workos-tars Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Added bounded recognition for literal npm workspace selectors, explicit npm exec -- targets, and literal npx package selectors. Matching boolean output guards are recognized only when the output is stable: a dependency job, or a unique earlier step. Future/duplicate step IDs, mutable environment guards, unsupported selectors, and post-teardown downloads stay visible.

103 targeted classifier/integration regression tests and formatting passed. Strict execution assurance is unchanged; no repository allowlist or runtime protection claim was added.

@workos-tars
workos-tars Bot marked this pull request as ready for review September 15, 2026 20:16
@workos-tars workos-tars Bot changed the title Add read-only Socket Firewall gap detector Add deterministic weekly Socket Firewall review Sep 15, 2026
@peakematt
peakematt merged commit bc8f5ce into main Sep 16, 2026
15 checks passed
@peakematt
peakematt deleted the feature/help-724-rollout-verifier-foundation branch September 16, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants