chore: restructure repo for agent-first discovery by tonyxiao · Pull Request #192 · stripe/sync-engine

tonyxiao · 2026-03-29T20:24:41Z

Summary

Restructures the repo following the "harness engineering" pattern — AGENTS.md as a map, structured knowledge base, mechanical enforcement, progressive disclosure.

Flatten docs/: Moved all content from docs/pages/ to docs/{architecture,engine,service,plans,guides}/. Updated build.mjs to scan from root with exclusion filter.
Architecture knowledge base: Added docs/architecture/{principles,decisions,quality}.md — golden rules, design decision records, per-package scorecard.
AGENTS.md rewrite: From ~77 lines of build commands to a progressive-disclosure map with package table, key rules, and navigation links.
Layer enforcement test: e2e/layers.test.ts mechanically validates source/dest isolation, protocol independence, connector deps, and service isolation (8 tests).
tsconfig inheritance: Extracted tsconfig.base.json with shared compiler options. All 11 package tsconfigs extend it.
Cleanup: Deleted stale .eslintrc.js, TODOs.md. Created docs/plans/backlog.md from actionable items.

Stacked on fix-postgres-tls-semantics (docs infrastructure).

Test plan

pnpm build — passes (pre-existing openapi failure only)
pnpm test — passes
pnpm lint — passes
pnpm format:check — passes
docs build — 34 pages built
e2e/layers.test.ts — 8/8 pass
CLAUDE.md symlink resolves correctly

🤖 Generated with Claude Code

…ing rejectUnauthorized:false The previous commit (841d023) hardcoded ssl: { rejectUnauthorized: false } for all connection strings, silently overriding any sslmode the caller set (e.g. sslmode=verify-full on an RDS instance). Parse the sslmode query parameter and map it to the correct pg ssl option: - disable → ssl: false - verify-ca/full → ssl: true (verify cert against system CA) - require/prefer/unset → ssl: { rejectUnauthorized: false } (TLS on, no cert check — safe for proxied connections) Applied in destination-postgres buildPoolConfig, state-postgres setupStateStore, createStateStore, and runMigrationsWithContent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

Committed-By-Agent: claude

…rate-openapi.sh --config SSL is now opt-in: connections without an explicit sslmode use ssl:false by default. Use sslmode=require for SSL without cert verification, or sslmode=verify-ca / verify-full for full verification. Also fixes generate-openapi.sh to pass --config .prettierrc when writing to a temp dir (--check mode), so prettier uses printWidth:100 from the project config instead of the default 80. This resolves persistent DRIFT detection on CI where the enum array was wrapped differently. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Committed-By-Agent: claude

- Flatten docs/pages/ into docs/{architecture,engine,service,plans,guides} - Update build.mjs to scan from docs/ root with exclusion filter - Rewrite AGENTS.md as a progressive-disclosure map with package table - Add docs/architecture/{principles,decisions,quality}.md - Add e2e/layers.test.ts enforcing source/dest isolation, protocol independence, connector deps, and service isolation - Extract tsconfig.base.json, all packages extend it - Delete stale .eslintrc.js, TODOs.md; add docs/plans/backlog.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Committed-By-Agent: claude

- scripts/audit-repo.sh: local script using Claude Code CLI to audit docs accuracy, architecture violations, dead code, quality gaps, and convention drift. Supports --pr flag to auto-fix and create PR. - .github/workflows/audit.yml: weekly GitHub Action (Monday 9am UTC) that runs the audit. Warns if ANTHROPIC_API_KEY secret is missing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Committed-By-Agent: claude

Copilot

Pull request overview

This PR restructures the repository to be more “agent-first” (AGENTS.md as the navigation hub), adds an architecture knowledge base, and introduces mechanical enforcement for package layering while also deduplicating TypeScript compiler configuration.

Changes:

Added a root tsconfig.base.json and updated package/app tsconfigs to extend it.
Introduced an e2e “layer enforcement” test to validate isolation rules across packages/apps.
Expanded/reshuffled docs into a more structured knowledge base (architecture, engine, service, plans, guides) and updated the docs build script accordingly.

Reviewed changes

Copilot reviewed 31 out of 71 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tsconfig.base.json	New shared TS compiler options used by package/app tsconfigs.
scripts/generate-openapi.sh	Pins Prettier invocation to use repo config.
scripts/audit-repo.sh	Adds a Claude-driven repo audit script (optionally PR-creating).
packages//tsconfig.json, apps//tsconfig.json	Switch to extending tsconfig.base.json.
packages/state-postgres/src/state-store.ts	Changes Postgres SSL handling to derive from connection-string sslmode.
packages/state-postgres/src/migrate.ts	Same SSL semantics change for migrations client.
packages/destination-postgres/src/index.ts	Same SSL semantics change for destination pool config + tests updated.
e2e/layers.test.ts	Adds mechanical enforcement for layering/dependency rules.
docs/**	New/reshuffled documentation structure and new example/code docs assets.
.github/workflows/audit.yml	Adds scheduled workflow to run audit-repo.sh and open PRs.
AGENTS.md	Rewritten to be a repo map with rules and navigation.
docs/build.mjs	Updates docs build to scan from docs/ root with exclusions.
TODOs.md, .eslintrc.js	Removes stale files.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-30T15:40:15Z

docs/architecture/principles.md

+
+## 2. Connector isolation
+
+Sources never import destinations. Destinations never import sources. Both depend only on `@stripe/sync-protocol`. This is enforced by `e2e/layers.test.ts`.


The connector isolation principle says sources/destinations "depend only on @stripe/sync-protocol", but destination-postgres (and likely others) also depend on @stripe/sync-util-postgres per this repo’s package graph. Consider rewording this rule to reflect the actual allowed dependencies (e.g., protocol + approved shared utility packages) so the principle matches what the codebase enforces.

Suggested change

Sources never import destinations. Destinations never import sources. Both depend only on `@stripe/sync-protocol`. This is enforced by `e2e/layers.test.ts`.

Sources never import destinations. Destinations never import sources. Both depend only on `@stripe/sync-protocol` and a small set of approved shared utility packages (for example, `@stripe/sync-util-postgres`). This dependency boundary is enforced by `e2e/layers.test.ts`.

Copilot · 2026-03-30T15:40:16Z

packages/destination-postgres/src/index.ts

+/**
+ * Map the `sslmode` query parameter from a Postgres connection string to a pg
+ * `ssl` option. Defaults to `false` (no SSL) when no sslmode is present — SSL
+ * must be opted into explicitly via `sslmode=require` (or `verify-ca`/`verify-full`).
+ */
+function sslConfigFromConnectionString(connStr: string): PoolConfig['ssl'] {
+  try {
+    const sslmode = new URL(connStr).searchParams.get('sslmode')
+    if (sslmode === 'disable') return false
+    if (sslmode === 'verify-ca' || sslmode === 'verify-full') return true
+    if (sslmode === 'require') return { rejectUnauthorized: false }
+    return false
+  } catch {
+    return false
+  }


sslConfigFromConnectionString silently falls back to ssl: false on parse errors and for any unrecognized/absent sslmode. This changes connection behavior from the previous forced TLS and can also hide invalid connection strings (e.g., non-URL libpq formats) by quietly disabling SSL. Consider (a) validating/throwing on unparseable URLs, and (b) explicitly handling common sslmode values (prefer/allow) or choosing a safer default that better matches Postgres/libpq semantics.

Copilot · 2026-03-30T15:40:16Z

packages/state-postgres/src/state-store.ts

+function sslConfigFromConnectionString(connStr: string): PoolConfig['ssl'] {
+  try {
+    const sslmode = new URL(connStr).searchParams.get('sslmode')
+    if (sslmode === 'disable') return false
+    if (sslmode === 'verify-ca' || sslmode === 'verify-full') return true
+    if (sslmode === 'require') return { rejectUnauthorized: false }
+    return false
+  } catch {
+    return false
+  }
+}


sslConfigFromConnectionString defaults to false (no TLS) for missing/unknown sslmode and on URL parse failures. That can break deployments that rely on libpq defaults (sslmode=prefer) and can also mask invalid connection strings by silently disabling SSL. Consider making parse failures explicit (throw) and aligning the default/unknown handling with expected Postgres sslmode semantics.

Copilot · 2026-03-30T15:40:16Z

packages/state-postgres/src/migrate.ts

+function sslConfigFromConnectionString(connStr: string): ClientConfig['ssl'] {
+  try {
+    const sslmode = new URL(connStr).searchParams.get('sslmode')
+    if (sslmode === 'disable') return false
+    if (sslmode === 'verify-ca' || sslmode === 'verify-full') return true
+    if (sslmode === 'require') return { rejectUnauthorized: false }
+    return false
+  } catch {
+    return false
+  }
+}


sslConfigFromConnectionString silently returns false on parse errors and for unrecognized/absent sslmode, which can change default behavior compared to typical Postgres/libpq semantics and conceal malformed connection strings. Consider failing fast on invalid URLs and handling additional sslmode values (e.g., prefer/allow) or documenting/enforcing the expected default explicitly.

tonyxiao force-pushed the chore/repo-reorg-harness-engineering branch from 23a8c17 to 17f1540 Compare March 29, 2026 20:45

tonyxiao and others added 4 commits March 29, 2026 14:02

chore: regenerate engine OpenAPI spec

7791c98

Committed-By-Agent: claude

tonyxiao force-pushed the chore/repo-reorg-harness-engineering branch from 826e4b7 to 8cda3d4 Compare March 29, 2026 21:39

tonyxiao force-pushed the fix-postgres-tls-semantics branch from b8542de to d1bf372 Compare March 30, 2026 01:03

kdhillon-stripe requested a review from Copilot March 30, 2026 15:33

Copilot started reviewing on behalf of kdhillon-stripe March 30, 2026 15:34 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Base automatically changed from fix-postgres-tls-semantics to v2 March 30, 2026 16:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: restructure repo for agent-first discovery#192

chore: restructure repo for agent-first discovery#192
tonyxiao wants to merge 5 commits intov2from
chore/repo-reorg-harness-engineering

tonyxiao commented Mar 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		## 2. Connector isolation

		Sources never import destinations. Destinations never import sources. Both depend only on `@stripe/sync-protocol`. This is enforced by `e2e/layers.test.ts`.

Conversation

tonyxiao commented Mar 29, 2026

Summary

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants