Skip to content

Add a multi-key sync support#211

Open
Yostra wants to merge 5 commits intov2from
multi_org
Open

Add a multi-key sync support#211
Yostra wants to merge 5 commits intov2from
multi_org

Conversation

@Yostra
Copy link
Copy Markdown
Collaborator

@Yostra Yostra commented Mar 31, 2026

Summary

  • Adds support for syncing multiple Stripe accounts into a single Postgres schema. Each row includes an _account_id column with the real Stripe account ID, and tables use a composite primary key (id, _account_id) so records from different accounts coexist without collisions.
  • Introduces a sync-multi CLI subcommand that accepts a JSON config with multiple pipelines, auto-resolves Stripe account IDs from API keys, runs setup sequentially (to avoid Postgres schema creation races), then syncs all pipelines concurrently.
  • Uses the resolved account ID as the syncId for the state store, so page cursors are isolated per account and don't interfere across pipelines.

How to test (optional)

replace STRIPE_API_KEY_1 and STRIPE_API_KEY_2 with actual keys

node apps/engine/dist/cli/index.js sync-multi --config '{
  "pipelines": [
    {
      "source": { "name": "stripe", "api_key": "[STRIPE_API_KEY_1]" },
      "destination": {
        "name": "postgres",
        "connection_string": "postgresql://postgres:postgres@localhost:55432/postgres?sslmode=disable",
        "schema": "stripe"
      },
      "streams": [{ "name": "customers" }]
    },
    {
      "source": { "name": "stripe", "api_key": "[STRIPE_API_KEY_2]" },
      "destination": {
        "name": "postgres",
        "connection_string": "postgresql://postgres:postgres@localhost:55432/postgres?sslmode=disable",
        "schema": "stripe"
      },
      "streams": [{ "name": "customers" }]
    }
  ]
}'

@Yostra Yostra marked this pull request as ready for review March 31, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant