connector-batch-1: foundation + 81 new SaaS connectors (in progress)#232
Merged
Conversation
Three pieces of infrastructure to keep the adapter catalog maintainable
as it grows from 40 to 100+ entries:
1. scripts/regenerate-catalog.mjs — codegen that scans every JSON under
packages/backend/src/adapters/{de,gb,intl,br,in,jp,ng}/ and rewrites
the AUTOGEN-marked sections of catalog.ts (imports + RAW_ADAPTERS
array). Wired into backend's prebuild script so nest build always
has the catalog in sync with the JSON files. Eliminates the chore of
hand-editing two lists in catalog.ts on every new adapter and the
class of bugs where someone forgets one of the two.
2. scripts/validate-adapters.mjs — quality gate with hard errors (must
pass for CI) on the things that genuinely break the system (missing
required fields, slug-filename mismatch, unknown connector/auth
types, empty tools array) and soft warnings (instructions length,
tool description length, parameter descriptions, slug-prefixed tool
names) that surface low-quality adapters without blocking the
batch. Existing 40 adapters all pass the hard gate.
3. Frontend store: +18 category labels (CRM, Email, Marketing
Automation, Project Management, Scheduling, Forms, Customer Support,
Payments, E-commerce, Analytics, Publishing, E-signature, Lead
Enrichment, Knowledge Base, Social, Maps & Geo, Travel, CMS) so the
incoming connector batch renders with proper labels in the store UI.
CI: added two new steps before backend tests — validate-adapters runs
the gate, then we regenerate catalog.ts and fail the run if it differs
from what's committed (catches the case where someone adds a JSON but
forgets to regenerate). Both steps are fast (<5s).
Three high-priority greenfield SaaS connectors with rich instructions fields, live edge specs (RUN_*_LIVE gated), and full tool coverage. - Pipedrive (CRM): 23 tools across deals, persons, organizations, activities, pipelines, stages, users, custom field discovery and universal search. QUERY_AUTH with api_token in query string. Mixed v1 + v2 API as per Pipedrive's own structure. - Mailchimp (Email/MA): 16 tools across audiences, members, segments, campaigns, templates and account info. BEARER_TOKEN with the datacenter-prefixed base URL pattern (us6 etc) baked into the env var template. - SendGrid (Transactional email): 14 tools across mail send, templates, stats, suppressions (bounces / spam / invalid), marketing contacts, marketing lists, SGQL search. BEARER_TOKEN. All three pass the validator hard gate and their live edge probes return 401 from the vendor router (proves endpoint paths are recognized). Catalog auto-regenerated to 43 adapters total.
- Calendly v2 (Scheduling): 12 tools — event types, availability
windows, scheduled events, invitees, single-use scheduling links,
cancellation (POST not DELETE per Calendly convention),
organization memberships. Personal access token Bearer auth.
Heavy use of URI-as-ID model — instructions field documents it.
- Telegram Bot API: 13 tools — text/photo/document/audio/voice
(OGG-Opus)/video/location/poll send, message edit/delete, chat
info, getUpdates pull. authType=NONE because the bot token lives
in the URL path (Telegram-specific) — the {{TELEGRAM_BOT_TOKEN}}
template substitutes into baseUrl.
- Discord REST API v10 (bot): 13 tools — guilds/channels/messages
list, send/edit/delete message, reactions (PUT), DM channel open,
thread spawn, guild member fetch. API_KEY auth with literal 'Bot '
prefix (NOT 'Bearer') as Discord requires. Inbound gateway,
voice, slash commands explicitly out of scope.
Catalog now at 46 adapters. All pass validator hard gate.
Finishes the verified Top 10 batch (no vendor-official MCP server exists yet for these as of audit). - Zendesk Support v2: 18 tools — tickets CRUD with comment envelope, users, organizations, groups, views, macros, search, side-loading. BASIC_AUTH with email/token convention. Subdomain in baseUrl. - Typeform: 10 tools — list/get forms, list responses (cursor and date-range pagination), delete responses (GDPR), workspaces, insights, themes. Personal Token Bearer. Read-focused since the form-builder write side is rarely worth automating. - Lemon Squeezy v1: 14 tools — stores, products, variants, customers, orders, subscriptions (with cancel-at-period-end DELETE), license keys, discounts, payouts. JSON:API spec with required Accept header set per endpoint, side-loading via include=. Bearer auth. - Klaviyo (revision 2024-10-15): 14 tools — accounts, profiles CRUD, lists, subscribe with double-opt-in respect, segments, events tracking (the heart of Klaviyo flows), metrics, campaigns, flows. Custom 'Klaviyo-API-Key' auth prefix (not Bearer). All tools pin the required revision header. Catalog: 50 adapters total. Validator clean.
Wave 2 of the greenfield batch — CRM extension + email/MA platforms not yet covered by vendor-official MCP servers. - Close (CRM): 16 tools — leads/contacts/opportunities CRUD with custom-field discovery, activities (note creation only — typed call/email/sms activities should come from real channels), tasks, smart-view queries, statuses. BASIC_AUTH with key-as-username. - Brevo (formerly Sendinblue, EU email/SMS): 14 tools — transactional email + SMS, contacts CRUD with double-opt-in awareness, lists and folders, campaigns create+sendNow, aggregated stats. Lowercase 'api-key' header convention. - Kit / ConvertKit v4: 14 tools — subscribers, tags, sequence enrollment, forms, broadcasts. Modern v4 Bearer API (uses the new key format from Settings → Advanced). - Loops (modern product email): 9 tools — contacts upsert via /contacts/create and /contacts/update, send transactional email with dataVariables, fire behavioral events to trigger Loops automations, list mailing lists and custom fields. - ActiveCampaign v3: 16 tools — contacts (sync, update, delete), list memberships with status integers (1=Active/2=Unsub/0=Pending double-opt-in), tags, deals + pipelines + stages, custom fields. Custom Api-Token header, account-templated base URL. Catalog: 55 adapters total. All 5 smoke-tested (401 from vendor edge confirms path recognition).
Wave 3 — sales/CRM completion. Five connectors covering contact-database enrichment, sales-engagement cadences, and Google-Workspace-native CRM. - Copper CRM: 12 tools — people, companies, opportunities CRUD with custom-field discovery, pipelines, activities, search via POST (Copper-specific convention). Requires 3 auth headers simultaneously (token + appname + user email) — added extraHeaders support to RestEngine API_KEY profile to handle this without polluting per-tool headers. - Apollo.io: 11 tools — people/org search across 275M+ database, enrichment by email/linkedin/id, search of your CRM contacts, cadence enrollment. Instructions document credit-consumption behavior for reveal operations. - Salesloft v2: 11 tools — people/accounts CRUD, cadence list and enrollment via cadence_memberships, call activity logging, user listing. Bearer OAuth2 with short-lived tokens. - Outreach v2: 11 tools — prospects/accounts CRUD with JSON:API envelope, sequences, sequenceStates for enrollment, mailboxes (sending identity), tasks, stages. PATCH for updates. - Lemlist v1: 11 tools — campaigns and their stats, leads in campaign (add/pause/resume/delete), global unsubscribe, webhooks list. BASIC_AUTH with empty username (key as password — Lemlist convention). Engine change: API_KEY auth now supports extraHeaders for vendors like Copper that need multiple fixed headers alongside the key. Backward compatible. Catalog: 60 adapters. All 5 smoke-tested.
Wave 4 — project / task management batch. Five connectors covering the full spectrum from personal todos (Todoist) to enterprise project management (Basecamp, ClickUp). - ClickUp v2: 16 tools — workspace/space/folder/list hierarchy navigation, tasks CRUD with custom_task_ids support, comments, custom fields per list, team members. Raw token in Authorization header (no Bearer prefix — ClickUp-specific). - Trello v1: 13 tools — boards, lists, cards CRUD with move/archive, comments, URL attachments, board labels, universal search. QUERY_AUTH with key+token (Trello's classic 2-value scheme). - Todoist API v1: 12 tools — projects, tasks with natural-language due dates, comments, labels. Switched to the modern /api/v1 base (rest/v2 is deprecated as of 2025 — vendor returned 410 Gone). - Basecamp 4: 12 tools — projects with dock discovery, todolists and todos with complete/uncomplete via POST/DELETE on completion, message board posts, comments on any recording, people. Vendor REQUIRES User-Agent header on every request — pinned per-tool. - Coda v1: 13 tools — docs, tables, columns, rows with formula-based filtering and upsert via keyColumns, async mutation status check, named-formula evaluation. Catalog: 65 adapters. All 5 smoke-tested.
Wave 5 — utility batch (enrichment + maps + docs). - Hunter.io v2: 7 tools — domain-search, email-finder, email-verifier, combined person+company enrichment. QUERY_AUTH with api_key. - NeverBounce v4.2: 6 tools — single sync check + bulk async job flow (create/parse/start/status/results). QUERY_AUTH with key. - Mapbox: 8 tools — forward/reverse geocoding v6, directions, isochrones, matrix, static images, tilequery, list styles. Documents the lng,lat coordinate convention quirk. - Nominatim (OpenStreetMap, FREE no-auth): 4 tools — search, reverse, lookup by OSM ID, status. authType=NONE. Every tool pins User-Agent (Nominatim usage policy requirement). Instructions document the 1 req/sec self-throttling requirement. - Mintlify v1: 5 tools — trigger update, project info, assistant conversations log, search-queries analytics, site analytics. Catalog: 70 adapters (30/81 of the greenfield batch done). Smoke-test summary: Hunter+Mapbox 401 (auth rejected, path OK); NeverBounce 200 (endpoint public-by-default for /account); Nominatim 200 (truly public).
Wave 6 — mixed batch covering forms, support, payments, e-signature.
- Tally: 6 tools — read workspaces/forms/submissions, get form
questions. Modern Typeform alternative with generous free tier.
- SurveyMonkey v3: 8 tools — surveys, responses bulk, collectors,
contact lists. Heavy on read for response analysis.
- Freshdesk v2: 16 tools — tickets CRUD with reply/note distinction,
Lucene-like search, contacts/companies/agents/groups, ticket fields
for custom-field composition. BASIC_AUTH with key as username + 'X'
as password (Freshdesk convention). Subdomain-templated base URL.
- Mollie v2: 14 tools — payments, refunds, customers, subscriptions
with mandates, mandates revoke, methods. Documents the
{currency,value:'10.00'} amount shape (value is STRING with exact
decimals — sending number errors).
- PandaDoc v1: 10 tools — create document from template, send for
signature, poll status, download PDF, templates with role/token
discovery. Custom 'API-Key' prefix (NOT Bearer).
Catalog: 75 adapters (35/81 greenfield batch done).
Wave 7 — publishing + social + customer support. - Ghost Admin v5: 12 tools — posts/pages/tags/members/newsletters CRUD with the required optimistic-concurrency updated_at field on writes. JWT-based auth (operator generates short-lived JWT via Ghost SDK and passes as GHOST_ADMIN_JWT env var). - Substack public: 5 tools — public posts/comments/RSS read. NO auth. Substack has no public write API, so this is read-only by necessity. - Reddit OAuth2: 12 tools — subreddit listings, search, comments/posts/submit/vote/save, my subreddits. Uses oauth.reddit.com (not www.) for authenticated requests. Reddit REQUIRES a meaningful User-Agent header — adapter pins one via extraHeaders on API_KEY auth (the engine extension shipped in the Copper commit). - Help Scout Mailbox v2: 13 tools — conversations CRUD with the HAL+JSON spec, threads (replies + internal notes), customers, tags, users. PATCH for status transitions uses JSON Patch ops. - Front: 13 tools — conversations with filter[q] syntax, messages, replies (creates new message in conversation), internal comments, contacts with multi-handle support, tags, teammates. Catalog: 80 adapters (40/81 of the greenfield batch done — ~50%).
Wave 8 — e-commerce + scheduling + analytics + email automation. - BigCommerce v3: 14 tools — products CRUD with variants, categories, customers, orders (uses v2 endpoints for orders since v3 hasn't migrated them). X-Auth-Token header + store-hash in baseUrl. Paths explicitly carry /v2 vs /v3 prefix. - Acuity Scheduling v1: 12 tools — appointment types, calendars, availability windows (dates + times), appointments CRUD with reschedule/cancel, clients, forms. BASIC_AUTH with userId+key. - Drip v2: 12 tools — subscribers upsert, tags, events for workflow triggers, campaigns enrollment, e-commerce order tracking, custom fields discovery. BASIC_AUTH with key as username + empty password. Account-templated base URL. - Heap server-side: 5 tools — single + bulk event tracking, user properties, account properties (B2B), user-account association. authType=NONE (Heap's server-side ingest uses app_id in body). - Fathom Analytics v1: 6 tools — sites, events list/create, aggregations (the flexible query API), current visitors realtime. Catalog: 85 adapters (45/81 of the greenfield batch done, ~56%).
Wave 9 — payments + support + feature flags. - Chargebee v2: 14 tools — customers, subscriptions with item-prices, pause/resume, invoices, items, item_prices, coupons. BASIC_AUTH. Site-templated baseUrl. form-urlencoded bodies (Chargebee convention). - Recurly v2021-02-25: 11 tools — accounts, subscriptions with cancel/terminate, invoices, plans, coupons. BASIC_AUTH. Required vendor-specific Accept header pinned per-tool. - Crisp v1: 11 tools — conversations CRUD with state transitions and routing, messages with internal-note support, people profiles, operators. Website ID baked into baseUrl. X-Crisp-Tier: plugin header on every request (Crisp requirement). - Adyen Checkout v71: 10 tools — paymentMethods discovery, create payment, payments/details for 3DS finalization, capture, refund, cancel, reversal, sessions, origin keys. X-API-Key header. Test environment by default. - Statsig: 10 tools — Server SDK (check_gate, get_config, get_experiment, initialize, log_event) + Console (list/create gates, list/get experiment results). Single adapter spans TWO distinct API hosts via per-tool absolute URLs. Engine change: RestEngine now allows endpointMapping.path to be a full absolute URL — bypasses baseUrl when the path starts with http(s)://. Lets adapters cover multi-host vendors (Statsig) without splitting into two connector records. Backward compatible. Catalog: 90 adapters (50/81 of the greenfield batch done, ~62%).
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Wave 10 — enrichment + knowledge + publishing + forms. - Clearbit (HubSpot Breeze Intelligence): 6 tools — person enrich by email, company by domain, combined, discovery search, reveal by IP, autocomplete. Multiple distinct subdomains (person/company/discovery/reveal/autocomplete) handled via per-tool absolute URLs. - Snov.io v1: 9 tools — domain search, email finder by name/URL, email verifier, prospect lists CRUD, drip campaigns list. OAuth2 client-credentials Bearer. - GitBook v1: 9 tools — orgs/spaces/collections, page tree, page content (JSON AST or Markdown), full-text search, markdown import as a page write path. Bearer. - beehiiv v2: 11 tools — publication info, posts CRUD with audience/platform/status filters, subscriptions CRUD with custom fields, segments, test-email send. Publication ID baked into baseUrl. - Wufoo v3: 6 tools — forms, fields discovery, entries with the vendor's quirky filter syntax, programmatic entry submission. Subdomain-templated URL. Basic auth with literal 'footastic' password (Wufoo convention). Catalog: 95 adapters (55/81 of the greenfield batch done, ~68%).
Wave 11 — e-commerce + social + CRM + forms continuation. - Magento 2 / Adobe Commerce: 12 tools — products with SKU as primary key, categories tree, stock update, orders with cancel, customers. Bearer admin-token. Heavy use of Magento's searchCriteria filter_groups DSL via templated query params. - Etsy Open API v3: 9 tools — user/shops, listings active/single, receipts with shipping status filters, reviews, listings by section. DUAL AUTH — x-api-key header (App key) + Authorization Bearer (OAuth user token). Engine extraHeaders pattern. - YouTube Data v3: 9 tools — search (100 quota units!), videos, channels, playlists, playlist items, comment threads, video categories, captions. QUERY_AUTH with key. Documents quota cost model per endpoint. - Insightly v3.1: 12 tools — contacts/organisations/leads/ opportunities CRUD, pipelines, projects, tasks. BASIC_AUTH with pod-templated baseUrl (na1/eu2/...). - Fillout v1: 6 tools — forms, submissions list+get+create+delete. Bearer auth. Modern Typeform alternative. Catalog: 100 adapters (60/81 of the greenfield batch done, ~74%).
Wave 12 — modern CRM + project mgmt + knowledge + e-signature. - Attio v2: 12 tools — schema-flexible object model (people/ companies/deals/custom), query with attio filter DSL, get, create, update (PATCH), upsert via matching_attribute, lists, notes (markdown). Bearer. - Folk v2: 9 tools — people CRUD with group memberships, companies, groups with group-scoped custom fields. Bearer. - TickTick v1: 6 tools — projects (lists), get project with all tasks + columns, tasks CRUD with iCalendar RRULE recurrence, complete. OAuth2 Bearer. - Slab: 6 GraphQL tools — me, get post (markdown), search, topics, topic posts, create post from markdown. GRAPHQL connector type — auto-injected slab_graphql_query/mutation/ schema for arbitrary operations. - Dropbox Sign (formerly HelloSign) v3: 10 tools — signature requests CRUD (with test_mode=1 default), send-with-template vs send-from-file_urls, reminder, cancel, download final PDF, templates list+get. BASIC_AUTH with key as user. Catalog: 105 adapters (65/81 of the greenfield batch done, ~80%).
Wave 13 — publishing + project mgmt + scheduling + comms + e-sig. - Medium v1: 5 tools — me/publications/contributors (read), create post under user OR publication with html/markdown content. Note Medium API is write-mostly (no public read for posts since 2019). - Height: 9 tools — workspace, lists, tasks with JSON filters, search, CRUD, users, custom fields. api-key prefix (NOT Bearer). - SavvyCal v1: 6 tools — scheduling links, meetings, cancel. Bearer. - MessageBird v1: 8 tools — balance, SMS send/list/get, voice call with callFlow steps, phone lookup, verify create+check (OTP flow). AccessKey prefix (NOT Bearer). - SignWell v1: 8 tools — documents list/get, create from URLs OR template, reminders, cancel, download completed PDF, templates. Dual-header auth (X-Api-Key + X-Api-Application via extraHeaders). Catalog: 110 adapters (70/81 of the greenfield batch done, ~86%).
…, Instantly Wave 14 — scheduling + support + e-commerce + analytics + cold email. - Microsoft Bookings (via Graph v1.0): 8 tools — businesses, services, staff, appointments CRUD + cancel-with-message. Bearer OAuth2 against Microsoft Graph. - Kustomer v1: 10 tools — customers search/CRUD with multi-handle support, conversations search/get/update status/assign, internal notes, outbound messages (email/sms/chat). Subdomain-templated. - Ecwid v3: 10 tools — store profile, products search/CRUD, categories, orders search/get/update with payment+fulfillment status codes, customers search. Store-templated baseUrl. - Vercel Analytics: 4 tools — user, projects list, deployments list, Speed Insights aggregations. Bearer. - Instantly v2: 8 tools — campaigns list/get/analytics, leads list/add with dedupe options, update interest_status (hot/warm/ cold), lead lists, sending accounts. Catalog: 115 adapters (75/81 of the greenfield batch done, ~93%).
Wave 15 — final batch of the greenfield catalog growth. - Mailshake: 8 tools — campaigns list/get/pause/unpause, recipients add (with dedupe) / list / pause. BASIC_AUTH with key as user. - Microsoft Teams (via Graph v1.0): 9 tools — joined teams, channels list/get, messages list/send/reply with HTML body + mentions[], channel + team members. Shares MICROSOFT_GRAPH_ACCESS_TOKEN env var with the Bookings adapter. - LinkedIn v2: 5 tools — userinfo (OIDC), create/get/delete post for w_member_social scope, register image upload. Documents the Marketing Developer Platform gating limitation that blocks most other useful endpoints. - Amadeus Self-Service (travel): 8 tools — flight offers search + price confirmation + destinations inspiration, airport/city autocomplete, hotel list + offers, POIs, flight status. OAuth2 client-credentials Bearer. Test environment by default. Catalog: 119 adapters total (79/81 of the greenfield batch — the plan called for 81 minus the 1 (WooCommerce) that landed via external PR while I was working). This closes the greenfield connector batch. PR #232 is ready for final review + merge once CI passes.
- slab.json: GRAPHQL connectors put the operation type in
endpointMapping.method ('query'|'mutation') and the GraphQL
document in .path — NOT POST + body. The catalog.spec.ts test
enforces this distinction (VALID_GRAPHQL_METHODS) and CI was
failing on 6 slab tools. Rewrote to follow the existing Sorare
pattern with variables under bodyMapping.variables.
- statsig.live.spec.ts: replaced substring URL match with new URL()
hostname comparison — CodeQL flagged the startsWith pattern as
incomplete-url-sanitization (false positive in test code but it
was blocking the PR merge). Stricter check is also more correct.
keysersoft
added a commit
that referenced
this pull request
May 20, 2026
…ks (#233) PR #232 introduced a backend prebuild hook that runs node ../../scripts/regenerate-catalog.mjs to keep catalog.ts in sync with adapter JSONs. The Docker build only COPYs packages/backend into the image, so the prebuild was failing with MODULE_NOT_FOUND. Add a single COPY for the script (one tiny file, no deps).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status: WORK IN PROGRESS — do not merge yet
Long-lived branch that batches the next ~81 SaaS connector adapters in one PR. Building incrementally; merging the whole branch in one go at the end (`gh pr merge --squash --admin` after final QA).
Wave 0 — Foundation (this initial commit)
Wave 1+ — Connectors (incoming commits)
Will add ~81 new connector adapters across CRM, email/MA, project mgmt, scheduling, forms, support, payments, e-commerce, analytics, comms, publishing, e-signature, lead enrichment, knowledge, social, maps. Each follows the WhatsApp-Business-shaped pattern:
Excluded from this batch (verified, vendor MCP already ships): HubSpot, PayPal, Atlassian (Jira+Confluence), Monday, Cal.com, Resend, Mixpanel, PostHog, Square, JotForm, Paddle, WooCommerce, Customer.io, Vonage, Zoom, DocuSign, Mailgun.
Test plan