Update Salesforce Connector To Create Source Models#9491
Open
cwarden wants to merge 2 commits into
Open
Conversation
The Salesforce connector now uses Salesforce's OAuth 2.0 endpoints (SOAP login is being decommissioned) and fits the warehouse-style connector + model flow used by other warehouses. - Authentication: drop SOAP login; pick between OAuth password flow, client credentials flow, and JWT bearer based on which credentials are populated. Adds a required `client_secret` for username/password authentication. - Schema: mark Salesforce as a warehouse so credentials are written to `connectors/<name>.yaml` and the SOQL query lives in `models/<name>.yaml`. The `key` field is uploaded as a file and base64-encoded before being stored in `.env` so embedded newlines do not break dotenv parsing; the driver accepts raw PEM as well. - Driver: accepts `sql:` as an alias for `soql:` to match the model shape produced by the explorer.
- Bulk API 2.0: replace the v1 query path with `Bulk2QueryJob`. v2 manages chunking server-side and derives the SObject from the query itself, so `sobject:` is no longer a required model property. - SELECT * rewrite: SOQL does not accept `SELECT *`, but the connector explorer's "Table" mode produces it. The driver detects `SELECT * FROM <SObject>` (with optional WHERE / ORDER BY / LIMIT), calls DescribeSObject, and rewrites the query into an explicit field list. Compound `address` / `location` types are skipped — their atomic components remain because Salesforce exposes them as their own fields. - Information schema: implement `ListDatabaseSchemas` / `ListTables` / `GetTable` via `force.ListSobjects` and `force.DescribeSObject`, so the connector explorer can browse SObjects as tables and clear the "does not implement information schema" error. - Shared auth helpers: extract `connection.authOptions()` and `sourceProperties.applyOverrides()` so InformationSchema, Ping, and QueryAsFiles all build their auth options the same way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update the Salesforce driver to create salesforce connectors and source models rather than the deprecated sources. Re-enable it in the Add Data modal.
Update the Salesforce connector to use OAuth Username/Password flow because SOAP login is being decommissioned. Client Credentials and JWT flows are also supported.
Update Salesforce models to use Bulk API 2.0, which doesn't require the sobject property, allowing it to work with the New Model modal.