Skip to content

feat: add sql-schema-transform cloud function#7

Open
pyramation wants to merge 22 commits into
mainfrom
devin/1768145100-sql-schema-transform-fn
Open

feat: add sql-schema-transform cloud function#7
pyramation wants to merge 22 commits into
mainfrom
devin/1768145100-sql-schema-transform-fn

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

Summary

Adds a new Knative cloud function that transforms schema names in SQL using AST-based parsing with plpgsql-parser. The function accepts SQL and a schema mapping, then returns the transformed SQL with schema names replaced according to the mapping.

The transformation handles:

  • Schema-qualified table references in SELECT/FROM (RangeVar nodes)
  • DDL statements: CREATE TABLE, CREATE INDEX, ALTER TABLE, TRUNCATE, CREATE TRIGGER, CREATE RULE, CREATE POLICY
  • CREATE SCHEMA, DROP SCHEMA, GRANT ON SCHEMA statements
  • SET search_path statements
  • Schema-qualified function calls and type names
  • Schema names inside PL/pgSQL function bodies (via hydrated AST)

API:

POST /
{
  "sql": "SELECT * FROM old_schema.users",
  "schema_mapping": { "old_schema": "new_schema" }
}
// Returns: { transformed_sql, schemas_found, schemas_transformed }

Review & Testing Checklist for Human

  • Verify DDL coverage is complete - Compare against the original transform-schemas-ast.ts in constructive-db. Other DDL statements may need handlers (e.g., ViewStmt, CreateSeqStmt, CommentStmt)
  • Test edge cases - Quoted identifiers, mixed-case schema names, %TYPE/%ROWTYPE suffixes in PL/pgSQL
  • Deploy and invoke - Verify the function can be deployed to Knative and invoked via HTTP
  • Consider adding unit tests - No automated tests are included; the function was only tested locally with curl

Recommended test plan:

cd functions/sql-schema-transform && pnpm build && PORT=8080 node dist/index.js

curl -X POST http://localhost:8080 \
  -H "Content-Type: application/json" \
  -d '{"sql": "CREATE TABLE old_schema.users (id uuid); SELECT * FROM old_schema.users;", "schema_mapping": {"old_schema": "new_schema"}}'

Notes

  • Uses any types for AST nodes (consistent with plpgsql-parser patterns but reduces type safety)
  • No README or Dockerfile added (unlike some other functions in this directory)
  • Local testing confirmed CREATE TABLE, CREATE SCHEMA, SET search_path, GRANT, SELECT, and PL/pgSQL function body transformations work correctly

Requested by: @pyramation
Link to Devin run: https://app.devin.ai/sessions/2eed6a2d4f0142349bb3bdd50334b23f

Jovonni and others added 22 commits January 8, 2026 14:37
Adds a new Knative cloud function that transforms schema names in SQL using
AST-based parsing with plpgsql-parser. The function accepts SQL and a schema
mapping, then returns the transformed SQL with schema names replaced.

The transformation logic handles:
- Schema-qualified identifiers in RangeVar nodes (schema.table in SELECT/FROM)
- Schema names in CreateSchemaStmt, DropStmt, GrantStmt nodes
- DDL statements with embedded relations: CreateStmt (CREATE TABLE), IndexStmt,
  AlterTableStmt, TruncateStmt, CreateTrigStmt, RuleStmt, CreatePolicyStmt
- Schema-qualified function names in FuncCall nodes
- Schema-qualified type names in TypeName nodes
- Schema names in VariableSetStmt (SET search_path)
- Schema names inside PL/pgSQL function bodies (hydrated AST)

API:
POST / with JSON body: {sql, schema_mapping}
Returns: {transformed_sql, schemas_found, schemas_transformed}
@devin-ai-integration
Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@Anmol1696 Anmol1696 force-pushed the devin/1768145100-sql-schema-transform-fn branch from ce265a7 to dd2c7e9 Compare May 12, 2026 05:37
@Anmol1696
Copy link
Copy Markdown
Contributor

Heads-up: repository history was rewritten on 2026-05-12 to scrub leaked secrets (Postgres/pgAdmin default passwords, an AWS access key ID, generated k8s/manifests/interweb-*.yaml files) from every commit. Every branch on origin was force-pushed with new commit SHAs.

This PR shows "DIRTY" / merge-conflict status because main has a cleanup commit and this branch is based on pre-rewrite main. To clear it:

git fetch --all --prune
git checkout <this-branch>
git reset --hard origin/<this-branch>    # your local branch tip moved too — pick up the rewritten version
git rebase origin/main                    # rebase onto rewritten main
# resolve conflicts (usually trivial — mostly secret-placeholder + the 4 deleted interweb-*.yaml files)
git push --force-with-lease

Or merge instead of rebase if the branch is shared with others:

git merge origin/main
git push

Notes:

  • The secrets that leaked were all either dead (rotated AWS key) or were defaults that have since been rotated/replaced; no active credential is exposed.
  • Old commit SHAs are still accessible by direct URL on GitHub for ~90 days (can be expedited via GitHub Support if needed).
  • See k8s/SECRET-EXPOSURE-AUDIT.md on main for the full incident audit.

@devin-ai-integration
Copy link
Copy Markdown

Devin is archived and cannot be woken up. Please unarchive Devin if you want to continue using it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants