feat(appkit): add appkit db migrate by ditadi · Pull Request #666 · databricks/appkit

ditadi · 2026-10-13T13:22:49Z

🥞 Stacked PR (generated by Stack)

Use this link to review incremental changes.

stack/database-dev-flow/00-design [Files changed]
stack/database-dev-flow/01-migrations-on-deploy [Files changed]
- stack/database-dev-flow/02-default-app-schema [Files changed]
  - stack/database-dev-flow/03-db-migrate [Files changed] ← this PR
    - stack/database-dev-flow/04-db-introspect [Files changed]
      - stack/database-dev-flow/05-db-init [Files changed]
        
        stack/database-dev-flow/06-db-verify [Files changed]
        
        stack/database-dev-flow/07-apps-init [Files changed]
        
        stack/database-dev-flow/08-db-check [Files changed]

Stacked on #655. The design is #677.

What did you change, and why?

#654 applies committed migrations when the deployed app starts, so something has to write those migrations and try them first. A developer or an agent changes schema.ts, runs one command, reads the error and tries again, with no hand edits to drizzle's files.

appkit db migrate generates the next migration from config/database/schema.ts with drizzle-kit, regenerates shared/appkit-types/database.d.ts, and applies pending migrations to the branch .env points at. It applies only to your dev branch or this checkout's session branch. On any other branch it still generates, and says why nothing was applied. It warns about drops, renames, type changes and new NOT NULL columns, which break instances of the previous release. When the migration it wrote fails on the branch, it removes that file and restores the journal. Before connecting, it checks that PGHOST is a host of LAKEBASE_ENDPOINT. drizzle-kit is an optional peer dependency pinned to 0.31.10, and an ambiguous rename outside a TTY throws and writes nothing.

This revision changes these behaviors:

The tooling moves to @databricks/appkit/beta/database-tooling. The old @databricks/appkit/database-tooling subpath is gone, with no alias. oxlint still stops runtime code from importing the tooling or drizzle-kit.
Session branches are session-<user hash>-<id>, so a .env copied from another developer does not make their session branch yours.
The deploy boundary reads every target, every postgres resource of every app, the postgres_branch variable, the files include: names, and each target's .databricks/bundle/<target>/variable-overrides.json. A branch any of them names takes no writes.
A branch or database reference those files cannot resolve, such as an unset postgres_branch, makes migrate apply to no branch, and the message names the reference. So does an include: pattern with [, ] or \, or an overrides file that is not a JSON object.
migrate holds config/database/.appkit-db.lock for its whole run, including the discard of a failed migration. The lock file is created exclusively with mode 0600. A second appkit db command in the checkout fails and names the holder. AppKit never deletes a lock it does not hold, and the message gives the manual recovery step for a stale one.
bundle-size-baseline.json is regenerated. The packed tarball grows from 1,297,373 to 1,364,290 bytes (+5.16%), over the 5% budget, because of the CLI-only tooling entry.

appkit db stays hidden from appkit --help, and nothing a deployed app runs changes. Some messages tell you to run appkit db init, which #672 adds. The guard reads only this checkout's files. It does not see --var, BUNDLE_VAR_* or resources defined in Python, and it does not replace Lakebase permissions.

How do you know it works?

targets.test.ts, guard.test.ts, naming.test.ts and history-lock.test.ts cover every binding kind, overrides and included files, unresolved references, user-bound session names, and how the lock is taken, released and reported when stale. migrate.test.ts checks that a second concurrent run is refused and writes nothing.
migrate.pg.test.ts runs against Postgres. A NOT NULL column on a table with a row fails with 23502 and is removed. The fixed schema.ts then applies, and the row keeps its data.
At this PR's head, a clean build matched the new size baseline, typecheck passed, and the tooling and CLI db tests passed against a local PostgreSQL 14. The full vitest suite passed at an earlier revision of this PR and at the top of the stack (feat(appkit): add appkit db check #678). The packed tarballs export the same root and ./beta names as main, and ./beta/database-tooling is the only new subpath. The built CLI leaves db out of appkit --help. CI runs the full suite on this PR.
An earlier revision checked currentUser.me, an endpoint lookup and a branch lookup live and read-only. This revision was not run against a real Lakebase workspace.

github-actions · 2026-10-13T13:37:51Z

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

`@databricks/appkit`

npm tarball (packed): 1.3 MB (+7 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist	raw	gzip
JS (runtime)	1.3 MB	485 KB
Type declarations	468 KB	170 KB (+1 B)
Source maps	2.6 MB	904 KB
Other	11 KB	3.7 KB
Total	4.4 MB	1.5 MB (+1 B)

Per-entry composition (own code — deps external (as shipped))

Entry	Initial (gz)	Lazy (gz)	Total (gz)	node_modules (min)	Own code (min)
`.`	106 KB	2.5 KB	108 KB	external	343 KB
`./beta`	108 KB	485 B	108 KB	external	323 KB
`./beta/database-tooling`	28 KB	0 B	28 KB	external	82 KB
`./testing`	45 KB	32 KB	77 KB	external	223 KB
`./tsdown`	520 B	0 B	520 B	external	813 B
`./type-generator`	26 KB	0 B	26 KB	external	74 KB

Chunks:

Entry	Chunk	Load	Size (gz)
`.`	`index.js`	initial	101 KB
`.`	`utils.js`	initial	4.6 KB
`.`	`remote-tunnel-manager.js`	lazy	2.5 KB
`./beta`	`beta.js`	initial	88 KB
`./beta`	`modular.js`	initial	6.0 KB
`./beta`	`stream-manager.js`	initial	5.9 KB
`./beta`	`service-context.js`	initial	4.2 KB
`./beta`	`databricks.js`	initial	3.3 KB
`./beta`	`client.js`	initial	647 B
`./beta`	`index.js`	initial	20 B
`./beta`	`supervisor-api.js`	lazy	192 B
`./beta`	`databricks.js`	lazy	178 B
`./beta`	`index.js`	lazy	115 B
`./beta/database-tooling`	`index.js`	initial	28 KB
`./testing`	`manifest.js`	initial	32 KB
`./testing`	`index.js`	initial	10 KB
`./testing`	`wide-event-emitter.js`	initial	2.9 KB
`./testing`	`index.js`	lazy	28 KB
`./testing`	`remote-tunnel-manager.js`	lazy	2.5 KB
`./testing`	`utils.js`	lazy	1.8 KB
`./tsdown`	`index.js`	initial	520 B
`./type-generator`	`index.js`	initial	26 KB

`@databricks/appkit-ui`

npm tarball (packed): 397 KB (+4 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist	raw	gzip
JS (runtime)	429 KB	147 KB
Type declarations	267 KB	98 KB (+1 B)
Source maps	853 KB	286 KB
CSS	16 KB	3.2 KB
Total	1.5 MB	534 KB (+1 B)

Per-entry composition (consumer bundle — deps bundled, peerDeps external)

Entry	Initial (gz)	Lazy (gz)	Total (gz)	node_modules (min)	Own code (min)
`./js`	5.3 KB	49 KB	55 KB	208 KB	14 KB
`./js/beta`	2.1 KB	0 B	2.1 KB	0 B	5.0 KB
`./react`	433 KB	49 KB	481 KB	1.3 MB	177 KB
`./react/beta`	4.9 KB	0 B	4.9 KB	0 B	12 KB

Chunks:

Entry	Chunk	Load	Size (gz)
`./js`	`index.js`	initial	5.2 KB
`./js`	`chunk`	initial	120 B
`./js`	`apache-arrow`	lazy	49 KB
`./js/beta`	`beta.js`	initial	2.1 KB
`./react`	`index.js`	initial	431 KB
`./react`	`tslib`	initial	2.1 KB
`./react`	`apache-arrow`	lazy	49 KB
`./react/beta`	`beta.js`	initial	4.9 KB

github-actions · 2026-10-13T13:42:11Z

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 41109403208 -R databricks/appkit -n appkit-template-0.89.0-pr.7490e85-stack-database-dev-flow-03-db-migrate-666 -D appkit-pr-666 \
  && unzip -o "appkit-pr-666/appkit-template-0.89.0-pr.7490e85-stack-database-dev-flow-03-db-migrate-666.zip" -d "appkit-pr-666" \
  && databricks apps init --template "appkit-pr-666"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

`appkit db migrate` generates a migration from config/database/schema.ts with drizzle-kit, regenerates shared/appkit-types/database.d.ts, and applies pending migrations to the branch .env points at. It applies only to a branch created for this developer (dev-…) or this checkout (session-…); on any other branch, including the project default and any branch databricks.yml deploys to, it generates and says why nothing was applied. It connects only after the Lakebase API confirms that PGHOST is a host of LAKEBASE_ENDPOINT. The logic lives in a new @databricks/appkit/database-tooling entry point that the shared CLI loads lazily, so the runtime never imports drizzle-kit. drizzle-kit is an optional peer dependency: apps that generate migrations install it as a dev dependency, and the command says how when it is missing. `appkit db` stays hidden from --help until the database plugin is offered by `databricks apps init`. Signed-off-by: ditadi <victordperd@gmail.com>

A generated migration that fails to apply is removed and the journal restored, so a fix in schema.ts replaces it instead of stacking behind it. Also: refuse a drizzle-kit other than the pinned one or one loading another drizzle-orm, parse the branch and endpoint responses the guard decides on, type the CLI against a contract appkit's typecheck enforces, and enforce the drizzle-kit and tooling import boundary with oxlint. Signed-off-by: ditadi <victordperd@gmail.com>

…nch pool resolveEnvBranch returns the branch kind (default, protected, deploy target, dev, session, other), and takesWrites is the one rule for which kinds take writes. It needs PGHOST, PGDATABASE and LAKEBASE_ENDPOINT, like the pool. withBranchPool opens and always closes the pool. The drizzle-kit v7 file names live only in migrations/paths.ts. Signed-off-by: ditadi <victordperd@gmail.com>

…aks the previous release Apps keeps instances of the previous release serving while the new one migrates on start, and a code rollback cannot undo a migration. When the generated migration drops, renames, retypes or makes a column NOT NULL, the report lists those statements and the CLI says to ship code that works with both schemas first. Signed-off-by: ditadi <victordperd@gmail.com>

…lient Signed-off-by: ditadi <victordperd@gmail.com>

Signed-off-by: ditadi <victordperd@gmail.com> # Conflicts: # pnpm-lock.yaml

…tabase-tooling The CLI-only tooling was exported as @databricks/appkit/database-tooling, outside the beta boundary. Export it only as @databricks/appkit/beta/database-tooling and drop the old subpath. Migrate the CLI loader, its ambient declaration, the runtime import restriction, the test mock and the comments that name the path. Signed-off-by: ditadi <victordperd@gmail.com>

A session branch was session-<id>, and the guard accepted it for anyone whose .env carried that id, so a copied .env let another developer write to it. Name session branches session-<user hash>-<id>, the hash dev branch names already use, so only the user who created the branch passes the guard. Signed-off-by: ditadi <victordperd@gmail.com>

… not name The guard read only each target's postgres_branch variable and dropped references it could not resolve, so a branch bound through an app's postgres resource, an included file or an unresolved reference could pass as a dev branch. Read every postgres binding of every app and target, including included files, into one deploy boundary that keeps unresolved branch and database references. While any reference is unresolved, no branch takes writes. Signed-off-by: ditadi <victordperd@gmail.com>

Generation read the journal, waited for drizzle-kit, then wrote the next SQL, snapshot and journal, and a failed apply restored the journal read before. Two commands in one checkout could overwrite or remove each other's migration. appkit db migrate now holds config/database/.appkit-db.lock, created exclusively, from before it reads the history until it finishes. Generation and discard take the held lock, and discard restores the journal that generation itself replaced. A lock left by a killed process is reported as stale with the manual recovery step, never taken over. Signed-off-by: ditadi <victordperd@gmail.com>

Document where the beta tooling lives, which branches migrate writes to, the refusal while databricks.yml has an unresolved branch or database reference, the file-only limits of that check, and how to recover from a stale config/database/.appkit-db.lock. Signed-off-by: ditadi <victordperd@gmail.com>

The Databricks CLI applies .databricks/bundle/<target>/variable-overrides.json ahead of a target's variables and their defaults, so an override could deploy the app to a developer's dev branch while the guard still classified it as dev. Read each target's override file and add its bindings to the deploy boundary next to databricks.yml's own. A value that is not a resolvable string is unresolved, and a file that is not a JSON object throws. Signed-off-by: ditadi <victordperd@gmail.com>

includedFiles matched only * and ?, so an include: pattern with [, ] or \\ matched no file and the bindings in the files it named were left out of the deploy boundary. Throw "Cannot read include pattern ..." for such a segment instead. Signed-off-by: ditadi <victordperd@gmail.com>

A lock whose pid runs on this host was reported only as busy, so a pid reused after a reboot left no way forward. End the message with the stale-lock recovery step, and say in the docs when each message appears. Signed-off-by: ditadi <victordperd@gmail.com>

Regenerated with pnpm clean && pnpm build && pnpm size:baseline. Against main's baseline the @databricks/appkit packed tarball grows from 1297373 to 1364290 bytes (+66917, +5.16%), over the 5% budget. The growth is the CLI-only @databricks/appkit/beta/database-tooling entry (28 KB gzipped) and the database tooling it ships. @databricks/appkit-ui is unchanged at 406741 bytes. Signed-off-by: ditadi <victordperd@gmail.com>

ditadi added this pull request to stack #667 October 13, 2026 13:34

This was referenced Oct 14, 2026

feat(appkit): add appkit db introspect #670

Draft

docs: design 001, database development flow #677

Open

ditadi force-pushed the stack/database-dev-flow/03-db-migrate branch from 9f3a34f to 1827ebd Compare October 19, 2026 12:44

ditadi added 5 commits October 20, 2026 12:53

refactor(appkit): run the database tooling on the modular workspace c…

0d1dfaf

…lient Signed-off-by: ditadi <victordperd@gmail.com>

ditadi force-pushed the stack/database-dev-flow/03-db-migrate branch from df1b1c9 to 0d1dfaf Compare October 20, 2026 12:23

ditadi added 10 commits October 30, 2026 15:19

Merge branch 'readiness/stack667-02' into readiness/stack667-03

c2f1db9

Signed-off-by: ditadi <victordperd@gmail.com> # Conflicts: # pnpm-lock.yaml

Navigation Menu

Repository navigation

feat(appkit): add appkit db migrate - #666

feat(appkit): add appkit db migrate#666
ditadi wants to merge 15 commits into
stack/database-dev-flow/02-default-app-schemafrom
stack/database-dev-flow/03-db-migrate

ditadi commented Oct 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 13, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ditadi commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🥞 Stacked PR (generated by Stack)

What did you change, and why?

How do you know it works?

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Bundle size report

@databricks/appkit

@databricks/appkit-ui

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 AppKit PR bot

🔬 Run evals

📦 Try this PR's app template

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ditadi commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

`@databricks/appkit`

`@databricks/appkit-ui`

github-actions Bot commented Oct 13, 2026 •

edited

Loading