Skip to content

fix(storage): make cockroachdb, mysql, mariadb and sql server work with gorm AutoMigrate - #826

Merged
lakhansamani merged 5 commits into
mainfrom
fix/cockroachdb-automigrate
Oct 30, 2026
Merged

lakhansamani merged 5 commits into
mainfrom
fix/cockroachdb-automigrate

Conversation

@lakhansamani

@lakhansamani lakhansamani commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

What this is

It started as "add a CockroachDB test". The test did not pass, and neither did
the next two engines I pointed it at. Four of the SQL backends in the supported
list could not create their own schema, and one more was rewriting four tables
on every single restart.

Backend On main After this PR
PostgreSQL / Yugabyte boots, but AutoMigrate rewrites 4 tables under ACCESS EXCLUSIVE on every restart boots; 5 one-time ALTERs on the upgrade boot, 0 on every boot after
CockroachDB boots once, then never starts again (ALTER COLUMN TYPE is experimental, SQLSTATE XCEXF); legacy-unique cleanup silently does nothing boots and re-boots clean; cleanup actually drops the constraints
MySQL / MariaDB never worked. CREATE TABLE fails (Error 1170), then DROP CONSTRAINT fails (1091), then DeleteSessionTokensByUserID fails (1064) fresh install and v1 upgrade both green, 0 DDL after first boot
SQL Server broken since 2.3.0-rc.10. CREATE UNIQUE INDEX on an nvarchar(MAX) column is rejected outright schema creates; fresh install and both data-bearing upgrade paths green, 0 DDL after. 2 upsert tests still red — separate driver bug, see below
SQLite / LibSQL fine unchanged, still green
PlanetScale same MySQL-wire code path as MySQL same fixes apply; hosted, not run here

The five defects

1. char(36) churns forever on Postgres, and is fatal on CockroachDB.
GORM's postgres driver reads these columns back from information_schema as
bpchar, which never prefix-matches the declared char(36), so AutoMigrate
re-emits the same no-op type change on every boot:

ALTER TABLE "authorizer_sessions" ALTER COLUMN "user_id" TYPE char(36) USING "user_id"::char(36)

On PostgreSQL that is a silent table rewrite under ACCESS EXCLUSIVE on four
tables at every restart. On CockroachDB it is fatal — ALTER COLUMN TYPE sits
behind an experimental flag. Five non-PK columns move to varchar(36), which is
what the catalog reports; values are 36-char uuids either way.

2. CockroachDB reports unique indexes as unique constraints.
GORM then issues DROP CONSTRAINT "uni_<table>_<column>" — a name it only ever
creates from a unique tag — for every gorm:"uniqueIndex" field, and startup
dies on constraint ... does not exist (SQLSTATE 42704). Fixed by wrapping the
dialector and skipping the drop when the model declares the index.

3. CockroachDB cannot ALTER TABLE ... DROP CONSTRAINT a UNIQUE.
unimplemented: ... use DROP INDEX CASCADE instead (SQLSTATE 0A000). Without
this the v1 legacy-uniqueness cleanup silently no-ops, leaving UNIQUE on
authorizer_otps.phone_number — where v2 writes '' for every email OTP, so
the second one is rejected.

4. MySQL and MariaDB, three separate aborts.
GORM's mysql driver sizes an unsized string to varchar(191) when the field
carries index or unique — but it checks only those two tags, so a
uniqueIndex field becomes longtext and MySQL refuses the table:
Error 1170: BLOB/TEXT column 'client_id' used in key specification without a key length. Then COLUMN_KEY='UNI' triggers the same phantom drop as (2)
(Error 1091). Then ESCAPE '\' in DeleteSessionTokensByUserID is
Error 1064 — MySQL does not accept a backslash escape character there, so the
escape char is now ! and the LIKE escaping is portable.

5. SQL Server, the same sizing gap as MySQL.
Identical two-tag check, nvarchar(MAX) instead of longtext:

mssql: Column 'client_id' in table 'authorizer_clients' is of a type that is
invalid for use as a key column in an index.

Fixed the same way, at the driver's own width (256). No MigrateColumnUnique
override needed — SQL Server reads uniqueness from
INFORMATION_SCHEMA.TABLE_CONSTRAINTS, which lists real constraints only, so
catalog and models agree as on PostgreSQL.

Backward compatibility

Each dialector embeds the driver's concrete Dialector/Migrator, not the
gorm.Dialector/gorm.Migrator interface, so every method the driver defines —
including the optional ones GORM type-asserts — stays promoted. That detail is
load-bearing: the first version of the CockroachDB wrapper embedded the
interface and panicked the moment a table with an index was created, because
migrator.BuildIndexOptionsInterface is asserted without an ok-check
(migrator.go:272, :822). It only showed up when MySQL was actually run. The
shared helper's doc comment records this so the next backend does not repeat it.

Postgres, Yugabyte, SQLite and LibSQL keep their existing gorm.Open paths
untouched.

Fresh install Upgrade from char(36) Per-boot churn Data integrity
PostgreSQL 18 suite green, 0 ALTERs exactly 5 ALTERs, then 0 0 DDL, measured with log_statement=all '' → length 0, short-id → 8, uuids → 36; byte-identical
CockroachDB v24.3 make test-cockroachdb PASS boots and re-boots 0 DDL —
MySQL 8.0.40 suite green legacy char(36) + seeded authorizer_users_email_key / uq_otps_phone migrate in one boot 0 ALTER TABLE authorizer_* across 3 boots rows intact; two phone_number='' OTPs now insertable
MariaDB 11 suite green — — —
SQL Server 2022 schema creates; 2 known-red upserts v1-shaped and 2.0.0–2.3.0 schemas each upgrade in one boot; both legacy UNIQUEs dropped 0 DDL on the next 2 boots legacy@example.com joins on a 36-char user_id; otp_rows = 2; seeded authenticator row intact with the composite index built over the widened column
SQLite / LibSQL green, incl. a legacy char(36) test.db no-op — —

The MySQL varchar(191) width and the SQL Server nvarchar(256) width only
touch columns in tables that do not exist in v1 (clients, organizations,
org_memberships, federated_identities, scim_endpoints, trusted_issuers,
webauthn_credentials), so nothing is narrowed under existing data.

Known limitations

  • SQL Server upserts. gorm.io/driver/sqlserver's MergeCreate builds its
    MERGE predicate from the primary key and ignores
    clause.OnConflict.Columns, so AddAuthenticator (keyed on
    (user_id, method)) and AddVerificationRequest (keyed on
    (email, identifier)) insert a second row instead of replacing one and hit
    the unique index. Two tests red. It is a write-path driver bug, unrelated to
    migration, and belongs in its own PR — so make test-sqlserver exists but is
    not in test-all-db.
  • SQL Server debris. 2.3.0-rc.10 is where the unsized uniqueIndex
    columns and the composite index over authenticators.user_id both landed, so
    the two never coexist in a database that booted. A database touched by a
    2.3.0-rc.10 or later SQL Server boot has user_id as char(36) inside
    that index, and SQL Server will not retype an indexed column — drop it before
    upgrading; no boot there ever finished, so it holds no data. Databases from
    2.0.0 through 2.3.0 did boot, carry no index over any widened column, and are
    verified above to upgrade with rows intact.
  • GetIndexes is not support on the sqlserver driver, so step 2 of the
    legacy cleanup (standalone unique indexes on email/phone_number) is a
    no-op there. Pre-existing; the constraint form, which is what v1 created, is
    still cleaned.
  • PlanetScale is hosted MySQL-wire and was not run; it shares the MySQL code
    path verbatim.
  • Yugabyte is unchanged — same postgres.Open path, same bpchar catalog —
    and was not run.

Also in here

  • make test-cockroachdb, test-mysql, test-mariadb, test-sqlserver and
    their cleanup targets; test-all-db now covers 12 DBs. Every one of them was
    executed, not just make -n'd: make test-sqlserver boots the container,
    creates the database, runs the whole repo suite (integration tests included)
    and ends with exactly the two upserts above red and nothing else.
  • scripts/wait-for-sql.sh, because nc -z lies on Docker Desktop: it connects
    to the published-port proxy the instant docker run -p returns. MySQL binds
    3306, runs its init scripts and then restarts — mysqladmin ping answers
    during that window, so the probe runs a real query against authorizer_test
    instead. This produced one spurious FAIL before it was fixed; ready time went
    6s → 11s.
  • TestDeleteUserCascadesAllUserOwnedRows failed on any second run against a
    persistent DB (fixed fixtures, no cleanup). Pre-existing — it fails
    identically on Postgres on main — now rerunnable.
  • A metacharacter case on testAnchoredSessionTokenDelete, which is what the
    ESCAPE change needs to be pinned by.

GORM's postgres driver reads these columns back from information_schema
as `bpchar`, which never prefix-matches the declared `char(36)`, so
AutoMigrate emits the same no-op type change on every boot after the
first:

  ALTER TABLE "authorizer_sessions" ALTER COLUMN "user_id"
    TYPE char(36) USING "user_id"::char(36)

On PostgreSQL that is a silent table rewrite under ACCESS EXCLUSIVE on
four tables at every restart. On CockroachDB it is fatal: ALTER COLUMN
TYPE is gated behind an experimental flag (SQLSTATE XCEXF), so the
server starts once and never starts again.

varchar(36) matches what the catalog reports, which is why none of the
existing varchar columns churn. Values are 36-char uuids either way.

Upgrade cost is one ALTER per column, on the first boot after this
change, and none after. Verified against postgres 18 with
log_statement=all: 5 alters on the upgrade boot, 0 on every boot after.
CockroachDB has been in the supported list and routed to the postgres
driver, but nothing ran the storage suite against it. It aborted on
startup, so the server came up once on a fresh database and never
again. Two independent causes, both in AutoMigrate:

- Cockroach lists UNIQUE *indexes* in information_schema
  .table_constraints; postgres lists only real constraints. GORM reads
  that row as a stale `unique` tag on every `uniqueIndex` field and
  issues DROP CONSTRAINT "uni_<table>_<col>" — a NamingStrategy name
  that never exists — so startup dies on SQLSTATE 42704.
- `ALTER TABLE ... DROP CONSTRAINT` is unimplemented there for UNIQUE
  (SQLSTATE 0A000), so the v1 legacy-uniqueness cleanup silently did
  nothing. That leaves UNIQUE on authorizer_otps.phone_number, where v2
  writes '' for every email OTP and the second one is rejected.

A cockroach-only dialector skips the bogus drop and retries the real
one as DROP INDEX ... CASCADE. Postgres and yugabyte keep the plain
driver.

`make test-cockroachdb` runs the storage suite against a single-node
container, and cockroachdb joins test-all-db and the SQL migration
tests. Verified green on a fresh database with cockroach v24.3.13;
docker pull was unavailable here, so the make target itself is only
make -n validated.
It used a fixed email and org id per backend and never deleted the user
it re-provisions at the end, so a second run against a database that
still holds the first run's rows failed at AddUser — on every SQL
backend, not just one. Nobody hit it because the make targets always
start a fresh container, but it makes a developer's local database a
one-shot and turns an unrelated earlier failure into a confusing
"user with given email already exists" on the next run.

Fixtures carry a uuid and the re-provisioned user is deleted through the
cascade the test just proved.
MySQL and MariaDB are in the supported list, but nothing ever ran the
storage suite against them and they could not create their schema at
all, let alone serve a request. Three defects, each verified fixed
against mysql:8.0.40 and mariadb:11:

- An unsized string column under `gorm:"uniqueIndex"` became `longtext`,
  which MySQL refuses to index: *Error 1170 (42000): BLOB/TEXT column
  'client_id' used in key specification without a key length*, so
  CREATE TABLE authorizer_clients failed on a brand new database. The
  driver already widens such a column to varchar(191) for `index` and
  `unique`, but checks only those two tags. The dialector now applies
  the same width to `uniqueIndex` — the 10 affected columns all live in
  tables that do not exist in v1, so nothing is narrowed under existing
  data, and three consecutive boots emit zero ALTER TABLE.

- MySQL reports COLUMN_KEY='UNI' for unique *indexes*, so AutoMigrate
  read every `uniqueIndex` field as a stale `unique` tag and tried to
  drop a constraint name that was never created: *Error 1091: Can't DROP
  'uni_authorizer_oauth_states_state_key'*. Same root cause as the
  CockroachDB abort, so the skip is now shared.

- `ESCAPE '\'` is a syntax error on MySQL — backslash escapes inside
  string literals, so the literal is unterminated (Error 1064) — and
  both session-token revocation statements carried it. Revocation
  therefore failed outright rather than revoking. The LIKE escape
  character is now `!`, which needs no quoting on any engine; the
  cross-backend anchored-delete test gained a metacharacter case that
  fails if the escape character itself stops being escaped.

Also fixes the CockroachDB wrapper committed earlier: it embedded the
gorm.Migrator *interface*, which drops the driver methods gorm
type-asserts without an ok-check. That panicked outright on MySQL
(`BuildIndexOptions`) and was latent on CockroachDB. Both wrappers now
embed the driver's concrete migrator, so every method survives.

Adds `make test-mysql` / `make test-mariadb` on separate ports, pulls
both plus libsql into test-all-db, and replaces the port probe with
scripts/wait-for-sql.sh for the engines whose port opens before they
serve queries (MySQL binds 3306, runs its init and then restarts;
CockroachDB accepts TCP before it accepts clients).
@lakhansamani lakhansamani changed the title fix(storage): make cockroachdb work with gorm AutoMigrate, and test it fix(storage): make cockroachdb, mysql, mariadb and sql server work with gorm AutoMigrate Oct 29, 2026
gorm.io/driver/sqlserver sizes an unsized string column to nvarchar(256)
when the field carries `index` or `unique`, but it checks only those two
tags. A field tagged `uniqueIndex` — named or composite — is missed and
becomes nvarchar(MAX), which SQL Server refuses as an index key, so
AutoMigrate dies on the first such table and the server never starts:

  mssql: Column 'client_id' in table 'authorizer_clients' is of a type
  that is invalid for use as a key column in an index.

Ten columns are affected, across clients, organizations,
org_memberships, federated_identities, scim_endpoints, trusted_issuers
and webauthn_credentials — all tables that arrived in 2.3.0-rc.10, so
nothing is narrowed under existing data.

Same shape as the mysql fix: wrap the driver's dialector, give an
unsized `uniqueIndex` string the width the driver already picks for the
tags it does recognise, and repoint the migrator's Config.Dialector so
DDL type decisions route through it. No MigrateColumnUnique override is
needed here — SQL Server reads column uniqueness from
INFORMATION_SCHEMA.TABLE_CONSTRAINTS, which lists real UNIQUE
constraints only, so catalog and models agree as they do on PostgreSQL.

Verified against mssql/server:2022-latest. A fresh install creates the
whole schema and the next two boots emit zero DDL. Both upgrade paths
that can carry data are green in one boot, with no churn after:

- v1-shaped (char(36) user_id, UNIQUE on users.email and
  otps.phone_number): rows intact, both legacy UNIQUEs dropped, two
  phone_number='' email OTPs insertable.
- 2.0.0 through 2.3.0, the releases that could still boot on SQL Server
  because their schema had no unsized `uniqueIndex` string:
  authenticators.user_id widens from char(36) to varchar(36) with the
  seeded row intact, and idx_authenticator_user_id_method is created
  over it afterwards.

From 2.3.0-rc.10 on, AutoMigrate could not complete on SQL Server. A
database touched by one of those boots has
authorizer_authenticators.user_id as char(36) *inside* that composite
index, and SQL Server will not retype an indexed column — drop it before
upgrading. No boot there ever finished, so it holds no data.

Two upsert tests stay red on a separate driver bug: MergeCreate builds
its MERGE predicate from the primary key and ignores
clause.OnConflict.Columns, so an upsert keyed on a composite unique
index inserts a second row instead of replacing one. make test-sqlserver
is therefore kept out of test-all-db.
@lakhansamani
lakhansamani force-pushed the fix/cockroachdb-automigrate branch from 1b9f389 to 1ec1116 Compare October 29, 2026 06:30
@lakhansamani
lakhansamani merged commit 3a5fc33 into main Oct 30, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant