Skip to content

Make SQL namespace creation atomic and report conflicts - #136

Merged
osipovartem merged 3 commits into
embucket-sync-df55.0.0from
fix/sql-namespace-duplicate-conflict
Oct 26, 2026
Merged

osipovartem merged 3 commits into
embucket-sync-df55.0.0from
fix/sql-namespace-duplicate-conflict

Conversation

@osipovartem

@osipovartem osipovartem commented Oct 26, 2026 •

Copy link
Copy Markdown

Why

SQL namespaces were represented only by property rows. Duplicate CREATE with disjoint property keys could succeed, empty namespaces required a synthetic property, and concurrent updates/drops could lose identity. SQL values were also interpolated into queries.

Change

  • Add a dedicated iceberg_namespaces primary-key row. CREATE writes identity and properties atomically and reports unique violations as CommitConflict.
  • Preserve property-only legacy namespaces. A small identity flag retains the old hidden exists marker without deleting user data; explicitly updated exists becomes visible.
  • Bind namespace, table, property, and metadata-location values in SQLx queries for SQLite, PostgreSQL, and MySQL.
  • Make namespace update/drop transactional. Lock the existing row or legacy property, avoid MySQL missing-row gap locks during promotion, and retry the whole transaction up to three times only for aborted deadlock/serialization SQLSTATEs.

Verification

  • Local SQLite catalog tests: 20/20 passed; Rustice draft #954 executor create_schema 17/17 and catalog if_not_exists_ 2/2 passed against the equivalent runtime code.
  • CI on the final head: Rust run 40266933328 passed lint and test. The SQL catalog job passed 23/23, including Docker-backed PostgreSQL and MySQL namespace-locking races and the existing full table integration test.
  • Local Rust 1.95 targeted Clippy with -D warnings, cargo fmt --all --check, and git diff --check passed.
  • Independent read-only code review approved the change, including a separate review of the final test-only lint fix.

Concurrent old-version writers should be quiesced during rollout. This migration reads old property-only catalogs but is not a mixed-version write protocol.

@osipovartem
osipovartem marked this pull request as draft October 26, 2026 03:25
@osipovartem
osipovartem marked this pull request as ready for review October 26, 2026 06:16
@osipovartem
osipovartem merged commit 7ac9615 into embucket-sync-df55.0.0 Oct 26, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant