Repository navigation
Conversation
table.duplicate() and "sqlite-utils duplicate" used CREATE TABLE ... AS SELECT, so the copy lost the primary key, NOT NULL, UNIQUE, CHECK and DEFAULT constraints, foreign keys, STRICT, WITHOUT ROWID, indexes and rowid values, and INTEGER columns became INT. The copy is now created from the original CREATE TABLE statement with only the table name changed, and rows are copied along with their rowids. Indexes are recreated under new names, since index names must be unique. Self-referencing foreign keys and table.column references point at the copy, and the AUTOINCREMENT counter is carried over. Triggers are not copied. Virtual tables are still copied with CREATE TABLE ... AS SELECT. This is a behavior change: the copy now enforces the original table's constraints. Refs simonw#956
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes:
table.duplicate()andsqlite-utils duplicateusedCREATE TABLE ... AS SELECT, so the copy lost the primary key,NOT NULL,UNIQUE,CHECKandDEFAULTconstraints, foreign keys,STRICT,WITHOUT ROWID, indexes and rowids, andINTEGERcolumns becameINT. The documentation says the schema is copied.Behavior change
The copy now enforces the original table's constraints. Code that duplicated a table and then wrote rows into the copy that the original would reject (duplicate primary keys,
NULLin aNOT NULLcolumn, values failing aCHECK, dangling foreign keys withPRAGMA foreign_keyson) will now get anIntegrityError. Column types in the copy are the declared types (INTEGER, notINT).Change
The copy is created from the original table's
CREATE TABLEstatement with only the table name changed. Rows are then copied with theirrowidvalues (except forWITHOUT ROWIDtables). Generated columns are left out of theINSERTso the copy computes them itself.CREATE INDEXstatements under new names, since index names must be unique in a database. The original table name in the index name is replaced (idx_pets_namebecomesidx_pets_copy_name). Other index names are prefixed with the new table name. A numeric suffix is added if the name is taken.table.columnreferences (inCHECKconstraints, generated columns and partial or expression indexes) are rewritten to the new name. Without this the copy's foreign key would point at the original table, and the qualified references would makeCREATE TABLEfail withno such column.AUTOINCREMENTcounter is carried over, astransform()does sincesqlite-utils transformremoves theAUTOINCREMENTkeyword #602, so the copy does not reuse IDs deleted from the original.CREATE TABLE ... AS SELECT, as before.The renaming uses the existing lexer in
create_table_parser.py(newrename_create_statement(),rewrite_table_references()andparse_without_rowid()), so quoting, comments and string literals are left alone.The duplicate sections of
docs/python-api.rstanddocs/cli.rstnow say what is and is not copied.Tests
The existing
test_duplicateexpectedINTfor itsINTEGERcolumns. That was the bug:CREATE TABLE ... AS SELECTrewrites declared types to their affinity. It now expectsINTEGER.New tests in
tests/test_duplicate.pycover:CREATE TABLEstatement and enforced constraintsWITHOUT ROWID,STRICTand generated columnsAUTOINCREMENTcountertable.columnreferences and names that need quotingtests/test_cli.pyadds a test forsqlite-utils duplicate.The new tests fail on
mainand pass with this change. Locally the full suite passes (1511 passed, 16 skipped), and so do black, flake8, mypy, pyright,cog --check, codespell and a Sphinx build with-W. I also ran the suite on Python 3.10 against SQLite 3.23.1 and 3.46 built the waytest-sqlite-support.ymldoes, with no failures. TheSTRICTand generated column tests are skipped on 3.23.1.ty check sqlite_utilsreports oneredundant-castininsert_all()(db.py:4792onmain). It fails the same way onmain, so it is unrelated to this change.Written with help from Claude Code.
📚 Documentation preview 📚: https://sqlite-utils--957.org.readthedocs.build/en/957/