Repository navigation
Conversation
Both come from MyMovie's move to 2.6.0. F70, High: drop a table or a view by its name. Its migration drops a table whose class is gone, which DROP_IF_EXISTS can't name, so it borrows another table's withName. DROP_INDEX and DROP_TRIGGER already take names; the kind goes in the name, DROP_TABLE and DROP_VIEW, as a bare name can't tell which it is, and DROP TABLE fails on a view. F71, Low: reads that depend on a page in the PagingSource of F13, for a one-to-many relation that queries need as rows. MyMovie no longer needs it, having made its relation a converted column, and such a source can be written on the tokens of F66 today, so it waits for a user. The item records what the discussion settled: the reads are for a whole page and return an item for each row, their tokens join the page's, and they must not be skipped for what the page holds. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The guide to testing, in English and Chinese, is about testing against a real SQLite on each platform. What it says was measured first: - Robolectric 4.17 runs SQLite 3.32.2 at every SDK level up to 34, 3.44.3 at 36 and 3.50.6 at 37, so the level of @config chooses the framework code, not the SQLite version: a statement that needs a newer SQLite passes at sdk 24, and DROP COLUMN fails at sdk 34. What depends on the version needs an emulator or a device. - Without --add-exports=java.base/jdk.internal.access, the tests of SDK 36 and 37 fail on JDK 21, and those of 34 and lower pass. - Robolectric gives each test a data directory of its own, so a host test only has to close its database, where a JVM or native test has to delete the file too. - sqllin-processor added to kspJvmTest or kspAndroidHostTest generates the tables of the @DBRow classes of the test source sets, and only those. The code that uses such a table has to be in the platform's test source set: in commonTest it fails with an unresolved reference, as the table belongs to the platform's compilation. - observe in the test's own context, with runCurrent, shows that nothing is emitted, and PRAGMA_TABLE_INFO of a table that doesn't exist returns an empty list, which the recipe for a migration test checks with. Both ran as a test on the JVM. The guide to getting started now says that the createSQL of a table with a converted column is computed when it is read, so the generated code holds no CREATE statement to search for. The guide to advanced queries says that a page of pagingSource is one SELECT, what to do with a one-to-many relation, and not to skip its read for what a page holds, or its table is in no token. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A migration drops a table whose class is gone, for which there is no table object to give to DROP or DROP_IF_EXISTS. MyMovie's upgrade to its version 3 had to borrow another table's withName to name one. DROP_TABLE(name), DROP_TABLE_IF_EXISTS(name), DROP_VIEW(name) and DROP_VIEW_IF_EXISTS(name) take the name, as DROP_INDEX and DROP_TRIGGER do, and are experimental. The kind is in the function's name, as a bare name can't tell a table from a view, and SQLite's DROP TABLE fails for a view, and DROP VIEW for a table, with IF EXISTS too. A blank name is rejected as the statement is built, as withName rejects one, so that no statement of its scope runs. The statements are the ones DROP builds, so they are schema changes to the observed queries, as before. testDropByName covers a migration that drops a table and a view by their names, what doesn't exist, the other kind, an FTS table and blank names. sqllin-dsl-test passes jvmTest (118) and testAndroidHostTest (236, on Robolectric API 26 and 37); the native tests are compiled for macosArm64, linuxX64 and mingwX64, and not run, as this machine can't. The guide to modifying the database, in English and Chinese, describes them in a new section, and the guide to testing drops its table so. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
libs/linux/libsqlite3.a and libs/windows/libsqlite3.a, which the native tests on Linux and Windows link, were binaries in the repository, built on one machine, with nothing in the repository that told how. scripts/build-sqlite.sh builds one from the sources of a SQLite version that sqlite.org publishes, whose SHA-256 it checks, with zig, which compiles for both targets on any host. It keeps the options and targets of the libraries that were in libs: for SQLite 3.51.1 it gives their bytes, on this machine with the zig of Homebrew and with that of PyPI, which CI installs. It takes the autoconf sources, as sqlite.org has no amalgamation of some patch releases, as 3.18.2, and the sqlite3.c of both is the same file. The composite action .github/actions/sqlite restores the library from the cache, or builds it: a version's library only changes when the script or the action does, which the key holds, so neither zig nor the sources are downloaded otherwise. The Linux and Windows jobs build 3.51.1 with it. The libraries are removed and libs is ignored; whoever runs those tests on a Linux or Windows machine runs the script. .gitattributes keeps the script's line endings, as the checkout of a Windows runner would make them CRLF, which bash doesn't run. On Windows the tests now look for the library in libs before the directory of MSYS2, as they do on Linux before the system's, so that they link the SQLite that was built for them. What ran here: the script, for both targets and five versions. The workflow can only run in CI, which doesn't run on this branch. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
No test ran on a SQLite older than 3.32.2, which Robolectric has at every SDK level up to 34, while Android has 3.9 at API 24, SQLlin's minimum. The new job test-on-older-sqlite runs the native tests of sqllin-driver and sqllin-dsl on Linux, linked with SQLite 3.9.2, 3.18.2, 3.22.0 and 3.28.0, those of Android API 24, 26, 28 and 30, which the action of the commit before builds, and checks that the test binary holds that version. It tells what those versions take, not what Android's build of them or its framework does. The tests of what depends on the version now ask for it, as those of the newer PRAGMAs did: RENAME COLUMN (3.25.0), the tree of a query plan (3.24.0), the CHECK constraints that integrity_check verifies (3.18.0), a rename that rewrites the views that read the table (3.26.0, and a legacy one always before 3.25.0), and the date and time functions in an index (3.20.0), which the guide to SQL functions now says. This machine can't run a Linux binary, so what the job will meet was looked for on the JVM, with the SQLite of older releases of sqlite-jdbc, 3.14.2 to 3.28.0, which the tests don't run on otherwise. What is left there is those releases' own: their error messages, and functions that only they bundle. Six tests stop at such a message on 3.14.2, so the job of 3.9.2 may still find more. The Linux tests link 3.9.2 here. Two things found so that aren't fixed, and are marked KNOWN ISSUE where the tests go around them: - Before SQLite 3.20.0, Android 7.0 to 8.1, the result columns of a join that reads a view, or a derived table that SQLite flattens, are named with their tables, as person_with_id.name, so SQLlin, which looks them up as name, can't read the rows. testSubquery runs that join from 3.20.0 on. - An INSERT binds the values of all its rows in one statement, and a SQLite before 3.32.0, Android before API 31, takes 999 parameters by default. testPragmas and testMorePragmas insert their 300 rows 100 at a time. sqllin-dsl-test passes jvmTest (118) and testAndroidHostTest (236), and sqllin-driver its 13 and 27; the native tests are compiled. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The first run of the job of SQLite 3.9.2 failed in testMorePragmas: PRAGMA_COMPILE_OPTIONS() threw "This SQLite doesn't have PRAGMA compile_options." A PRAGMA that a SQLite may lack is checked by the name of its first column, as one that SQLite lacks returns no columns, and before SQLite 3.16.0 the column of compile_options is named compile_option. Measured with the SQLite of sqlite-jdbc: 3.8.11, 3.14.2 and 3.15.1 name it compile_option, and 3.16.1 and 3.18.0 compile_options; the source of 3.9.2 sets compile_option, and that of 3.18.2 the PRAGMA's name. The other PRAGMAs that SQLlin reads by name have the same columns in 3.9.2 as in 3.51.1. The check now takes a list of names, and compile_options both. It came with the PRAGMA in 2.6.0 (F30). Android's SQLite is built without the PRAGMA, which fails as before, and the Apple systems that SQLlin supports have newer versions, so it met a SQLite that an app links on Linux or Windows. On the JVM with sqlite-jdbc 3.15.1, testMorePragmas fails with the error of CI without the fix and passes with it. sqllin-dsl-test passes jvmTest (118) and testAndroidHostTest (236); the native tests are compiled. Whether the rest of testMorePragmas passes on 3.9.2, which the failure kept from running, the next run of CI tells. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An INSERT of entities bound the values of all its rows to one statement, and a SQLite before 3.32.0, as Android's before API 31, takes 999 parameters in a statement by default, so that 334 rows of 3 columns failed there with "too many SQL variables". The job of SQLite 3.9.2 would have met it in testPragmas, which inserts 300 rows of 4. Where the rows bind more than 999 values, the statement now runs as batches of rows that bind at most 999 each, whatever the SQLite: the limit is a compile-time option that a connection can lower, which not every platform can read, and the error has no code of its own to retry on. The batches run in a transaction, so that all the rows are inserted or none, as one statement does, nested in the scope's own; but those of INSERT OR FAIL, which keeps the rows a statement inserted before the one that fails, and so the batches before. The statement's SQL is still one statement of all the rows, which a trigger writes into its body, where nothing is bound, and which the log shows. A row is never split. An IN of a list binds its values too, and can't be split, as ORDER BY, LIMIT and aggregates would change. Its documentation, and the guide to advanced queries, now say that a list of more than 999 values fails before SQLite 3.32.0, and the roadmap has F72, Low, to write such a list into the SQL as literals. testInsertManyRows inserts 12,000 rows of 3 columns, 36,000 values, more than any SQLite takes by default, and 400 rows whose last conflicts, in the second batch: INSERT and INSERT_OR_ABORT insert none of them, INSERT_OR_FAIL keeps 399, INSERT_OR_IGNORE skips the last, and INSERT_OR_ROLLBACK in a transaction undoes the statement before too. Without the transaction the first batch's 333 rows stay, and with one around INSERT OR FAIL its 399 rows go, which the test fails on. On the JVM with sqlite-jdbc 3.14.2.1, whose SQLite takes 999 parameters, it fails with "too many SQL variables" without the fix and passes with it; newer releases of sqlite-jdbc raise the limit. testPragmas and testMorePragmas insert their 300 rows in one INSERT again. sqllin-dsl-test passes jvmTest (119) and testAndroidHostTest (238); the native tests are compiled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first part of 2.7.0, from
feature/new-features-2.7.0: a guide to testing, dropping a table or a view by its name (F70), and the native tests on older SQLite versions in CI, whose SQLite libraries CI now builds. Every item has its entry inCHANGELOG.md. This is the first run of the CI changes, which can't run on a feature branch.Roadmap and documentation
pagingSource, for a one-to-many relation, which no user needs yet.--add-exportsJVM argument is needed at SDK 36 and 37; each Robolectric test has a data directory of its own; and sqllin-processor added tokspJvmTestorkspAndroidHostTestgenerates the tables of the@DBRowclasses of the test source sets, which only the platform's test source set can use. It ends with a recipe for testing a migration.createSQLof a table with a converted column is computed when it's read; the guide to advanced queries, that a page ofpagingSourceis one SELECT, and what to do with a one-to-many relation (bf9c299).New feature
DROP_TABLE(name),DROP_TABLE_IF_EXISTS(name),DROP_VIEW(name)andDROP_VIEW_IF_EXISTS(name), experimental, so that a migration can drop a table or a view whose class is gone, asDROP_INDEX(name)andDROP_TRIGGER(name)do an index and a trigger. The kind is in the name, as SQLite'sDROP TABLEfails for a view. (7958510)Fixes
PRAGMA_COMPILE_OPTIONS()failed with "This SQLite doesn't have PRAGMA compile_options" on a SQLite before 3.16.0 that has the PRAGMA, whose column is namedcompile_optionthere, rather thancompile_options, by which SQLlin told whether it has it. It came with the PRAGMA in 2.6.0 (F30), and the first run of the job of SQLite 3.9.2 found it. Android's SQLite, built without the PRAGMA, still fails, as documented. (484f9ae)INSERTof entities that bind more than 999 values failed with "too many SQL variables" on a SQLite before 3.32.0, as on Android before API 31, as all the values were bound to one statement. It now runs in batches of at most 999 values, whatever the SQLite, in a transaction, so that it inserts all or none, as one statement does, butINSERT_OR_FAIL, which keeps the rows before the one that fails. Its SQL, which a trigger writes into its body, is still one statement. AnINof a list that long still fails there, which its documentation now says, and the roadmap has F72, Low, to write such a list as literals. (47f6641)CI
libs/linux/libsqlite3.aandlibs/windows/libsqlite3.aare removed.scripts/build-sqlite.shbuilds them from the sources of sqlite.org, whose SHA-256 it checks, with zig; for 3.51.1 it gives the bytes of the removed ones. The action.github/actions/sqlitecaches them, and builds them on a miss with the zig of PyPI. On Windows, the tests now link the library inlibsbefore that of MSYS2.test-on-older-sqliteruns those of sqllin-driver and sqllin-dsl on Linux, linked with SQLite 3.9.2, 3.18.2, 3.22.0 and 3.28.0, those of Android API 24, 26, 28 and 30, which no test ran on, and checks that the test binary holds that version. The tests of what depends on the version now ask for it: RENAME COLUMN, the tree of a query plan, the CHECK constraints that integrity_check verifies, a rename that rewrites the views, and the date and time functions in an index, which the guide to SQL functions now says.Known issue, not fixed here
The test goes around it, marked KNOWN ISSUE; whether and how to fix it is to be decided.
person_with_id.name, so SQLlin, which looks them up asname, can't read the rows.testSubqueryruns that join from 3.20.0 on.Testing
sqllin-dsl-testpassesjvmTest(119 tests) andtestAndroidHostTeston Robolectric API 26 and 37 (238), andsqllin-driverits JVM (13) and Android (27) tests.macosArm64,linuxX64andmingwX64, and the Linux test binaries are linked with SQLite 3.9.2.testMorePragmason B1. With B1,testMorePragmaspasses on the JVM with sqlite-jdbc 3.15.1, where it failed so without it; the rest of it, aftercompile_options, hasn't run on 3.9.2 yet.testInsertManyRowsof B2 fails with "too many SQL variables" without the fix on sqlite-jdbc 3.14.2.1, whose SQLite takes 999 parameters, and passes with it.🤖 Generated with Claude Code