Repository navigation
Give the templates module tests, and run them in the template smoketest - #6445
Draft
krisajenkins wants to merge 2 commits into
Draft
krisajenkins wants to merge 2 commits into
krisajenkins wants to merge 2 commits into
Conversation
krisajenkins
force-pushed
the
kris/tests-in-templates
branch
2 times, most recently
from
November 2, 2026 12:08
1fc3b1f to
2f9a145
Compare
Since 2.12, TypeScript and Rust modules can be unit tested in-process: the harness runs reducers, views and procedures against the real datastore in memory, with no server and no publish. The docs cover it at https://spacetimedb.com/docs/how-to/unit-testing, but nothing a new project is born with mentions it. That matters most for coding agents. `spacetime init` writes CLAUDE.md, AGENTS.md, .windsurfrules, .github/copilot-instructions.md and .cursor/rules/ by concatenating skills/*/SKILL.md, and none of those skills say a word about testing. Ask an agent working in a fresh template how to test the module and it has to go digging through node_modules to discover that `spacetimedb/server/test-utils` exists at all, or more likely it recommends publishing to a local server and poking it with `spacetime call`. The template itself is no help either: of the TypeScript templates only chat-react-ts ships a test, and basic-ts, whose module 13 other templates copy verbatim, ships no test runner, so the first test anyone writes starts with working out the Vitest plugin incantation. This adds a Testing section to the typescript-server and rust-server skills (setup, a minimal test, and the points that trip people up, such as withReducerTx committing if you swallow an error), a unit-test step to the concepts skill's feature checklist, and "unit-testing" to both skill descriptions so the skills trigger on testing questions. It also gives the basic templates a passing test suite that new projects inherit. - basic-ts/spacetimedb and the 13 templates sharing its module (angular, browser, bun, deno, nextjs, nodejs, nuxt, react, remix, solid, svelte, tanstack, vue) gain vitest.config.ts, src/index.test.ts, a `test` script, a vitest dev dependency and "type": "module". The dependency is vitest 4, which @angular/build requires as a peer, except in remix-ts, which stays on vitest 3 because vitest 4 needs Vite 6 and Remix is on Vite 5. - Each of those templates' top-level package.json, and chat-react-ts's, gets a `test` script that runs the module's tests with `vitest run --root spacetimedb`, plus vitest as a dev dependency, so `npm test` or `pnpm test` works from the project root after the single top-level install a new project already does. Nothing installs spacetimedb/'s own dependencies in a generated project, and Node resolves vitest and spacetimedb from the module's files by walking up to the top-level node_modules. chat-react-ts runs the module's unit tests before its client suite, whose integration test needs a server. - chat-react-ts's client vitest config now excludes spacetimedb/**. It was already picking up the module's tests without the SpacetimeDB plugin, and they failed. - The lockfile churn beyond the new vitest entries is pnpm re-resolving peers (@types/node) and bun-ts's `"@types/bun": "latest"`. - `spacetime init --lang typescript` embeds the two new files. Because `spacetime publish` type-checks the whole module, a project that replaces src/index.ts must also delete or rewrite src/index.test.ts. The TypeScript chat tutorial, which has the reader replace index.ts, now has them delete the test first, and the smoketest helpers that scaffold a module with `init` and overwrite index.ts (the shared TypeScript helper and the quickstart test) do the same. - chat-react-ts's and money-exchange-react-ts's vitest configs cast the React plugin to vitest's own Plugin type. Their vite and vitest's can resolve to different copies, whose plugin types don't match. - basic-rs gains a #[cfg(test)] module and a test-utils dev dependency, with the icu_calendar 2.1.x pin from the guide (#6317), without default features so it constrains the version without adding ixdtf. - nextjs-ts and remix-ts client tsconfigs glob **/*.ts, so they now exclude spacetimedb/ (a separate project whose tests import vitest from its own node_modules), and their module tsconfigs include vitest.config.ts so eslint's project service finds it. - The codex-plugin copy of skills/ is resynced.
The templates now ship tests, but nothing in CI runs them. The template smoketest initialises each template, publishes its module and installs its client, and stops there. Root `pnpm test` would reach the in-repo copies, but that CI step is commented out. So a template whose tests had broken would ship that way, and nobody would know until a user ran `npm test` in a fresh project. That has already happened: chat-react-ts's integration test hardcodes ws://localhost:3000 and a database called quickstart-chat, so it can only pass on someone's laptop, and it has never run anywhere that could tell us it doesn't. After publishing a TypeScript template's module, the smoketest now installs the generated project and runs its `test` script, preferring the project's own over the module's. It sets VITE_SPACETIMEDB_HOST and VITE_SPACETIMEDB_DB_NAME to the database it just published, and deletes the database after the tests rather than straight after publishing. - The smoketest jobs install wasm-bindgen to build the SDK's test datastore. They already run Node 22, which the module test harness needs (Symbol.dispose, Set.prototype.isSubsetOf). - build_typescript_sdk runs once per test run behind a file lock. Every TypeScript smoketest used to rebuild the SDK, and tsup's clean build wipes dist/ while other tests, running in parallel, are still using it. - A new build_typescript_sdk_test_utils also generates the test datastore, once per run in the same way. Only the template smoketest calls it, so the other TypeScript smoketests don't need wasm-bindgen. - chat-react-ts's integration test reads the same environment variables as main.tsx. It no longer reads a token from localStorage: jsdom's storage always starts empty, and on Node 25+ a built-in global that is undefined unless configured shadows it. It allows its name to appear more than once, since it shows in both the profile and the user list. - chat-react-ts's client vitest config dedupes React, which otherwise loads twice when the SDK is linked from a local checkout. - money-exchange-react-ts's existing client tests now run too.
krisajenkins
force-pushed
the
kris/tests-in-templates
branch
from
November 2, 2026 13:52
2f9a145 to
d201f03
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
Since 2.12, TypeScript and Rust modules can be unit tested in-process with the module test harness, but nothing a new project is born with mentions it. The skills that
spacetime initconcatenates into CLAUDE.md, AGENTS.md and friends say nothing about testing, so a coding agent in a fresh project either goes digging throughnode_modulesto findspacetimedb/server/test-utilsor falls back to publishing and poking withspacetime call. The templates don't help either: of the TypeScript templates only chat-react-ts ships a test, and basic-ts, whose module 13 other templates copy, has no test runner at all.And the tests the templates do ship never run in CI. The template smoketest stops after publishing the module and installing the client, and the root
pnpm teststep is commented out. chat-react-ts's integration test hardcodesws://localhost:3000and a database calledquickstart-chat, so it can only pass on someone's laptop, and nothing has ever told us.This PR has two commits:
testscript (vitest run --root spacetimedb) sonpm testworks after the one install a new project already does.spacetime init --lang typescriptembeds the new files.testscript against the database it just published (viaVITE_SPACETIMEDB_HOST/VITE_SPACETIMEDB_DB_NAME), deleting the database afterwards. The SDK build now runs once per run behind a file lock, since parallel tests were wiping each other'sdist/, and the template smoketest also generates the SDK's test datastore the same way. chat-react-ts's integration test reads the same env vars asmain.tsx.Because
spacetime init --lang typescriptnow scaffoldssrc/index.test.ts, andspacetime publishtype-checks the whole module, a project that replacesindex.tshas to delete or rewrite that test too. The TypeScript chat tutorial tells the reader to replaceindex.ts, so it now has them delete the test first, and the smoketest helpers that scaffold withinitand overwriteindex.tsdo the same.Each commit message has the full detail.
Needs a change in the private repo: its Internal Tests run the public template smoketests, which now build the SDK's test datastore and so need wasm-bindgen-cli 0.2.104 installed (as this PR adds to the public smoketest job). The other TypeScript smoketests don't need it.
Templates use vitest 4, except remix-ts, which stays on vitest 3 because vitest 4 needs Vite 6 and Remix is on Vite 5.
API and ABI breaking changes
None.
Rollback safety impact
n/a
Expected complexity level and risk
Testing
testscript against a freshly published database.initand replaceindex.tspass (quickstart, HTTP routes, views, index source names).emcc; .NET ILCompiler package won't restore on osx-arm64). Relying on CI.