Repository navigation
V69: a SQL result reaches the lab whole - #169
Merged
Merged
Conversation
The SQL console paints 200 rows, and its three ways out were built from those painted rows: a query returning 590 rows handed 200 to the ML Lab, and a NULL left as the mark the console shows for it, which turned a numeric column into text. The how-to page presents this handoff as the normal path. An export is now the query, not the screen. The engine writes the CSV itself with COPY, as it already did for Parquet; the CSV export, the lab handoff and a new Parquet button all go through it, on the query behind the result on screen. The lab's cell budget is applied before anything is built, and each query gets a file name of its own so the test-set ledger no longer takes two queries of the same shape for one file. Measured with the real engine: the 590 rows arrive, the column is numeric with missing values, and SELECT * over each of the six demos gives back the same rows, columns and inferred types as the file. Docs in both languages, README, PLAN row V69 and the nine remaining proposals (G2 to G10), version 1.69.0. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AyBsUet8WybTG986ubroar
dapiced
marked this pull request as ready for review
November 13, 2026 20:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
G1, the first of ten proposals from a new reading of the repository after V68. The how-to page presents « filter, join or aggregate in SQL, then send to the ML Lab » as the normal path. In the code that path was cut short.
The console paints 200 rows, which is sound. Its three ways out were built from those painted rows:
toLaband the CSV export senttableToCsv(result), a table already cut to 200. A note in the interface said so, but nobody could train on a SQL result of more than 200 rows, and the 118-row demo hid it.Measured before the fix
In the browser, with the real engine, on a query returning 590 rows with a NULL in a numeric column:
∅, the mark the console paints for it, and the lab took it for a category.The fix
An export is the query, not the screen.
SqlEngine.toCsvasks DuckDB for the file withCOPY (…) TO … (FORMAT CSV, HEADER), astoParquetalready did.query-3f2a91c4.csv, a fingerprint, never the text). Every handoff used to bequery.csv, and the test-set ledger took two queries of the same shape for one file. The FNV-1a function moved from the ledger tolib/fingerprint.tsinstead of being copied.Measured after
SELECT *over each of the lab's six demos gives back the same rows, the same columns and the same inferred types as the file loaded directly.Not in this wave
Making SQL read a file the way the rest of the app does (decimal comma, windows-1252, Excel, the cleaned or joined table): what leaves the console is complete, what enters it is still DuckDB's own reading, and the limits page says so. Carrying the recipe or the contract with the handoff. Raising the 200 painted rows.
Docs
« Hand a SQL result to the ML Lab » (new section on what leaves the console), Data Studio reference, limits, in both languages; README; PLAN row V69; changelog; version 1.69.0. The nine remaining proposals (G2 to G10) are now written in PLAN.md's « Proposed features » section.
Checks
xlsxtypes unreachable in my sandbox), format: clean.spreadsheet.test.ts(thexlsxtarball cannot be fetched in my sandbox; CI is the reference). 10 new tests; the panel's test now checks that the lab receives what the engine wrote, not the painted rows.lab.spec.ts:31, the Excel upload, same cause).e2e/sql.spec.tsnow runs the 590-row query with the real engine: CSV row count, no∅, a valid Parquet file, and the lab showing 590 rows with the NULL column read as numeric.Generated by Claude Code