LibRed: ADO.NET metadata surface, and every stored query kind read and written - #302
Open
ChrisJollyAU wants to merge 6 commits into
Open
ChrisJollyAU wants to merge 6 commits into
ChrisJollyAU wants to merge 6 commits into
Conversation
ACE never forces the OS cache to disk - no FlushFileBuffers on a statement, on an explicit commit or at close - and opens the file without write-through, so its durability is the OS cache plus the commit-byte protocol. Its commit-sync settings change only whether a write is issued inside the statement or after it returns. Two orderings hold in every mode: a page's lock is released only after that page has been written, and the connection's own commit slot is written immediately before the first page of a batch and again after the last. An explicit transaction writes nothing until it commits, and growth writes the new page's last byte before the page. The transactions design note claimed a commit fsyncs twice; it now records the measured protocol, and what LibRed does against it. Also note that how aggressively freed long-value pages are reclaimed is an engine setting, RecycleLVs, which ships as 1 on ACE and 0 on Jet 4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GetSchema served nothing, a reader could not describe its own columns, and a command took only text. All three now answer from the catalog and the planner, in the shapes ACE's OLE DB provider uses - verified row for row against it over Northwind: tables, columns, indexes, primary and foreign keys, the four constraint rowsets, statistics, views and procedures all match, and the two ACE serves through neither call (procedure parameters, view column usage) follow the OLE DB shapes. TABLE_TYPE comes from MSysObjects.Flags, a view's columns from planning its query - so a passed-through column reports the stored column behind it, facets and all - and a table's logical indexes carry the foreign-key direction byte, which is what lets an incoming relationship stay hidden as ACE hides it. The reader implements IDbColumnSchemaGenerator and GetSchemaTable from the same description: each output column carries the stored column it came from, so an alias keeps the stored name, a joined column names the other table, and a computed one stands alone as read-only. Key and base information is reported whether or not KeyInfo was asked for, since the round trip that flag exists to avoid is one this provider never makes. A command now accepts StoredProcedure and TableDirect, executing the named query with its arguments bound by name. An Access PARAMETERS clause ends in a semicolon, so the batch splitter no longer breaks it away from the query it declares for - which is the form every stored parameterized query reads back as. CommandBehavior is honoured rather than ignored: SchemaOnly plans a statement for its shape and runs none of it (measured: ACE leaves an INSERT's table untouched under it), SingleRow stops at the row already buffered, and CloseConnection closes the connection with the reader. Describing needs the executor to build a plan that reads nothing, so every leaf yields no rows and no seek key, offset or count is evaluated - which is also why a stored procedure can be described without the parameter values nobody supplied. Reporting a view's columns goes through the same path, where it used to sort and buffer the whole of any view with an ORDER BY. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A stored action query was read back as executable SQL for only two of its kinds, and written for the same two. The rest - UPDATE, DELETE, make-table, and an append fed by a SELECT - were recognised, named and refused, although the engine could run every one of those statements already. They are recognised because Access stores them the way it stores a view: one row per source table, one per join condition, one for the WHERE, one per declared parameter. Only the action row and the meaning of the column rows differ - an assignment names its target column, qualified when the update runs over a join; an append names the column it fills; a DELETE keeps the verbatim `table.*` when the statement names one, and nothing at all when it does not. So the rebuild now shares the clause builders with the view path, and every kind reads back and runs by name. Writing them is the same arrangement in reverse, and what it writes is row for row what ACE writes for the same statement, object flags included, which is how each kind is checked. The MSysObjects flags are DAO's QueryDef.Type in the low byte - not the action row's own numbering, which differs. A declared parameter now survives the round trip. It could not before: its size never reached the type mapper, so Text(50) was stored as a memo, and its facets were dropped, so Access rendered Text(255) back. Those facets live in the parameter row's LvExtra - a length for text, precision and scale packed together for a decimal, nothing for the types that record neither, a sized binary included. Elsewhere in the row set LvExtra holds uninitialised bytes, so nothing reads or writes it there. The parser lowered PARAMETERS names in a SELECT and an INSERT but not in an UPDATE or a DELETE, which left a parameterized update comparing a column against itself. A procedure or view body with no FROM at all - `SELECT 1 AS n`, which Access stores and opens - threw NullReferenceException out of the decomposer, and would have been dropped silently on the way back in. Access stores it as any other query minus the table rows; so does LibRed now. ACE will open such a query but not use it as a source, on its own files as much as on ours. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The spec gave text's 1..255 limit but not what happens when DDL asks for more, and the bullet next to it is about an over-long value, which is a different refusal with a different message. Measured: VARCHAR(255) creates the column, VARCHAR(256) and anything above it fails the whole CREATE TABLE with "Size of field is too long" - not promoted to a memo, not clamped. LibRed refuses the same declarations at the same threshold and opens its message with ACE's wording, which is already covered by tests on both sides. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two-argument LTRIM/RTRIM, as SQL Server 2022 has them: the second argument is a SET of characters rather than a substring, so every leading (or trailing) character found in it is removed and the stripping stops at the first that is not. An empty set strips nothing and either argument Null gives Null. The one-argument form is untouched - still Access's, stripping the space and U+3000 alone - and Trim still takes only the one, since Access's own second argument does not exist. STRING_AGG turned out to be a second spelling of the LISTAGG already here, so it shares the implementation, DISTINCT, FILTER (WHERE …) and the window form. What differs is what each insists on: LISTAGG needs its WITHIN GROUP and lets the separator go, where STRING_AGG needs the separator and lets the order go - and without a WITHIN GROUP the values list in the order the rows arrive, which over a window is window order. The separator is written as a string, because it is the one value the whole group shares. The ACE arity test records LTrim/RTrim as taking one argument or two, the way it already records Log: ACE rejects the second, deliberately. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GetDataTypeName answered with the CLR type's name, so every text column read as "String" whether it was a Char, a VarChar or a LongText - while the same reader's GetColumnSchema and GetSchemaTable had the provider's name for it all along. It now gives that name, which is also the one the DataTypes collection and Columns.TYPE_NAME use, so one type has one name across the whole surface and the fixed/variable distinction survives. ACE answers this with the OLE DB spelling instead (DBTYPE_WVARCHAR where this says VarChar), but its own schema rowsets use these short names, so matching them is what keeps the provider consistent with itself. A result with nothing described behind it - a system-variable select - has no stored column to name and still falls back to the CLR name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three commits, all in LibRed, all verified against ACE on Northwind rather than against assumption.
Serve ADO.NET's metadata surface from the catalog
GetSchemaserved nothing, a reader could not describe its own columns, and a command took only text.GetSchemanorGetOleDbSchemaTable(procedure parameters, view column usage) follow the OLE DB shapes.TABLE_TYPEcomes fromMSysObjects.Flags; a view's columns come from planning its query, so a passed-through column reports the stored column behind it, facets and all.GetSchemaTable/IDbColumnSchemaGeneratorfrom the same description: each output column carries the stored column it came from, so an alias keeps the stored name, a joined column names the other table, and a computed one stands alone as read-only. Key and base information is reported whether or notKeyInfowas asked for — the round trip that flag exists to avoid is one this provider never makes.CommandType.StoredProcedureandTableDirect, executing the named query with arguments bound by name. An AccessPARAMETERSclause ends in a semicolon, so the batch splitter no longer breaks it away from the query it declares for — the form every stored parameterized query reads back as.CommandBehavioris honoured rather than ignored:SchemaOnlyplans a statement for its shape and runs none of it (measured: ACE leaves an INSERT's table untouched under it),SingleRowstops at the row already buffered,CloseConnectioncloses the connection with the reader. Describing needs a plan that reads nothing, so every leaf yields no rows and no seek key, offset or count is evaluated — which is also why a stored procedure can be described without parameter values nobody supplied. Reporting a view's columns goes through the same path, where it used to sort and buffer the whole of any view with anORDER BY.Read and write every stored query kind Access stores
UPDATE, DELETE, make-table and an append fed by a SELECT were recognised, named and refused, although the engine could already run every one of those statements. Access stores them the way it stores a view — one row per source table, one per join, one for the WHERE, one per declared parameter — and they differ only in the action row and in what the column rows mean. All four now read back as runnable SQL and are written, row for row as ACE writes them, object flags included.
Along the way:
Text(50)was stored as a memo; its facets were dropped, so Access renderedText(255)back. They live in the parameter row'sLvExtra— a length for text, precision and scale packed together for a decimal, nothing for the types that record neither (a sized binary included).PARAMETERSnames in a SELECT and an INSERT but not in an UPDATE or a DELETE, which left a parameterized update comparing a column against itself.FROMat all —SELECT 1 AS n, which Access stores and opens — threwNullReferenceExceptionout of the decomposer, and would have been dropped silently on the way back in.Record what ACE writes, and when, on a commit
Measured: ACE never forces the OS cache to disk, and its commit-sync settings change only whether a write is issued inside the statement or after it returns. The transactions design note recorded a commit fsyncing twice; it now records the protocol and what LibRed does against it.
Tests
LibRed.Ado.Tests111,LibRed.Engine.Tests4018,LibRed.Core.Tests639,LibRed.Engine.AccessTests517,LibRed.Core.AccessTests985,LibRed.EFCore.Tests15 — all passing, solution builds clean. New suites cover the schema collections, the reader's column provenance, and each stored query kind both ways: run in ACE and in LibRed on two copies of Northwind and compared, and written by LibRed then read and run by ACE.The grammar gained
updateStatement/deleteStatementas procedure bodies; the parser is regenerated fromAccessSql.g4with the committed script.🤖 Generated with Claude Code