fix: wrap bulk load serialization errors in InputError - #1772
fix: wrap bulk load serialization errors in InputError#1772arthurschreiber wants to merge 23 commits into
Conversation
In the RPC request payload, only errors thrown while generating a parameter's data were wrapped in an `InputError` naming the failing parameter. Errors thrown while generating the parameter's type info or length prefix (e.g. a `RangeError` from writing an out-of-range length) escaped as raw, unattributed errors instead. All three serialization steps now go through the same error wrapping. The downstream behavior is unchanged and remains sound: a payload error aborts the partially written request message by setting the packet status IGNORE bit (MS-TDS s2.2.3.1.2) and surfaces on the request's callback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Reviewed the diff. This is a small, well-targeted fix — feedback below. SummaryThe fix is correct and minimal: moving Correctness
Test coverage
Minor/optional observations (not blocking)
Nice, focused fix with good test coverage and a well-reasoned validation note on the wire-protocol side. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1772 +/- ##
==========================================
+ Coverage 82.50% 82.51% +0.01%
==========================================
Files 92 92
Lines 5035 5039 +4
Branches 961 961
==========================================
+ Hits 4154 4158 +4
Misses 592 592
Partials 289 289 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Apply the same error wrapping to bulk loads: errors thrown while generating a column's metadata previously escaped the row transform without any deliberate handling, and errors thrown while serializing row values surfaced raw and without naming the affected column. Both now surface as an `InputError` naming the column, with the original error as `cause`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewNice, focused fix — the RPC and bulk-load error paths now attribute failures to the parameter/column that caused them, and the new unit tests do a good job pinning down each of the three serialization steps independently. A few things worth a look: Inconsistency:
|
Restructure the error wrapping so that only the data type's serialization calls sit inside the try/catch blocks. Errors from stream internals (e.g. `Transform#push`) or from the consumer of the payload generator are no longer misattributed to the parameter or column being serialized. The RPC parameter data generator is iterated manually so that it keeps streaming values lazily (relevant for table-valued parameters, which yield many buffers); the bulk load path collects a single column value's buffers (a small, bounded set) before pushing them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
|
Reviewed the changes to A few observations, none blocking: 1. Asymmetric protection around const parameterData = type.generateParameterData(param, this.options)[Symbol.iterator]();
while (true) {
let result;
try {
result = parameterData.next();
} catch (error) { ... }The call parameterDataBuffers = [...c.type.generateParameterData(parameter, this.mainOptions)];In practice every real 2. Double-wrapping for TVP parameters
3. Minor test duplication The Other notes
Nice work on isolating exactly which calls needed the try/catch in the final commit — that's the trickiest part of this kind of change to get right. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 67fcfd9433
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
`generateParameterData` does not have to be implemented as a generator function - a plain function implementation performing synchronous setup before returning its iterator throws at call time, which the narrowed error wrapping no longer covered. Wrap the iterator construction as well, restoring the coverage the original `yield *` form had. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewOverall this is a solid, well-scoped fix — consistent 1.
|
…contract Splits parameter handling into two phases. `resolveParameter` validates the value and determines the declaration facts (length, precision, scale, collation) once, before a request is sent; `writeTypeInfo` and `writeValue` serialize the resolved parameter into a `WritableTrackingBuffer`. Types can implement `resolve`, `writeTypeInfo` and `writeValue` natively; the helpers adapt everything else from the existing `validate` / `resolve*` / `generate*` methods, so types can be migrated one at a time. Int, NVarChar and VarBinary are migrated. `Request.validateParameters` now resolves the request's parameters and keeps the result; the RPC payload takes resolved parameters and only serializes. `Connection.execSql`, `callProcedure`, `prepare`, `unprepare`, `execute` and the Always Encrypted metadata request build their payloads from resolved parameters. Bulk load writes column metadata and row values through the same helpers. Two behaviour changes come with the shared resolution: - Lengths are resolved for every type that can resolve one, not only for type ids matching the legacy variable-length bit pattern (the fix proposed in #1771). - Errors thrown while writing a parameter's TYPE_INFO are wrapped in the same `InputError` as errors from writing its value (the RPC half of #1772). A new unit test serializes 40 parameter cases across every type, on TDS 7.4 and 7.2, with and without a collation, through the new payload and through an inline copy of the previous serialization, and asserts the bytes are identical. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
Constraint names on temporary tables are unique per database, so the
named constraint collided when two CI jobs ran this test against the
same Azure database at the same time ("There is already an object named
'chk_id' in the database").
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
- `BulkLoad.addColumn` no longer gates length resolution on the legacy variable-length type id bit pattern, so the RPC and bulk load paths agree and #1771 is covered in full. - `resolveParameter` treats an explicitly specified length, precision or scale of 0 as specified instead of falling through to the type's resolver. Every existing resolver re-checked for an explicit value itself, so this changes no bytes for existing types; it removes the trap for a future type whose resolver does not. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
`resolveParameter` calls `type.validate(value, collation)` without the connection options, as every caller did before. Passing the options would activate the `useUTC`-dependent range checks in the date and time types, which have never run; that is a behaviour change to make on its own. A unit test pins the call shape. `Connection.resolveParameter` is renamed `resolveRequestParameter` so it is not misread as recursion into the free function it wraps. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
…quest The Always Encrypted metadata request is built from raw parameters rather than a `Request` that went through `validateParameters`, so the parameters are resolved inline. A comment says so. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
…ror-handling Narrow this change to the bulk load half. The RPC half (attributing TYPE_INFO and length-prefix errors to the parameter) is delivered by the parameter contract, which wraps every serialization step of a parameter in the same InputError, so the payload change and the test file exercising the legacy generate* path are dropped. The bulk load wrapping is re-expressed on top of the contract's writeTypeInfo and writeValue calls. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewReviewed the diff ( OverallClean, well-scoped change. It extends the same
Because Bugs / correctnessDidn't find any. I checked for:
Test coverageGood coverage for the new behavior:
One small gap: the "row values" unit test throws from API surface (minor, possibly out of scope for this PR)
Style nit
Security / performanceNo concerns — this only changes error attribution/typing on already-thrown synchronous errors; no new I/O, no change to wire behavior, and no user-controlled data flows into anything new (the column name interpolated into the message is a column identifier already sent over the wire in TDS COLMETADATA, not attacker-controlled free text beyond what a caller of Nice work — the scoping down from the original multi-commit attempt (only wrapping the type's serialization call, not stream/generator-consumer errors) is exactly the right call, and the PR description does a good job explaining that evolution. |
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewNicely scoped fix — the wrapping is applied precisely around the type's serialization call in both
A couple of minor things worth a look, non-blocking:
Test coverage looks solid: the three new unit tests cleanly isolate metadata-generation, value-length, and chunk-generation failure points using a minimal stub |
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
Review of #1772: wrap bulk load serialization errors in
|
Takes the bulk load serializer from #1779. The transform this branch had routed cells through writeValue is gone; the payload's loop routes them through writeValue instead, and COLMETADATA keeps going through writeTypeInfo. The wide-row unit test spies on writeValue now that VarBinary implements it natively. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
…ror-handling Takes the bulk load serializer from #1779 via the parameter-contract branch. The InputError wrapping this branch put around the transform's writeValue moves into the payload's loop; the wrapping around writeTypeInfo in getColMetaData is unchanged. COLMETADATA is now written inside the payload's try, so a TYPE_INFO failure closes the row source like a row failure does. The serialization error tests drive the payload instead of the transform. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewNicely scoped change. I read through Correctness
Minor observations (non-blocking)
Test coverageGood coverage of the three failure points (column metadata, length prefix, data generator) plus the source-cleanup behavior. The integration test update correctly asserts both the new Overall this looks correct, well-tested, and appropriately narrow — no issues that should block merging. |
Takes #1771, which removed the legacy-id gate on length resolution that this branch had already removed in resolveParameter and BulkLoad.addColumn; only its comment is new. Its three unit tests drove the old payload constructor and are covered by parameter-contract-test.ts and bulk-load-test.ts here. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
…ror-handling Brings in the merge of master (#1771) from the parameter-contract branch. No conflicts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
The fireTriggers integration test randomizes its table name to survive
parallel runs against one database, but created its trigger under a
fixed name. Trigger names are scoped to the schema, so two Azure CI
jobs running the test at the same time collided on it ("There is
already an object named 'bulkLoadTest' in the database"). The trigger
is now named after the table, like the CHECK constraint fix before it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
…ror-handling Takes the trigger-name fix for the fireTriggers integration test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewReviewed Code quality / correctness
Potential issue:
|
Bulk load and RPC callbacks hand an InputError to the caller for a parameter or column that could not be serialized, but the class was only reachable through lib/errors, so `err instanceof InputError` was not possible from outside the package. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewSolid, well-scoped change. The fix does what it says: Correctness
Minor / nit
Tests Security/Performance: no concerns — the added Nice, focused PR — the scoping decisions (leaving |
ReviewNice, focused change — the fix is small and the test coverage is thorough (unit tests for TYPE_INFO/length/data-generator failures, an explicit "row source is closed" test, and the integration test updated to match the new wrapping behavior). Correctness
Minor observations (non-blocking)
Security / performanceNo concerns — this only adds Test coverageSolid. The unit tests cover all three failure sites ( Overall: a clean, well-scoped fix with good test coverage. The only follow-up worth considering is whether the two wrap sites should have distinguishable messages. |
It was named after RowTransform's _transform, which no longer exists. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewReviewed the diff ( Correctness
API surface
Test coverage
Minor observations (non-blocking)
No security or performance concerns — this only changes error attribution on failure paths, not the wire format or serialization logic itself. Nice, minimal PR. |
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewWent through the diff ( Code quality / best practices
Potential bugs
Performance
Security
Test coverage
Note on behavior changeThe PR body is upfront that this is a deliberate change: bulk load callbacks that used to receive a raw Nice work overall - scoping this down from the original combined RPC+bulk-load PR into just the bulk-load half (with the RPC half already merged via #1774) keeps it reviewable, and the commit history shows the error-wrapping was iterated on carefully (narrowing what's caught, handling the non-generator-function edge case, etc.). |
The column metadata case had a test for this; the row value case relied on the validation-failure test exercising the same catch block. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug
ReviewReviewed the diff ( Correctness
Test coverageGood breadth: separate unit tests cover errors from Minor / non-blocking observations
SecurityNo concerns — this only changes error attribution/wrapping, not wire behavior or data handling. Error messages interpolate column names, which come from caller-supplied schema definitions, not untrusted external input. Overall: clean, well-tested, appropriately narrow fix. Nice work tracing through the async-iterator close semantics. |
Problem
Bulk load serialization errors are not attributed to the column that caused them. An error thrown while writing a column's TYPE_INFO in
getColMetaDataescapes with no deliberate handling, and an error thrown while serializing a row value (e.g. aRangeErrorfor an out-of-rangeDECIMAL) surfaces raw, without naming the column.RPC requests had the same gap for TYPE_INFO and length-prefix errors. That half was delivered by #1774 (now merged), which wraps every serialization step of a parameter in the
InputErrornaming it, so this PR is narrowed to bulk loads.Fix
Both bulk load serialization steps now go through the same wrapping:
Column '<name>' could not be serialized, anInputErrorwith the original error ascause, for column metadata (writeTypeInfo) and row values (writeValue). Only the type's serialization call is wrapped, so errors from the payload's own plumbing are not misattributed to the column. COLMETADATA is written inside the payload'stry, so a TYPE_INFO failure closes the row source the way a row failure does (#1779's ownership contract).InputErroris now exported from the package entry point alongsideConnectionErrorandRequestError.masteralready hands it to callers from the RPC and TVP paths, and this PR adds the bulk load, but it was only reachable throughlib/errors, soerr instanceof InputErrorwas not possible from outside the package.Deliberately out of scope: errors from
validatestill surface as thrown, unwrapped. Validation errors are a different failure mode (a bad value rather than a value the type cannot serialize) and already carry the type's own message; wrapping them is a separate decision.One deliberate behaviour change: bulk load serialization failures that previously surfaced the raw error on the bulk load's callback now surface the wrapping
InputError, with the original error available ascause. The existing integration test covering this path (should not throw in _transform function) is updated accordingly.Validation
The downstream recovery path is unchanged: a failed bulk load closes its row source and surfaces the error on the bulk load's callback as before. This change only affects error typing and attribution, not the wire behaviour.
New unit tests cover errors thrown from column metadata generation, from a row value's length prefix and from its data generator, and that a column metadata failure closes the row source. Unit suite (529 tests) and the bulk load integration suite (against SQL Server 2022) pass. Lint and typecheck clean.
Was stacked on #1774; with that merged, this PR targets
masterand its diff is the bulk load work alone (5 files).🤖 Generated with Claude Code
https://claude.ai/code/session_01Fxv5h4UMCGJEpjgCKcAxug