Repository navigation
Add located root-array streaming with per-element source spans (1.8.0-dev) - #78
Merged
Merged
Conversation
Add JSONLocatedTopLevelArraySequence and JSONLocatedTopLevelArrayEvent to JSONStreaming for 1.8.0-dev. Each root-array element also reports its JSONSourceSpan: zero-based half-open UTF-8 byte offsets in the complete stream, excluding whitespace and separators, independent of chunking. Both root-array facades share the one internal JSONArrayStreamDriver, so grammar, quotas, errors, statistics and completion are identical; the value-only next() delegates to nextLocated(). No bytes are retained. Add public-consumer tests for spans, UTF-8, chunk splits, empty arrays, facade parity on errors/limits and unchanged value-only behavior. Update DocC, architecture notes, CHANGELOG, test discovery counts, the parameterized-case registry and the regenerated CLI source identity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an additive public JSONStreaming API for the next minor line (1.8.0-dev):
streamed root-array elements together with their original source location.
sourceSpanis a zero-based, half-open range of UTF-8 byte offsets in the complete stream.It covers the element token or subtree only; whitespace and separators are excluded. Offsets
come from the shared token machine, so they don't depend on how the source splits chunks.
documentCompleted, like the elements themselves. The readerkeeps no copy of the original bytes. A consumer that needs raw slices keeps its own immutable
source and applies the span to it.
JSONArrayStreamDrivergoes fromprivateto internal and gainselementStarttracking plusnextLocated(). The existingnext()delegates to it, so bothfacades share strict grammar, per-element and lifetime quotas, error values, statistics,
cancellation and the completion boundary.
JSONTopLevelArraySequencekeeps its public API andbehavior.
Why / consumer
Chat Library's ProviderExport01 (private app) vendors a fork of Swift-JSON
(
ProviderExport01/Vendor/Swift-JSON) only because it needs a source span for each record inlarge provider exports. A comparison against 1.7.1 found that this API is the fork's only private
delta. It depends on the internal stream driver, so it can't be built outside the package. With
this in a tagged 1.8.0, Chat Library can drop the vendored tree and depend on
.package(url: "https://github.com/ThorFuchs/Swift-JSON.git", exact: "1.8.0"). The code herematches that private candidate's API (
1e950354), with full DocC comments added.Overlap with other 1.8 work
codex/1.8-selection-main-qa) and Add bounded structural batches and read-only stores for 1.8 #71 (codex/1.8-batches-and-store) don't touchSources/JSONStreaming.CHANGELOG.md: both add## Unreleased — 1.8.0-dev. Merge the bullets under one heading.Documentation/Qualification/TestDiscovery.json: counts are additive.Scripts/testing_events.py: this PR adds one parameterized entry.Sources/JSONCLIEngine/CLIBuildIdentity.swift: regenerate withpython3 Scripts/generate_cli_identity.pyafter merging.CLIReleaseVersionstays at1.7.1. The version bump, tag and release are left to themaintainer.
Tests
New
Tests/JSONStreamingTests/LocatedTopLevelArrayTests.swift. It imports onlyJSONandJSONStreaming, so it tests as a public consumer. It adds 6 functions and 18 parameterized cases:tabs and CRLF, and separator spacing. Each input runs with 1-byte chunks and with a two-chunk
split at every byte position, and every slice re-parses to the yielded value
Cyrillic keys and combining sequences
[], with whitespace) yield only completion withrecords == 0JSONTopLevelArraySequenceon 18 valid and malformed inputs at every split: thesame
JSONFailure, indices, values, records and statistics, and no completion after a failuremaxDepth,totalRecords,nodes,arrayItems,chunkBytes), plusper-element quota reset and lifetime counters over a 10,000-element generated array
Inventories updated:
TestDiscovery.json(JSONStreamingTests 30→36 on native, portable and WASIlists; parameterized invocations 390→408 and WASI 353→371) and
Scripts/testing_events.py.Local results
Source:
de1fabb2c34ff7310dbf20165ecdbb474d4ef342, macOS 27 / SDK 27.2, Xcode toolchain Apple Swift 6.4.2(swiftlang-6.4.2.1.9), Swift language mode 6.
python3 Scripts/ci.py --lane nativeswift testboth ran 14 bundles with counts [10, 16, 16, 23, 34, 36, 47, 57, 79, 88, 97, 98, 190, 190] and 408 parameterized cases. CLI, ownership, Transform, schema-examples, DSL, navigation and qualification sub-gates also passed.source_inputs_unchanged: truepython3 Scripts/ci.py --lane docsHost note: the swiftly 6.4.0 toolchain on this machine was removed during the session. The lanes
above ran with
~/.swiftly/bintaken offPATH(Xcode/usr/bin/swift) and withSDKROOT=MacOSX27.2.sdk. Without that, the bareswiftcprobes inqualify_cli_ownership.pyandqualify_navigation.pypick a mismatched compiler or the Command Line Tools SDK 27.0. This is ahost setup problem, not caused by this change.
Not run:
safety,linux,linux-safety,wasmandarchivelanes, and performancemeasurement. The
LimitsMatrix.jsonaccounting-site line numbers forJSONTopLevelArraySequence.swiftwere already out of date before this change; no script checksthem, and they're left as they were.
Docs
JSONStreaming.md(overview paragraph and topics) andCompletionAndBackpressure.md(reader table row and a new "Locate root-array elements" section with an example).
Documentation/Architecture/Streaming.mdandAPI.md.CHANGELOG.md: a newUnreleased — 1.8.0-deventry.🤖 Generated with Claude Code