From 07ed28095dde5e8063570aa6a2436039b5bd43ca Mon Sep 17 00:00:00 2001
From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com>
Date: Wed, 28 Oct 2026 14:35:51 +0100
Subject: [PATCH 1/6] Add cumulative inspection batches and immutable read-only
stores
---
AGENTS.md | 19 +-
CHANGELOG.md | 17 +
CONTRIBUTING.md | 9 +
Documentation/Architecture/API.md | 13 +
.../Architecture/ExtractionManifest.json | 11 +
.../BoundedWorkflows1.8.md | 63 ++++
.../SOURCE-AUTHORITY.md | 9 +
Documentation/Qualification/Release1.8.md | 96 ++++++
Documentation/Qualification/ReleaseLines.md | 10 +
.../Qualification/TestDiscovery.json | 36 +-
Documentation/README.md | 7 +
Documentation/ReleaseRoadmap.md | 10 +
Examples/BoundedWorkflows/Package.swift | 13 +
.../Sources/BoundedWorkflows/Main.swift | 50 +++
README.md | 78 +++--
Scripts/build_documentation_site.py | 6 +-
Scripts/ci.py | 50 ++-
Scripts/qualify_store.py | 2 +-
Scripts/qualify_structural_workflows.py | 77 +++++
Scripts/test_qualification_harness.py | 37 ++-
Sources/JSONCLIEngine/CLIBuildIdentity.swift | 2 +-
Sources/JSONCLIEngine/CLIReleaseVersion.swift | 2 +-
.../Documentation.docc/BoundedWorkflows.md | 73 +++++
.../Documentation.docc/JSONInspection.md | 13 +
.../JSONInspection/JSONInspectionBatch.swift | 153 +++++++++
.../Store/JSONConfiguredStore.swift | 133 ++++++--
.../Store/JSONStoreProfile.swift | 256 +++++++++++----
.../Store/JSONStoreQueries.swift | 234 +++++++++++++
.../JSONNavigation/JSONTraversalContext.swift | 15 +
Tests/JSONCLITests/IntegrationTests.swift | 2 +-
Tests/JSONInspectionTests/WorkflowTests.swift | 309 ++++++++++++++++++
31 files changed, 1647 insertions(+), 158 deletions(-)
create mode 100644 Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
create mode 100644 Documentation/Qualification/Release1.8.md
create mode 100644 Examples/BoundedWorkflows/Package.swift
create mode 100644 Examples/BoundedWorkflows/Sources/BoundedWorkflows/Main.swift
create mode 100644 Scripts/qualify_structural_workflows.py
create mode 100644 Sources/JSONInspection/Documentation.docc/BoundedWorkflows.md
create mode 100644 Sources/JSONInspection/JSONInspectionBatch.swift
create mode 100644 Sources/JSONInspection/Store/JSONStoreQueries.swift
create mode 100644 Tests/JSONInspectionTests/WorkflowTests.swift
diff --git a/AGENTS.md b/AGENTS.md
index b0ca32d..88f7f62 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,9 +1,9 @@
# Contributor instructions
-Prepared release source is 1.7.0; publication is a separate owner action. Its
-parent development line is 1.7.0-dev and stable 1.6.1 remains separate. This is
-not a maintenance release or the frozen 6fe8067 RC. Preserve all source-bound
-qualification history; source integration never publishes a version.
+Development source is 1.8.0-dev, based on published 1.7.0 at
+8557badef562cf2f059e964c113d4a7f60ecbe6c. Stable 1.6.1 remains a separate
+maintenance line. Preserve every source-bound qualification record. No 1.8
+merge/tag/publication follows from implementation or passing tests.
Swift-JSON requires Swift tools/compiler 6.4+, Swift language mode 6 and
macOS/SDK 27+. Read README.md, CONTRIBUTING.md and the relevant contracts in
@@ -50,10 +50,13 @@ Scalar scanning and immutable plan-local regex sharing are selected private
optimizations. Source-attached N3 snapshots and P1 tree/sparse-table projections live in JSONInspection.
NAV10 set/matrix and NAV11 directed-simple-graph APIs are development capabilities
inside JSONInspection; no graph product, mutation or serializer is introduced.
-Private NAV12 store construction/preparation remains package-only and shares one
-original invocation ledger; no public store/follow API or persistence is exposed.
-Wider retrieval/persistence, NAV13 batches and JSON text-sequence framing remain
-separate, unadmitted work.
+The 1.8 candidate promotes the checked immutable configured store inside
+JSONInspection and adds ordered path/snapshot/store value batches, exact-ID lookup
+and bounded complete BFS follow. Preserve the original construction phases and
+one invocation-owned ledger per operation. Positions remain unforgeable and
+source-instance-local. No persistence or mutable store is exposed. See
+Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md.
+Wider retrieval/persistence and JSON text-sequence framing remain separate work.
JSONTransform uses one private compiler/VM and the CLI's accounted adapter.
Preserve its accepted 42 authored declarations and exact semantics. The original
diff --git a/CHANGELOG.md b/CHANGELOG.md
index dfe6eb8..4f4f047 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,22 @@
# Changelog
+## Unreleased — 1.8.0-dev
+
+- Add ordered typed-path, snapshot-ordinal and store-position value batches using
+ one cumulative native ledger per batch, complete source admission and atomic results.
+- Promote the original checked configured-store profile/construction inside
+ JSONInspection; add exact-ID lookup and finite complete BFS reference following.
+- Preserve exact numbers/scalar identity, unforgeable source-local positions,
+ explicit absent/null/link states and original construction error precedence.
+- Add external SwiftPM examples, ordinary compiler refusals, boundary/lifecycle/
+ concurrency tests and exact scoped CI discovery. Qualification remains source-bound;
+ no 1.8 release or performance/platform pass is implied.
+
+## Publication note — 28 October 2026
+
+Version 1.7.0 was published from 8557badef562cf2f059e964c113d4a7f60ecbe6c.
+The preparation records below retain their original historical wording.
+
## 1.7.0 — prepared source; publication separate
- Reconcile admitted set/matrix APIs with the reviewed public directed graph,
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index bfa44f0..ba2262c 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -1,3 +1,12 @@
+# 1.8 scoped workflow qualification
+
+This isolated development source is 1.8.0-dev from published 1.7.0.
+Run `python3 Scripts/ci.py --lane structural-workflows` for exact Debug/Release
+behavior, preserved store cases and public compiler/consumer checks. Run
+`--lane structural-workflows-safety` for scoped ASan, shared-state TSan and strict
+memory checks; complete native/Linux/docs/WASI gates still apply before release.
+Do not infer a pass when storage or an actual supported runtime is unavailable.
+
# Contributing
Choose the source line deliberately: maintenance changes use the 1.6 line;
diff --git a/Documentation/Architecture/API.md b/Documentation/Architecture/API.md
index c8e167f..55b2bcd 100644
--- a/Documentation/Architecture/API.md
+++ b/Documentation/Architecture/API.md
@@ -1,3 +1,16 @@
+# 1.8 additive API candidate
+
+The latest released source is 1.7.0 (8557bade); this feature source is 1.8.0-dev.
+No product or dependency is added. JSONInspection now includes
+`JSONInspectionBatch`, snapshot ordinal batches, public `JSONStoreBuilder`/
+`JSONConfiguredStore` and typed profile/metadata/error declarations, exact-ID
+lookup, position value batches and finite complete BFS follow.
+See [the exact behavior/ownership boundary](StructuralIntelligence/BoundedWorkflows1.8.md)
+and [qualification](../Qualification/Release1.8.md).
+Navigation public signatures stay unchanged; its new selection capability is
+package-only and uses the original admitted scope. Earlier inventories below
+remain historical; compiler graphs must establish the final additive surface.
+
# Current API inventory
Unreleased 1.7.0-dev consolidation has twelve library products plus swift-json and
diff --git a/Documentation/Architecture/ExtractionManifest.json b/Documentation/Architecture/ExtractionManifest.json
index e174b41..4e48765 100644
--- a/Documentation/Architecture/ExtractionManifest.json
+++ b/Documentation/Architecture/ExtractionManifest.json
@@ -355,6 +355,17 @@
"attribution": "Apache-2.0 prototype SPDX retained; original integration cases extend actual-engine qualification.",
"validation": "Scripts/ci.py --lane dsl; native and safety exact discovery",
"evidence": "Actual-engine Debug/Release/ASan receipts and real public consumer"
+ },
+ {
+ "component": "1.8 bounded structural workflows",
+ "source": "Original Swift-JSON implementation over the existing native prepared traversal and NAV12 checked store construction",
+ "paths": [
+ "Sources/JSONInspection/JSONInspectionBatch.swift",
+ "Sources/JSONInspection/Store/JSONStoreQueries.swift",
+ "Examples/BoundedWorkflows"
+ ],
+ "validation": "Exact Debug/Release behavior, preserved 65 store vectors, public SwiftPM consumer, codegen negatives and safety/docs/platform gates; see Documentation/Qualification/Release1.8.md for actual status",
+ "dependencies": "No new dependencies or numeric/key/schema authority; Apache-2.0 original contributions"
}
],
"original_implementation_notes": [
diff --git a/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md b/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
new file mode 100644
index 0000000..a22601d
--- /dev/null
+++ b/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
@@ -0,0 +1,63 @@
+# 1.8 bounded structural workflows
+
+Implementation contract for the maintainer-authorized 1.8 feature branch from
+published1.7.0 (8557bade). This is an additive implementation, not merge or release
+approval. Original NAV12 construction fixtures, failures and qualification retain
+their source-bound historical meaning; public promotion adds its own evidence.
+
+## Batch reads
+
+JSONInspectionBatch returns exact values for ordered typed paths. Snapshot values
+accept ordered snapshot-local ordinals. Store values accept owner-bound positions.
+Repeats are preserved and charged; empty requests still admit the complete source.
+An invocation returns all results or throws, never publishes a partial result and
+never substitutes null for an invalid selection. Explicit JSON null remains a value.
+
+Each batch creates one original native read/traversal scope. Profile/request work,
+one complete canonical source/source-map admission, every selection and output
+slot share its ledger. It does not loop over independently admitted public reads.
+Request counts are inclusively capped (default1024); native path/work/retention/
+cancellation caps remain. Output JSON aliases follow existing requiredValue/read
+semantics; subsequent caller copies/work are outside library-owned quotas. Logical
+slot accounting is conservative policy, not heap/RSS or throughput measurement.
+
+## Immutable public configured store
+
+Promote the existing typed closed construction profile and checked store facade
+inside JSONInspection; no new product, parser, database, resolver or index engine.
+References stay disabled unless explicitly configured. Strict markers retain exact
+collection/id spelling; only declared fields are interpreted. Preserve all eight
+construction phases, duplicate/error precedence, first/second locations, source
+spans, source-local owner tokens and the original one-ledger preparation/admission.
+The twelve existing construction ceilings remain finite policies, not calibrated
+joint capacity guarantees; the49 core metrics and all fixtures remain unchanged.
+
+Fixed record/collection/reference metadata access is trusted bounded-memory access.
+Lookup, value batches and follow are separately reported operations, independent
+of construction. Each admits the captured source once and reserves its immutable
+metadata aliases; helpers cannot reset budgets or attach another source. Lookup
+uses decoded scalar names/IDs, returns nil for a missing ID in a known collection,
+and throws unknownCollection for an undeclared collection. Position construction
+and the fully admitted store initializer remain inaccessible to external callers.
+
+Follow is deterministic breadth-first over configured link slots in field order,
+including its starting position exactly once. Absent/null slots remain distinct
+metadata and contribute no edge. Cycles use visited physical positions, not string
+normalization or recursive callbacks. Depth, frontier, visited and result ceilings
+are inclusive and finite. An unvisited reachable node exceeding any ceiling throws;
+no depth-truncated or partially successful closure is returned. Owner mismatch is
+an error. This operation performs no I/O and creates no persistent identity.
+
+## Compatibility and qualification
+
+Existing singular reads/traversal/graph/Transform semantics and public signatures
+remain unchanged. A caller pipeline is not implicitly one global ledger. New
+source needs actual Debug/Release, public consumer and compiler-boundary evidence,
+cancellation/late-failure/resource cases, supported safety/platform lanes and DocC.
+Update exact discovery identities/cardinalities rather than relaxing them. Reuse
+original independent fixtures and49 caps; add explicit new batch/follow policy
+cases. Keep partial profiling/G4 HOLD and the1.7-only P4 exception historical;
+1.8 performance is unqualified until measured or explicitly dispositioned.
+
+No persistence, public mutable store, online lookup, new query grammar, lower
+platform floor, parallel batch execution or wider retrieval belongs to this wave.
diff --git a/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md b/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md
index a7011ae..5d1582f 100644
--- a/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md
+++ b/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md
@@ -1,3 +1,12 @@
+# 1.8 source authority — 28 October 2026
+
+The 1.8 candidate starts at published 1.7.0 commit
+`8557badef562cf2f059e964c113d4a7f60ecbe6c`, tree
+`c914b8914c9130a3a07a3ed82fb41dde33c37402`. The maintainer commissioned
+bounded batches and the immutable public store; see [the contract](BoundedWorkflows1.8.md).
+Earlier private-only/pending status remains truthful for the dated sources below.
+Qualification and merge/release acceptance of this successor remain separate.
+
# Structural source authority
## Consolidated development edition — 25 October 2026
diff --git a/Documentation/Qualification/Release1.8.md b/Documentation/Qualification/Release1.8.md
new file mode 100644
index 0000000..9768c20
--- /dev/null
+++ b/Documentation/Qualification/Release1.8.md
@@ -0,0 +1,96 @@
+# 1.8 bounded structural workflow qualification
+
+Status: implemented local milestone, broader release qualification PARTIAL, on
+isolated `codex/1.8-batches-and-store`.
+No merge/tag/release/publication is commissioned or claimed. Source base is
+published 1.7.0 `8557badef562cf2f059e964c113d4a7f60ecbe6c`, tree
+`c914b8914c9130a3a07a3ed82fb41dde33c37402`.
+
+The [contract](../Architecture/StructuralIntelligence/BoundedWorkflows1.8.md)
+and [compiled public example](../../Examples/BoundedWorkflows) define this additive
+wave. Twelve library products plus CLI, thirteen catalogs, empty root dependencies,
+Swift6.4/language6 and Apple27 minima remain. Original NAV12 fixtures and49 metric
+policies are unchanged. Current discovery adds24 named nonparameterized behavior
+functions and one shared-state function to JSONInspection (214 roots,11 shared).
+Original390 native/portable and353 WASI parameter invocations are unchanged.
+
+## Milestone evidence
+
+Private raw receipts stay in ignored `Artifacts/Release1.8` in the feature worktree.
+Initial compiler reconciliation passed `swift build --target JSONInspection --jobs 2`
+and20 then22 targeted native behavior functions. Those were unfrozen-source probes,
+not final qualification. The independent sibling review requested native path-limit
+admission on store reads and deterministic inner-loop cancellation. Both are
+addressed in source; the final24 cases also pin nested snapshot scratch cleanup,
+metadata aliases, self-cycles and narrow chains. Final gates must record exact
+source manifests/commands/environments/events; no index entry implies a pass.
+
+## Completed local milestone — 28 October 2026
+
+Apple Swift6.4.2, Xcode27.2, macOS/SDK27 supported native preflight passed.
+`Scripts/ci.py --lane structural-workflows` passed:24 exact new functions in
+Debug and Release, original12 store functions with65 native-build parameter
+invocations plus the separate strict-parser case in each configuration, preserved
+26-command store compiler/consumer receipt, and12 public SwiftPM witnesses per
+configuration. The new public qualifier also compiles the DocC example and six
+ordinary public-boundary refusals per configuration. Policy/compile-only vectors
+remain distinct from native evaluated outcomes.
+
+`--lane structural-workflows-safety` passed24 ASan functions, one shared-state
+TSan function and strict-memory compilation. Supplemental exact-event regression
+passed all57 Navigation and214 Inspection functions in Debug/Release. The existing
+CLI version case passed. The harness passed28 checks; site regressions passed12.
+The affected Inspection DocC catalog compiled with warnings as errors. Compiled
+public graphs contain301 authored Inspection declarations (139 added, none removed)
+and56 Navigation declarations (unchanged), all documented. Three legacy Self versus
+concrete-owner graph renderings were explicitly compared against unchanged owning
+source files; no byte-identical graph claim is made. Full13-module extraction remains
+pending. The independent sibling source review closed its path-limit/cancellation
+findings; this is scoped agent review, not human release approval.
+
+Preserve the original failed external-consumer attempt: its fixture placed a null
+field outside the configured collections object and was correctly refused as an
+unconfigured collection. The harness trapped on that uncaught error; it was not a
+library/compiler crash. The successor scopes collections explicitly and passes.
+Original attempt logs and source manifests remain in private evidence.
+
+This status update changes only this qualification document after the passing
+runtime receipts. Final checkpoint evidence must show that all product, test,
+fixture, DocC and harness bytes match those receipts, and record the documentation
+successor separately. No broader qualification pass follows from these local results.
+
+## Required gates
+
+- Scoped Debug/Release: `Scripts/ci.py --lane structural-workflows`, including
+ original12 store functions/65 vectors, ordinary compiler refusals, public
+ SwiftPM consumer and compiled DocC example.
+- Scoped safety: `--lane structural-workflows-safety` (ASan24, TSan1, strict memory).
+- Complete native, safety and docs lanes, current public API graphs and no
+ removed public declarations; preserve Navigation's original56 and Transform42.
+- ARM64/x86_64 Linux, supported WASI and Apple SDK/runtime scopes on the final
+ source. Unavailable environments/storage remain blocked, not inherited passes.
+- Clean source inventory, deterministic archive and Git-free external consumer.
+- Measured capacity/performance before any improvement claim. Original1.7 P4
+ waiver, loaded-host timing, partial Instruments profiles, streaming disposition
+ and graph G4 HOLD remain historical and cannot qualify changed1.8 source.
+
+The maintainer chose to continue implementation without reclaiming other
+sessions' files. At kickoff about1.4GiB was free; fresh broad safety/docs/platform
+builds may remain storage-blocked. Record actual attempts and unrun commands in
+the private milestone report. No cleanup, weaker check or gate waiver is implied.
+
+## Read-only store decision
+
+The public surface promotes typed configuration, immutable admitted store/metadata,
+owner-bound positions and typed failures. New operations are exact-ID lookup,
+ordered value reads and complete finite BFS follow. Construction uses the original
+eight phases, one exclusive ledger, duplicate timing and marker/target precedence.
+The prior compiler-negative asserting profile invisibility is superseded by an
+external private-read-scope refusal; the other original seven negatives and all
+65 original vectors remain. Store/position forging is still forbidden. This is
+an intentional additive promotion; it adds no product or runtime dependency.
+
+Fixed metadata access and supplied callbacks are trusted caller work. Each later
+query is independent of construction and other queries. Logical retained bytes
+are conservative owned reservations, not process memory. No persistence, mutable
+store, online resolution, asynchronous batch scheduler or durable token exists.
diff --git a/Documentation/Qualification/ReleaseLines.md b/Documentation/Qualification/ReleaseLines.md
index 40adade..c575737 100644
--- a/Documentation/Qualification/ReleaseLines.md
+++ b/Documentation/Qualification/ReleaseLines.md
@@ -1,3 +1,13 @@
+# Current source lines — 28 October 2026
+
+Latest published stable is [1.7.0](https://github.com/ThorFuchs/Swift-JSON/releases/tag/1.7.0),
+commit `8557badef562cf2f059e964c113d4a7f60ecbe6c`, tree
+`c914b8914c9130a3a07a3ed82fb41dde33c37402`: twelve libraries plus CLI,
+thirteen catalogs, 1,500 authored declarations. The isolated 1.8.0-dev successor
+keeps those products and adds bounded Inspection batches/store operations.
+See [its separate qualification](Release1.8.md). 1.6.1 stays the eight-library
+maintenance line. The dated preparation inventories below remain history.
+
# Stable and development source lines
| Line | Source identity | Public scope |
diff --git a/Documentation/Qualification/TestDiscovery.json b/Documentation/Qualification/TestDiscovery.json
index ee42d25..646ae2f 100644
--- a/Documentation/Qualification/TestDiscovery.json
+++ b/Documentation/Qualification/TestDiscovery.json
@@ -12,7 +12,7 @@
"JSONCLITests": 88,
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
- "JSONInspectionTests": 190,
+ "JSONInspectionTests": 214,
"JSONTransformTests": 23,
"JSONDSLTests": 47
},
@@ -37,7 +37,7 @@
"JSONCLITests": 1,
"JSONNavigationTests": 2,
"JSONSchemaExamplesTests": 4,
- "JSONInspectionTests": 10,
+ "JSONInspectionTests": 11,
"JSONTransformTests": 1,
"JSONDSLTests": 1
},
@@ -84,7 +84,7 @@
"JSONCLITests": 88,
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
- "JSONInspectionTests": 190,
+ "JSONInspectionTests": 214,
"JSONTransformTests": 23,
"JSONDSLTests": 47
},
@@ -99,7 +99,7 @@
"JSONBridgesTests": 10,
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
- "JSONInspectionTests": 190,
+ "JSONInspectionTests": 214,
"JSONTransformTests": 23,
"JSONDSLTests": 47
},
@@ -115,7 +115,7 @@
"JSONCLITests": 1,
"JSONNavigationTests": 2,
"JSONSchemaExamplesTests": 4,
- "JSONInspectionTests": 10,
+ "JSONInspectionTests": 11,
"JSONTransformTests": 1,
"JSONDSLTests": 1
},
@@ -148,5 +148,31 @@
"JSONInspectionTests.storeR08DeclarationAndPhysicalPermutationMetamorphisms()",
"JSONInspectionTests.storeR09CompleteRowsDeepSourcesAndZeroCapacity()",
"JSONInspectionTests.storeR10NoInferenceOrModelMutationAndExactPayloads()"
+ ],
+ "workflow_function_ids": [
+ "JSONInspectionTests.workflowB01PathsPreserveOrderRepeatsNullAndExactScalars()",
+ "JSONInspectionTests.workflowB02OneCompleteAdmissionAndEmptyBatch()",
+ "JSONInspectionTests.workflowB03RequestCapacityConfigurationAndExactBoundary()",
+ "JSONInspectionTests.workflowB04MissingAndWrongKindRetainTypedPaths()",
+ "JSONInspectionTests.workflowB05UnselectedSuffixAndCumulativeWorkCannotReset()",
+ "JSONInspectionTests.workflowB06SnapshotBatchKeepsOneSourceAndOrdinalMeaning()",
+ "JSONInspectionTests.workflowB07InnerCancellationInvalidatesOriginalScope()",
+ "JSONInspectionTests.workflowB08RetentionBoundaryIsFiniteAndSourceIsUnchanged()",
+ "JSONInspectionTests.workflowS01PublicStorePreservesPhysicalAndDeclaredOrder()",
+ "JSONInspectionTests.workflowS02LookupUsesDecodedScalarIDsAndDistinguishesMissing()",
+ "JSONInspectionTests.workflowS03OwnerTokensCannotRebindAnotherInstance()",
+ "JSONInspectionTests.workflowS04StoreBatchValuesPreserveOrderExactnessAndOneAdmission()",
+ "JSONInspectionTests.workflowS05ReferenceSlotsKeepAbsentNullAndLinkDistinct()",
+ "JSONInspectionTests.workflowS06FollowCompletesCyclesAndPreservesBFSOrder()",
+ "JSONInspectionTests.workflowS07FollowAllFourInclusiveBoundsThrowInsteadOfTruncate()",
+ "JSONInspectionTests.workflowS08FollowAndLookupHaveCumulativeQuotaAndNoSecondAdmission()",
+ "JSONInspectionTests.workflowS09FollowInnerCancellationAndInvalidation()",
+ "JSONInspectionTests.workflowS10SourceReplacementCannotInventSpansOrRebindPositions()",
+ "JSONInspectionTests.workflowS11DisabledReferencesRemainOrdinaryData()",
+ "JSONInspectionTests.workflowSharedImmutablePlansHaveIndependentBatchAndStoreHealth()",
+ "JSONInspectionTests.workflowS12StorePathsHonorInvocationPointerAndDiagnosticLimits()",
+ "JSONInspectionTests.workflowS13FollowSelfCycleAndLongNarrowChain()",
+ "JSONInspectionTests.workflowB09NestedSnapshotRepeatsQuotasAndLateCancellation()",
+ "JSONInspectionTests.workflowS14MetadataAliasesAndScratchHaveExactRetentionDisposition()"
]
}
diff --git a/Documentation/README.md b/Documentation/README.md
index 8392303..f280c1a 100644
--- a/Documentation/README.md
+++ b/Documentation/README.md
@@ -1,3 +1,10 @@
+# Current authority — 1.8 candidate
+
+Published 1.7.0 has twelve libraries plus CLI and thirteen catalogs. This isolated
+1.8.0-dev branch adds bounded batches and a public immutable store inside
+JSONInspection. See [current qualification](Qualification/Release1.8.md).
+The dated predecessor inventories below are historical, not 1.8 pass evidence.
+
# Current documentation authority
Unreleased 1.7.0-dev consolidation has twelve libraries plus CLI and thirteen DocC
diff --git a/Documentation/ReleaseRoadmap.md b/Documentation/ReleaseRoadmap.md
index 550bb6c..1e7bd96 100644
--- a/Documentation/ReleaseRoadmap.md
+++ b/Documentation/ReleaseRoadmap.md
@@ -1,3 +1,13 @@
+# Execution update — 28 October 2026
+
+1.7.0 is published at `8557badef562cf2f059e964c113d4a7f60ecbe6c`.
+The maintainer has now commissioned the 1.8 bounded structural wave, including
+the public read-only store, on isolated `codex/1.8-batches-and-store`.
+[The implementation contract](Architecture/StructuralIntelligence/BoundedWorkflows1.8.md)
+freezes this wave; [qualification](Qualification/Release1.8.md) records actual results.
+The proposal and historical inventories below remain source-bound. Persistence,
+platform-floor reduction and new query grammars are not included.
+
# Release roadmap: 1.7 through 2.0
Maintainer working proposal, audited 27 October 2026. The next step is a release
diff --git a/Examples/BoundedWorkflows/Package.swift b/Examples/BoundedWorkflows/Package.swift
new file mode 100644
index 0000000..62eaba2
--- /dev/null
+++ b/Examples/BoundedWorkflows/Package.swift
@@ -0,0 +1,13 @@
+// swift-tools-version: 6.4
+// SPDX-License-Identifier: Apache-2.0
+import PackageDescription
+let package = Package(
+ name: "BoundedWorkflows",
+ platforms: [.macOS("27.0"), .iOS("27.0")],
+ dependencies: [.package(name: "Swift-JSON", path: "../..")],
+ targets: [.executableTarget(name: "BoundedWorkflows", dependencies: [
+ .product(name: "JSON", package: "Swift-JSON"),
+ .product(name: "JSONNavigation", package: "Swift-JSON"),
+ .product(name: "JSONInspection", package: "Swift-JSON")
+ ])], swiftLanguageModes: [.v6]
+)
diff --git a/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Main.swift b/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Main.swift
new file mode 100644
index 0000000..0d436bb
--- /dev/null
+++ b/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Main.swift
@@ -0,0 +1,50 @@
+// SPDX-License-Identifier: Apache-2.0
+import Foundation
+import JSON
+import JSONNavigation
+import JSONInspection
+
+@main struct BoundedWorkflows {
+ static func main() throws {
+ func witness(_ id: String, _ success: Bool) {
+ precondition(success, id); print("WORKFLOW_CASE \(id) passed")
+ }
+ let document = try JSONParser().parseDocument(Data(#"{"collections":{"u":[{"id":"é","amount":9007199254740993,"next":{"collection":"u","id":"e\u0301"}},{"id":"e\u0301","next":{"collection":"u","id":"é"}}]},"nothing":null}"#.utf8))
+ let paths: [[JSONLocationComponent]] = [[.key("collections"), .key("u"), .index(0), .key("amount")], [.key("nothing")], [.key("nothing")]]
+ let batch = try JSONInspectionBatch(document: document).values(at: paths)
+ witness("batch-exact-order", batch.result == [.number(try JSONNumber(parsing: "9007199254740993")), .null, .null])
+ do { _ = try JSONInspectionBatch(document: document).values(at: [[],[.key("missing")]]); preconditionFailure() }
+ catch let error as JSONAccessFailure { witness("missing-is-error", error.cause.code == .missingMember && error.path == [.key("missing")]) }
+ do { _ = try JSONInspectionBatch(document: document).values(at: paths, maximumRequests: 2); preconditionFailure() }
+ catch let error as JSONInspectionBatchFailure { witness("batch-capacity", error == .requestLimitExceeded(limit: 2, actual: 3)) }
+ let snapshot = try JSONStructuralSnapshotBuilder(document: document).build().result
+ witness("snapshot-root", try snapshot.values(for: [0,0]).result == [document.root, document.root])
+ let profile = JSONStoreProfile(version: .v1, collectionsAt: [.key("collections")], collections: [
+ .init(name: "u", recordsAt: [.key("u")], idKey: "id", referenceFields: [
+ .init(key: "next", presence: .required, nullable: false)])], references: .strict(marker: .v1, validateTargets: true))
+ let built = try JSONStoreBuilder(document: document, profile: profile).build(), store = built.result
+ witness("checked-construction", store.recordCount == 2 && store.references.count == 2 && built.statistics[.nodes] > 0)
+ guard let a = try store.lookup(collection: "u", id: "é").result,
+ let b = try store.lookup(collection: "u", id: "e\u{301}").result else { preconditionFailure() }
+ witness("scalar-identity", a != b && a.row == 0 && b.row == 1)
+ witness("missing-id", try store.lookup(collection: "u", id: "absent").result == nil)
+ let selected = try store.values(at: [b,a,b])
+ witness("store-batch", try selected.result[0].requiredMember("id") == .string("e\u{301}") && selected.result[0] == selected.result[2])
+ witness("complete-cycle", try store.follow(from: a).result == [a,b])
+ let another = try JSONStoreBuilder(document: document, profile: profile).build().result
+ do { _ = try another.value(at: a); preconditionFailure() }
+ catch let error as JSONStoreFailure { witness("owner-refusal", error.code == .invalidPosition) }
+ var bounds = JSONStoreFollowLimits(); bounds.maximumResults = 1
+ do { _ = try store.follow(from: a, bounds: bounds); preconditionFailure() }
+ catch let error as JSONStoreFailure { witness("complete-or-error", error.code == .capacityExceeded && error.capacity == "maximumResults") }
+ let measured = try store.follow(from: a)
+ var limits = JSONLimits.balanced; limits[.workUnits] = measured.statistics[.workUnits]
+ let equal = try store.follow(from: a, limits: limits)
+ limits[.workUnits] -= 1
+ do { _ = try store.follow(from: a, limits: limits); preconditionFailure() }
+ catch let error as JSONFailure {
+ witness("cumulative-report", equal.result == [a,b] && equal.statistics[.workUnits] == measured.statistics[.workUnits] && error.resource?.metric == .workUnits)
+ }
+ print("BOUNDED_WORKFLOWS passed cases=12 work=\(measured.statistics[.workUnits]) logical_retained=\(measured.statistics[.retainedBytes])")
+ }
+}
diff --git a/README.md b/README.md
index 4137295..cd0e3d6 100644
--- a/README.md
+++ b/README.md
@@ -1,17 +1,15 @@
# Swift-JSON
-This is **Swift-JSON 1.7.0 source prepared for release**. Version metadata identifies
-the source; the [official releases](https://github.com/ThorFuchs/Swift-JSON/releases)
-establish publication. At preparation, latest stable remains **1.6.1**. No tag or
-publication is implied by this checkout. The source combines exact Navigation and
-Inspection projections/graphs, JSONTransform and optional JSONValueDSL/JSONSchemaDSL.
-It has twelve library products plus the CLI and thirteen DocC catalogs including
-the shared, non-product JSONDSLCore reference.
-
-See the [combined release checklist](Documentation/Qualification/Release1.7Preparation.md),
-[release notes](Documentation/ReleaseNotes/1.7.0.md) and
-[1.7–2.0 roadmap](Documentation/ReleaseRoadmap.md). Every final candidate claim
-requires source-bound receipts; predecessor green checks alone do not qualify it.
+This is **Swift-JSON 1.8.0-dev**, based on the published
+[1.7.0 release](https://github.com/ThorFuchs/Swift-JSON/releases/tag/1.7.0)
+(`8557badef562cf2f059e964c113d4a7f60ecbe6c`). The next additive wave brings
+ordered batch reads and an immutable configured read-only store to JSONInspection.
+It is under qualification; it is not an admitted 1.8 release.
+Twelve libraries plus CLI, thirteen DocC catalogs and empty root dependencies remain.
+See the [1.8 implementation contract](Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md),
+[qualification record](Documentation/Qualification/Release1.8.md) and
+[release roadmap](Documentation/ReleaseRoadmap.md). Historical records below retain
+their original source and publication dates.
See the [release-line matrix](Documentation/Qualification/ReleaseLines.md) for exact
source scopes and publication boundaries. Version metadata identifies source; the
@@ -34,24 +32,15 @@ Maintained by [Thorsten Fuchs](https://github.com/ThorFuchs). Swift-JSON follows
semantic versioning. Use a released version and retain `Package.resolved` for
reproducible builds; `main` tracks ongoing development.
-**1.6.1** is the latest stable maintenance release: eight libraries plus CLI,
-eight DocC catalogs and the unchanged 1.6 API/runtime. The optional verified schema
-examples, structural traversal, structured HTML previews and bounded composition
-were introduced in 1.6.0. Exact JSON and offline schema validation remain the
-semantic authorities.
-The published 1.6.1 CLI reports `1.6.1`; this prepared source reports `1.7.0`.
-Both use a deterministic source-content digest.
-The [published 1.6.1 release](https://github.com/ThorFuchs/Swift-JSON/releases/tag/1.6.1)
-is pinned to `e431f157ececc0cf3d061192420b0e36a9062f83`. Use its matching eight-catalog
-DocC and qualification assets for the stable reference; see the
-[maintenance qualification notes](Documentation/Qualification/Release1.6.1.md).
-Historical N0/N1 plans and later candidates are distinguished in the
-[source authority map](Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md).
-
-This development source includes opt-in Inspection, Transform and authored DSLs:
-twelve libraries plus the CLI, thirteen products in total. Inspection, Transform and DSLs are absent from stable 1.6 releases. Pin a reviewed
-unreleased source commit to use those products; the compatible stable installation
-requirement below does not supply them. Integration into main does not publish 1.7.0.
+**1.7.0** is the latest stable release: twelve libraries plus CLI and thirteen
+DocC catalogs. Inspection, Transform and the authored DSL products are included.
+The 1.6.1 maintenance line retains its eight-library API separately. Published
+1.7.0 reports `1.7.0`; this feature source reports `1.8.0-dev`. Both expose a
+source-content digest distinct from the Git commit. Use exact version pins and
+matching release assets when reproducing a published edition.
+
+The new batch/store APIs require the 1.8 feature source. No package product,
+dependency, platform minimum or existing JSON/JSONSchema authority changes.
The 1.5.1 maintenance repairs remain included: best-effort schema diagnostics,
the complete 1.2.1 regression suites and Git-free qualification. See the
@@ -752,7 +741,7 @@ can have restricted descendant traversal; structural rectangular width is not
numeric matrix or schema admission. See the [compiled evidence guide](Sources/JSONInspection/Documentation.docc/ArrayEvidence.md),
[public consumer](Tools/InspectionQualification) and [accounting contract](Documentation/Architecture/StructuralIntelligence/Inspection.md).
-## Complete source-attached structural views (unreleased)
+## Complete source-attached structural views (introduced in 1.7)
Opt-in JSONInspection in this candidate combines array observations with a
complete parent-linked snapshot, selected physical tree views and strict sparse
@@ -767,7 +756,7 @@ Every construction has one cumulative finite ledger. Each later catalogue/cell/
location/value/span lookup is an explicitly separate full-source-admitted operation.
Ordinals are snapshot-local trusted positions, not durable identity. Logical
retention is not RSS; G4 controlled performance remains HOLD, with no speed claim.
-These additions are absent from published 1.6.1 and do not publish 1.7.0.
+These additions were absent from published 1.6.1 and are included in published 1.7.0.
The current development branch also contains package-private structural set and
exact numeric matrix experiments. They add no public API; see the
@@ -782,7 +771,7 @@ Every later member/cell read independently admits the whole captured source with
limits and an operation report. These are structural views; no set algebra, matrix
arithmetic or performance claim is included. See the [compiled public guide](Sources/JSONInspection/Documentation.docc/ExactProjections.md),
[independent consumer](Tools/ExactProjectionQualification) and [public boundary](Documentation/Architecture/StructuralIntelligence/NAV10/PublicAPI.md).
-Batch access remains explicitly deferred to NAV13. Stable 1.6.1 is unchanged.
+The 1.8 candidate adds aggregate value reads; set/matrix algebra remains outside scope. Stable 1.6.1 is unchanged.
NAV11 experiments with a package-only, exact source-backed directed simple graph
inside JSONInspection. Graph construction and BFS are not public API and are not
@@ -821,3 +810,26 @@ for source-bound gates, the explicit allocation-instrumentation exception and
platform limits. A path or revision consumer is preparation evidence; exact 1.7.0
tag consumption is a postauthorization check.
This product is absent from stable 1.6.1 and must pass qualification on its final versioned source before publication.
+
+## Bounded structural workflows (1.8 candidate)
+
+Use `JSONInspectionBatch.values(at:maximumRequests:)` for ordered typed paths,
+`JSONStructuralSnapshot.values(for:maximumRequests:limits:)` for snapshot ordinals,
+and `JSONConfiguredStore.values(at:maximumRequests:limits:)` for record positions.
+Each complete batch admits the source once and shares one finite ledger, preserves
+request order and repeats, and either returns every result or throws. Missing is
+an error; explicit JSON null remains a value. Empty batches still admit the source.
+
+Build a read-only store with `JSONStoreBuilder` and an explicit `JSONStoreProfile`.
+Collection paths, ID fields and reference fields are configured; exact scalar IDs
+are never normalized. `lookup(collection:id:limits:)` distinguishes a missing ID
+from an unknown collection. `follow(from:bounds:limits:)` returns a complete BFS
+closure, with finite depth/frontier/visited/result caps and no truncated success.
+Positions are local to one immutable store and cannot be forged or rebound.
+
+Run the [public SwiftPM example](Examples/BoundedWorkflows) with
+`swift run --package-path Examples/BoundedWorkflows BoundedWorkflows`.
+The [DocC guide](Sources/JSONInspection/Documentation.docc/BoundedWorkflows.md)
+explains separate operation reports, caller work and logical retention. This is
+an additive candidate API; no persistence, network resolver, new platform or
+performance improvement is claimed.
diff --git a/Scripts/build_documentation_site.py b/Scripts/build_documentation_site.py
index d17ad71..9bad227 100644
--- a/Scripts/build_documentation_site.py
+++ b/Scripts/build_documentation_site.py
@@ -20,8 +20,8 @@
REPOSITORY = 'https://github.com/ThorFuchs/Swift-JSON'
BASE_PATH = '/Swift-JSON'
SITE_URL = 'https://thorfuchs.github.io' + BASE_PATH + '/'
-SOURCE_VERSION = '1.7.0'
-STABLE_VERSION = '1.6.1'
+SOURCE_VERSION = '1.8.0-dev'
+STABLE_VERSION = '1.7.0'
GUIDES = {'JSON': ('Exact values and native coding', 'exactvaluesandcoding'),
'JSONSchema': ('Offline schema validation', 'offlinevalidation'),
'JSONSchemaExamples': ('Verified exact candidates', 'verifiedcandidates'),
@@ -115,7 +115,7 @@ def landing(commit):
Exact JSON. Clear contracts.
Keep decimal values exact, Unicode scalar keys distinct and resource use explicit. Explore {len(PRODUCTS)} Swift libraries that carry those semantics from input bytes to validated, transformed and encoded output.
Swift 6.4+ · Swift language mode 6 · macOS/SDK 27+ · No package dependencies
-Development main includes JSONInspection, JSONTransform and the optional authored DSLs; this source is prepared as 1.7.0. Stable {STABLE_VERSION} has eight library catalogs; use its matching compiled DocC assets or tagged source documentation.
+Development main includes JSONInspection, JSONTransform and the optional authored DSLs; this source is unreleased 1.8.0-dev with bounded batch/store work. Stable {STABLE_VERSION} has twelve library catalogs plus the shared JSONDSLCore reference; use its matching compiled DocC assets or tagged source documentation.
Start with the contract
Read the package overview and executable examples, then choose a reference above. The behavior contracts explain exact numbers, scalar identity, offline schema evaluation, cumulative budgets and stream completion.
diff --git a/Scripts/ci.py b/Scripts/ci.py
index 707114f..0b7cef7 100644
--- a/Scripts/ci.py
+++ b/Scripts/ci.py
@@ -8,6 +8,7 @@
from p2_acceptance import check_p2_cases, check_borrow_receipt, check_private_receipt, check_exact_cases, check_exact_receipt
from graph_acceptance import check_graph_cases, check_graph_receipt, check_public_graph_receipt
from store_acceptance import check_store_tests, check_store_consumer
+from qualify_structural_workflows import check_receipt as check_workflow_consumer
from projection_acceptance import check_projection_cases, check_public_projection_receipt
from test_discovery import DISCOVERY, ROOT_COUNTS, SHARED_COUNTS, PARAMETER_INVOCATIONS, check_restored
@@ -26,6 +27,7 @@
SHARED_TESTS += '|graphG30ConcurrentViewsHaveIndependentHealth|graphGP14ConcurrentQueries'
SHARED_TESTS += '|dslSharedFinalizationAndPlansAreInvocationOwned'
SHARED_TESTS += '|storeR06TerminalQuotasNeverPublishAndImmutableConcurrentHealth'
+SHARED_TESTS += '|workflowSharedImmutablePlansHaveIndependentBatchAndStoreHealth'
R0_TESTS = 'readerProbe|indexProbes|duplicateRecognitionPrecedesValueAndClosingSyntax'
def snapshot():
@@ -34,6 +36,26 @@ def snapshot():
def commands(lane, output):
# The qualification executable resolves output paths from its repository root.
output = output.relative_to(ROOT)
+ if lane in ('structural-workflows', 'structural-workflows-safety'):
+ if lane.endswith('safety'):
+ result = [
+ ['swift', 'test', '--test-product', 'JSONInspectionTests', '-c', 'release', '--jobs', '2', '--sanitize', 'address', '--scratch-path', '.build/workflow-address', '--filter', 'workflow'],
+ ['swift', 'test', '--test-product', 'JSONInspectionTests', '-c', 'release', '--jobs', '2', '--sanitize', 'thread', '--scratch-path', '.build/workflow-thread', '--filter', 'workflowSharedImmutablePlansHaveIndependentBatchAndStoreHealth'],
+ ['swift', 'build', '--target', 'JSONInspection', '--jobs', '2', '--scratch-path', '.build/workflow-strict', '-Xswiftc', '-strict-memory-safety', '-Xswiftc', '-warnings-as-errors'],
+ ]
+ else:
+ result = [
+ ['python3', '-B', 'Scripts/public_check.py'],
+ ['python3', '-B', 'Scripts/generate_cli_identity.py', '--check'],
+ ['python3', '-B', 'Scripts/test_qualification_harness.py'],
+ ['swift', 'test', '--test-product', 'JSONInspectionTests', '--jobs', '2', '--filter', 'workflow'],
+ ['swift', 'test', '--test-product', 'JSONInspectionTests', '-c', 'release', '--jobs', '2', '--filter', 'workflow'],
+ ['swift', 'test', '--test-product', 'JSONInspectionTests', '--jobs', '2', '--filter', 'store'],
+ ['swift', 'test', '--test-product', 'JSONInspectionTests', '-c', 'release', '--jobs', '2', '--filter', 'store'],
+ ['python3', '-B', 'Scripts/qualify_store.py', '--output', str(output / 'store-boundaries')],
+ ['python3', '-B', 'Scripts/qualify_structural_workflows.py', '--output', str(output / 'public-consumer')],
+ ]
+ return [c + ['--disable-xctest'] if sys.platform == 'linux' and c[:2] == ['swift', 'test'] else c for c in result]
if lane == 'store':
return [
['python3', '-B', 'Scripts/public_check.py'],
@@ -58,9 +80,9 @@ def commands(lane, output):
]
if lane == 'inspection':
result = [['python3', '-B', 'Scripts/public_check.py'],
- ['swift', 'test', '--filter', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store'],
- ['swift', 'test', '-c', 'release', '--filter', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store'],
- ['swift', 'test', '-c', 'release', '--sanitize', 'address', '--scratch-path', '.build/inspection-address', '--filter', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store'],
+ ['swift', 'test', '--filter', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store|workflow'],
+ ['swift', 'test', '-c', 'release', '--filter', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store|workflow'],
+ ['swift', 'test', '-c', 'release', '--sanitize', 'address', '--scratch-path', '.build/inspection-address', '--filter', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store|workflow'],
['swift', 'build', '--target', 'JSONInspection', '--scratch-path', '.build/inspection-strict',
'-Xswiftc', '-strict-memory-safety', '-Xswiftc', '-warnings-as-errors'],
['python3', '-B', 'Scripts/qualify_navigation_reducer.py', '--output', str(output / 'owned-codegen')],
@@ -69,6 +91,7 @@ def commands(lane, output):
['python3', '-B', 'Scripts/qualify_public_graph.py', '--output', str(output / 'public-graph-consumer')],
['python3', '-B', 'Scripts/qualify_store_preparation.py', '--output', str(output / 'prepared-codegen')],
['python3', '-B', 'Scripts/qualify_store.py', '--output', str(output / 'private-store-consumer')],
+ ['python3', '-B', 'Scripts/qualify_structural_workflows.py', '--output', str(output / 'public-workflow-consumer')],
['python3', '-B', 'Scripts/qualify_projections.py', '--output', str(output / 'projection-codegen')],
['python3', '-B', 'Scripts/verify_p2_fixtures.py'],
['python3', '-B', 'Scripts/qualify_p2_borrowing.py', '--output', str(output / 'p2-borrowing')],
@@ -229,6 +252,7 @@ def commands(lane, output):
['python3', '-B', 'Scripts/qualify_public_graph.py', '--output', str(output / 'public-graph-consumer')],
['python3', '-B', 'Scripts/qualify_store_preparation.py', '--output', str(output / 'prepared-codegen')],
['python3', '-B', 'Scripts/qualify_store.py', '--output', str(output / 'private-store-consumer')],
+ ['python3', '-B', 'Scripts/qualify_structural_workflows.py', '--output', str(output / 'public-workflow-consumer')],
['swift', 'test', '--package-path', 'Tools/MobileQualification'],
['swift', 'run', '-c', 'release', '--package-path', 'Tools/MobileQualification', 'MobileQualification'],
['swift', 'test', '--package-path', 'Tools/MCPReadiness'],
@@ -279,6 +303,7 @@ def check_result(command, code, log, events=None):
if events is not None: counts = events['bundle_counts']
selected_filter = command[command.index('--filter') + 1] if '--filter' in command else None
store_test = selected_filter in ('store', 'storeR06TerminalQuotasNeverPublishAndImmutableConcurrentHealth')
+ workflow_test = selected_filter in ('workflow', 'workflowSharedImmutablePlansHaveIndependentBatchAndStoreHealth')
prepared_test = selected_filter in ('navigationPreparedD0', 'navigationPreparedD0IndependentConcurrentHealth')
if command[:2] == ['swift', 'test']:
expected = [DISCOVERY['composition_external_functions']] if 'Tools/CompositionQualification' in command else [12] if 'composition|composed|caughtOperationalFailureCannotComplete' in command else [79] if 'JSONSchemaTests' in command else [4 if 'thread' in command else 98] if 'JSONSchemaExamplesTests' in command else [88] if 'cli|sharedFilterPlan' in command else sorted(DISCOVERY['export_functions'].values()) if 'export|Export|apple' in command else [14] if 'Tools/WasmAudit' in command else sorted(DISCOVERY['qa_functions'].values()) if 'qa' in command else [7] if R0_TESTS in command else [3] if 'Tools/MCPReadiness' in command or 'Tools/MobileQualification' in command else [1] if 'Tools/Qualification' in command else sorted(SHARED_COUNTS.values()) if 'thread' in command else sorted(ROOT_COUNTS.values())
@@ -291,7 +316,7 @@ def check_result(command, code, log, events=None):
expected = [DISCOVERY['root_functions']['JSONNavigationTests']]
if events is not None:
passed = passed and events['module_function_counts'] == {'JSONNavigationTests': expected[0]} and events['parameter_case_invocations'] == 0
- if any(f in command for f in ['inspection', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store']):
+ if any(f in command for f in ['inspection', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store|workflow']):
expected = [DISCOVERY['root_functions']['JSONInspectionTests']]
if events is not None:
passed = passed and events['module_function_counts'] == {'JSONInspectionTests': expected[0]} and events['parameter_case_invocations'] == 65 and events['parameterized_functions'] == {'storeD1NativeVector(id:)': 65} and check_projection_cases(events) and check_p2_cases(events) and check_exact_cases(events) and check_graph_cases(events) and check_store_tests(events, log)
@@ -306,12 +331,18 @@ def check_result(command, code, log, events=None):
shared_store = any(x.startswith('storeR06') for x in command)
expected = [1 if shared_store else 12]
passed = passed and events is not None and check_store_tests(events, log, shared=shared_store, scoped=True)
+ if workflow_test:
+ expected_ids = set(DISCOVERY['workflow_function_ids'])
+ if selected_filter != 'workflow':
+ expected_ids = {x for x in expected_ids if 'SharedImmutablePlans' in x}
+ expected = [len(expected_ids)]
+ passed = passed and events is not None and events['module_function_counts'] == {'JSONInspectionTests': len(expected_ids)} and set(events['function_names']) == expected_ids and len(events['function_names']) == len(expected_ids) and events['parameter_case_invocations'] == 0
passed = passed and sorted(counts) == expected
if 'Tools/CompositionQualification' in command and events is not None:
passed = passed and events['module_function_counts'] == {'CompositionScaleTests': DISCOVERY['composition_external_functions']} and events['parameter_case_invocations'] == 0
if 'qa' in command and events is not None:
passed = passed and events['module_function_counts'] == DISCOVERY['qa_functions'] and check_restored(events)
- if 'thread' in command and 'JSONSchemaExamplesTests' not in command and not prepared_test and not store_test and events is not None:
+ if 'thread' in command and 'JSONSchemaExamplesTests' not in command and not prepared_test and not store_test and not workflow_test and events is not None:
passed = passed and events['module_function_counts'] == SHARED_COUNTS and check_projection_cases(events, shared=True) and check_p2_cases(events, shared=True) and check_exact_cases(events, shared=True) and check_graph_cases(events, shared=True) and events['parameter_case_invocations'] == 0 and check_store_tests(events, log, shared=True)
if '--filter' not in command and '--test-product' not in command and not any(x.startswith('Tools/') for x in command):
if events is not None:
@@ -388,6 +419,9 @@ def check_result(command, code, log, events=None):
if command[:3] in (['python3', '-B', 'Scripts/qualify_graph_context.py'], ['python3', '-B', 'Scripts/qualify_graph.py']):
data = json.loads((ROOT / command[command.index('--output') + 1] / 'receipt.json').read_text())
passed = passed and check_graph_receipt(data, context='qualify_graph_context.py' in command[2])
+ if command[:3] == ['python3', '-B', 'Scripts/qualify_structural_workflows.py']:
+ data = json.loads((ROOT / command[command.index('--output') + 1] / 'receipt.json').read_text())
+ passed = passed and check_workflow_consumer(data)
if command[:3] == ['python3', '-B', 'Scripts/qualify_store.py']:
data = json.loads((ROOT / command[command.index('--output') + 1] / 'receipt.json').read_text())
passed = passed and check_store_consumer(data)
@@ -431,12 +465,12 @@ def check_result(command, code, log, events=None):
if command[:3] == ['python3', '-B', 'Scripts/qualify_html_browser.py']:
passed = passed and 'JSONHTML_BROWSER passed engines=3 cases=76 skips=0' in log.splitlines()
if 'Scripts/test_qualification_harness.py' in command:
- passed = passed and 'HARNESS_TESTS passed cases=26' in log.splitlines()
+ passed = passed and 'HARNESS_TESTS passed cases=28' in log.splitlines()
return passed, counts
def main():
parser = argparse.ArgumentParser(description=__doc__)
- parser.add_argument('--lane', choices=['store', 'store-preparation', 'composition', 'navigation', 'inspection', 'native', 'safety', 'docs', 'r0', 'qa', 'linux', 'linux-safety', 'wasm', 'browser', 'export', 'cli', 'schema-examples', 'schema-examples-safety', 'schema-examples-determinism', 'schema-examples-construction', 'schema-examples-advanced', 'schema-examples-qualification', 'coverage', 'linux-coverage', 'linux-qa', 'wasm-qa', 'archive'], required=True)
+ parser.add_argument('--lane', choices=['structural-workflows', 'structural-workflows-safety', 'store', 'store-preparation', 'composition', 'navigation', 'inspection', 'native', 'safety', 'docs', 'r0', 'qa', 'linux', 'linux-safety', 'wasm', 'browser', 'export', 'cli', 'schema-examples', 'schema-examples-safety', 'schema-examples-determinism', 'schema-examples-construction', 'schema-examples-advanced', 'schema-examples-qualification', 'coverage', 'linux-coverage', 'linux-qa', 'wasm-qa', 'archive'], required=True)
parser.add_argument('--output')
parser.add_argument('--skip-host-measurements', action='store_true', help='For WASI runtime qualification only: record optional host timings as NOT RUN.')
args = parser.parse_args()
@@ -448,7 +482,7 @@ def main():
repository = subprocess.run(['git', 'rev-parse', '--show-toplevel'], cwd=ROOT, capture_output=True, text=True)
is_checkout = repository.returncode == 0 and pathlib.Path(repository.stdout.strip()).resolve() == ROOT.resolve()
commit = subprocess.run(['git', 'rev-parse', 'HEAD'], cwd=ROOT, capture_output=True, text=True) if is_checkout else None
- environment = collect_linux() if args.lane.startswith('linux') or (args.lane in ('composition', 'inspection') and sys.platform == 'linux') else collect()
+ environment = collect_linux() if args.lane.startswith('linux') or (args.lane in ('composition', 'inspection', 'structural-workflows', 'structural-workflows-safety') and sys.platform == 'linux') else collect()
receipt = {'schema_version': 1, 'lane': args.lane, 'status': 'blocked', 'environment': environment, 'started_utc': datetime.datetime.now(datetime.timezone.utc).isoformat(), 'git_commit': commit.stdout.strip() if commit is not None and commit.returncode == 0 else None, 'github_run_id': os.environ.get('GITHUB_RUN_ID'), 'source_inputs': inputs, 'source_manifest_sha256': hashlib.sha256(json.dumps(inputs, sort_keys=True).encode()).hexdigest(), 'commands': []}
def save():
(output / 'receipt.json').write_text(json.dumps(receipt, indent=2) + '\n')
diff --git a/Scripts/qualify_store.py b/Scripts/qualify_store.py
index 950b137..fbca82e 100644
--- a/Scripts/qualify_store.py
+++ b/Scripts/qualify_store.py
@@ -31,7 +31,7 @@
'D069':('func illegal() { _ = JSONStoreProfile(version:.v1,collectionsAt:[],collections:[],references:.disabled,guessID:true) }',"extra argument 'guessID' in call"),
'store-initializer':('func illegal() { _ = JSONConfiguredStore() }','initializer is inaccessible'),
'position-forge':('func illegal() { _ = JSONStorePosition() }','initializer is inaccessible'),
- 'external-store':('func illegal(_ x: JSONStoreProfile) {}',"cannot find type 'JSONStoreProfile' in scope"),
+ 'external-scope':('func illegal(_ x: JSONReadScope) {}',"cannot find type 'JSONReadScope' in scope"),
'bound-member':('import JSONNavigation\nfunc illegal(_ c: inout JSONTraversalContext) { _ = c.member }',"cannot reference 'mutating' method as function value"),
'member-escape':('import JSONNavigation\nfunc illegal(_ c: inout JSONTraversalContext) -> () throws -> JSON? { { try c.member(.null,key:"x",at:[]) } }',"escaping closure captures 'inout' parameter"),
}
diff --git a/Scripts/qualify_structural_workflows.py b/Scripts/qualify_structural_workflows.py
new file mode 100644
index 0000000..c6908ed
--- /dev/null
+++ b/Scripts/qualify_structural_workflows.py
@@ -0,0 +1,77 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Real external SwiftPM consumer plus domain-specific public-boundary codegen."""
+import argparse, hashlib, json, pathlib, re, subprocess
+from archive_candidate import files
+ROOT = pathlib.Path(__file__).resolve().parents[1]
+CASES = {'batch-exact-order', 'missing-is-error', 'batch-capacity', 'snapshot-root',
+ 'checked-construction', 'scalar-identity', 'missing-id', 'store-batch',
+ 'complete-cycle', 'owner-refusal', 'complete-or-error', 'cumulative-report'}
+PROBES = {
+ 'position-forge': ('func invalid() { _ = JSONStorePosition() }', 'initializer is inaccessible'),
+ 'store-forge': ('func invalid() { _ = JSONConfiguredStore() }', 'initializer is inaccessible'),
+ 'private-context': ('func invalid(_ context: JSONTraversalContext) {}', "cannot find type 'JSONTraversalContext' in scope"),
+ 'version': ('func invalid() { _ = JSONStoreProfileVersion.v2 }', "has no member 'v2'"),
+ 'nullable': ('func invalid() { _ = JSONStoreReferenceField(key:"f",presence:.required,nullable:"yes") }', "cannot convert value of type 'String' to expected argument type 'Bool'"),
+ 'mutation': ('func invalid(_ store: JSONConfiguredStore) { store.document = JSONDocument(root:.null) }', "'document' is a 'let' constant"),
+}
+def check_receipt(data):
+ configs = {'debug', 'release'}
+ expected_names = {f'{c}-{name}' for c in configs for name in ['consumer','binpath','guide',*PROBES]}
+ commands = data.get('commands', [])
+ return (data.get('status') == 'passed' and len(commands) == len(expected_names)
+ and {c['name'] for c in commands} == expected_names
+ and all(c.get('passed') and c.get('compiler_crash') is False and
+ (c['exit'] > 0 and any(c['expected'].lower() in d.lower() for d in c['primary_diagnostics'])
+ if c.get('expected') else c['exit'] == 0) for c in commands)
+ and set(data.get('consumer_results', {})) == configs
+ and all(set(x['cases']) == CASES and len(x['cases']) == len(CASES)
+ and x['work_units'] > 0 and x['logical_retained_bytes'] > 0
+ for x in data['consumer_results'].values()))
+def main():
+ parser = argparse.ArgumentParser(); parser.add_argument('--output', required=True)
+ args = parser.parse_args(); out = ROOT / args.output; out.mkdir(parents=True, exist_ok=True)
+ receipt = {'status':'running', 'source_inputs':[
+ {'path':p.relative_to(ROOT).as_posix(),'sha256':hashlib.sha256(p.read_bytes()).hexdigest()} for p in files()],
+ 'commands':[], 'consumer_results':{}}
+ def save(): (out/'receipt.json').write_text(json.dumps(receipt, indent=2)+'\n')
+ def run(name, command, expected=None):
+ result = subprocess.run(command, cwd=ROOT, capture_output=True, text=True, timeout=1200)
+ log = result.stdout + result.stderr
+ signal = result.returncode < 0
+ crash = (signal and command[0] == 'swiftc') or any(x in log for x in ['Stack dump:', 'Assertion failed', 'Found ownership error', 'fatal error encountered during compilation'])
+ diagnostics = re.findall(r': error: ([^\n]+)', log)
+ passed = not signal and not crash and (result.returncode == 0 if expected is None else result.returncode > 0 and any(expected.lower() in d.lower() for d in diagnostics))
+ path = out/(name+'.log'); path.write_text(log)
+ receipt['commands'].append(dict(name=name, command=command, exit=result.returncode, passed=passed,
+ compiler_crash=crash, signal_termination=signal, expected=expected, primary_diagnostics=diagnostics,
+ log=str(path.relative_to(ROOT)), log_sha256=hashlib.sha256(path.read_bytes()).hexdigest()))
+ save(); print(name, result.returncode, 'PASS' if passed else 'FAIL', flush=True)
+ if not passed: raise ValueError(name)
+ return result.stdout
+ try:
+ for config in ['debug','release']:
+ common = ['swift', 'run', '-c', config, '--jobs', '2', '--package-path', 'Examples/BoundedWorkflows', '--scratch-path', '.build/workflow-consumer']
+ text = run(config+'-consumer', common+['BoundedWorkflows'])
+ cases = re.findall(r'^WORKFLOW_CASE (\S+) passed$', text, re.M)
+ summary = re.findall(r'^BOUNDED_WORKFLOWS passed cases=12 work=([1-9]\d*) logical_retained=([1-9]\d*)$', text, re.M)
+ if len(cases) != 12 or set(cases) != CASES or len(summary) != 1: raise ValueError('Incomplete actual consumer witnesses')
+ receipt['consumer_results'][config] = dict(cases=cases, work_units=int(summary[0][0]), logical_retained_bytes=int(summary[0][1]))
+ bp = pathlib.Path(run(config+'-binpath', ['swift','build','-c',config,'--package-path','Examples/BoundedWorkflows','--scratch-path','.build/workflow-consumer','--show-bin-path']).strip())
+ modules = bp/'Modules' if (bp/'Modules').exists() else bp
+ guide = ROOT/'Sources/JSONInspection/Documentation.docc/BoundedWorkflows.md'
+ blocks = re.findall(r'```swift\n(.*?)\n```', guide.read_text(), re.S)
+ if len(blocks) != 1: raise ValueError('Changed DocC example inventory')
+ source = out/'guide.swift'; source.write_text(blocks[0]+'\n')
+ run(config+'-guide', ['swiftc','-swift-version','6','-warnings-as-errors','-I',str(modules),
+ '-Onone' if config == 'debug' else '-O', '-parse-as-library','-c',str(source),'-o',str(out/(config+'-guide.o'))])
+ for name, (body, expected) in PROBES.items():
+ source = out/(name+'.swift'); source.write_text('import JSON\nimport JSONNavigation\nimport JSONInspection\n'+body+'\n')
+ run(config+'-'+name, ['swiftc','-swift-version','6','-warnings-as-errors','-I',str(modules),
+ '-Onone' if config == 'debug' else '-O', '-parse-as-library','-c',str(source),'-o',str(out/(config+'-'+name+'.o'))], expected)
+ receipt['status'] = 'passed'
+ if not check_receipt(receipt): raise ValueError('Incomplete receipt')
+ except (ValueError, OSError, subprocess.TimeoutExpired) as error:
+ receipt.update(status='failed', error=str(error))
+ save(); return 0 if receipt['status'] == 'passed' else 1
+if __name__ == '__main__': raise SystemExit(main())
diff --git a/Scripts/test_qualification_harness.py b/Scripts/test_qualification_harness.py
index 4662219..d766277 100644
--- a/Scripts/test_qualification_harness.py
+++ b/Scripts/test_qualification_harness.py
@@ -28,6 +28,33 @@ def store_markers():
'STORE_RUNTIME R%02d passed' % n for n in range(1,11)])
class HarnessTests(unittest.TestCase):
+ def test_workflow_discovery_requires_all_exact_identities(self):
+ names = DISCOVERY['workflow_function_ids']
+ self.assertEqual(len(names), 24)
+ events = dict(bundle_counts=[24], module_function_counts={'JSONInspectionTests':24},
+ function_names=names, parameter_case_invocations=0)
+ command = ['swift','test','--test-product','JSONInspectionTests','--filter','workflow']
+ self.assertTrue(check_result(command, 0, '', events)[0])
+ for broken in [dict(events, function_names=names[:-1]), dict(events, function_names=names+names[:1]),
+ dict(events, parameter_case_invocations=1), dict(events, bundle_counts=[21])]:
+ self.assertFalse(check_result(command, 0, '', broken)[0])
+
+ def test_workflow_consumer_rejects_missing_witnesses_or_incidental_diagnostics(self):
+ from qualify_structural_workflows import CASES, PROBES, check_receipt
+ data = dict(status='passed', commands=[dict(name=c+'-'+name, passed=True, compiler_crash=False,
+ exit=1 if name in PROBES else 0, expected=PROBES[name][1] if name in PROBES else None,
+ primary_diagnostics=[PROBES[name][1]] if name in PROBES else [])
+ for c in ['debug','release'] for name in ['consumer','binpath','guide',*PROBES]],
+ consumer_results={c:dict(cases=sorted(CASES),work_units=1,logical_retained_bytes=1) for c in ['debug','release']})
+ self.assertTrue(check_receipt(data))
+ for mode in ['missing-command','missing-case','incidental','crash']:
+ bad=copy.deepcopy(data)
+ if mode == 'missing-command': bad['commands'].pop()
+ elif mode == 'missing-case': bad['consumer_results']['release']['cases'].pop()
+ elif mode == 'incidental': bad['commands'][3]['primary_diagnostics']=['unrelated error']
+ else: bad['commands'][3]['compiler_crash']=True
+ self.assertFalse(check_receipt(bad))
+
def test_scoped_catalog_check_cannot_weaken_the_full_site_inventory(self):
with tempfile.TemporaryDirectory() as temporary:
root = pathlib.Path(temporary).resolve()
@@ -153,7 +180,7 @@ def test_composition_discovery_and_capacity_limits_are_not_blanket_successes(sel
# Combined-source checks must retain both independently owned optional products.
for module, count, command in [
('JSONNavigationTests', 57, ['swift', 'test', '--filter', 'navigation']),
- ('JSONInspectionTests', 190, ['swift', 'test', '--filter', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store']),
+ ('JSONInspectionTests', 214, ['swift', 'test', '--filter', 'inspection|snapshot|projectionP|p2|exactProjectionPUB|graphG|store|workflow']),
('JSONSchemaExamplesTests', 98, ['swift', 'test', '--test-product', 'JSONSchemaExamplesTests']),
]:
events = dict(bundle_counts=[count], module_function_counts={module: count},
@@ -167,12 +194,12 @@ def test_composition_discovery_and_capacity_limits_are_not_blanket_successes(sel
for key in ('root_functions', 'portable_root_functions', 'wasi_root_functions'):
self.assertEqual(policy[key]['JSONNavigationTests'], 57)
self.assertEqual(policy[key]['JSONSchemaExamplesTests'], 98)
- self.assertEqual(policy[key]['JSONInspectionTests'], 190)
+ self.assertEqual(policy[key]['JSONInspectionTests'], 214)
self.assertEqual(policy[key]['JSONIntegrationTests'], 34)
for key in ('shared_state_functions', 'portable_shared_state_functions'):
self.assertEqual(policy[key]['JSONNavigationTests'], 2)
self.assertEqual(policy[key]['JSONSchemaExamplesTests'], 4)
- self.assertEqual(policy[key]['JSONInspectionTests'], 10)
+ self.assertEqual(policy[key]['JSONInspectionTests'], 11)
self.assertEqual(policy[key]['JSONIntegrationTests'], 11)
def test_private_projection_identities_cannot_be_missing_renamed_or_duplicated(self):
@@ -364,8 +391,8 @@ def test_exact_discovery_rejects_missing_duplicate_and_changed_parameters(self):
identity['function_names'][0] = 'JSONTests.unrelatedReplacement()'
self.assertFalse(check_restored(identity))
harness = ['python3', '-B', 'Scripts/test_qualification_harness.py']
- self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=26')[0])
- for stale in ['HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', '']:
+ self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=28')[0])
+ for stale in ['HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', '']:
self.assertFalse(check_result(harness, 0, stale)[0])
lifetime = ['python3', '-B', 'Scripts/qualify_schema_example_lifetime.py']
marker = 'SCHEMA_EXAMPLES_CORPUS passed mode=owned scopes=128 values=6 exact=1'
diff --git a/Sources/JSONCLIEngine/CLIBuildIdentity.swift b/Sources/JSONCLIEngine/CLIBuildIdentity.swift
index a659e0c..8af8093 100644
--- a/Sources/JSONCLIEngine/CLIBuildIdentity.swift
+++ b/Sources/JSONCLIEngine/CLIBuildIdentity.swift
@@ -2,6 +2,6 @@
// Generated by Scripts/generate_cli_identity.py; excludes this file.
// SHA-256 of the ordered Package.swift + Sources content manifest, not a Git commit.
package enum CLIBuildIdentity {
- package static let revision = "sha256:138ae7e8bc2f4392db7aafaa000f60db5915658a1b34a55f12a9c27958da1312"
+ package static let revision = "sha256:43023670d6664373702741d960b010494d86149aaa181b4eeb9d6ff13701d60e"
package static let kind = "source-sha256"
}
diff --git a/Sources/JSONCLIEngine/CLIReleaseVersion.swift b/Sources/JSONCLIEngine/CLIReleaseVersion.swift
index 6ed3782..9927fbf 100644
--- a/Sources/JSONCLIEngine/CLIReleaseVersion.swift
+++ b/Sources/JSONCLIEngine/CLIReleaseVersion.swift
@@ -2,5 +2,5 @@
// Version of this source distribution. Publication is a separate action.
package enum CLIReleaseVersion {
- package static let value = "1.7.0"
+ package static let value = "1.8.0-dev"
}
diff --git a/Sources/JSONInspection/Documentation.docc/BoundedWorkflows.md b/Sources/JSONInspection/Documentation.docc/BoundedWorkflows.md
new file mode 100644
index 0000000..081d7b4
--- /dev/null
+++ b/Sources/JSONInspection/Documentation.docc/BoundedWorkflows.md
@@ -0,0 +1,73 @@
+# Read batches and configured stores
+
+Read ordered exact values, or follow explicitly declared links in an immutable source.
+
+## Choose the smallest operation
+
+Use ``JSONInspectionBatch`` for typed path requests, or
+``JSONStructuralSnapshot/values(for:maximumRequests:limits:)`` when you already
+have snapshot-local ordinals. Repeated requests remain repeated outputs. Missing
+selections throw; explicit JSON null is a value. Empty batches still admit the
+whole captured source, including its source map. Each batch has one cumulative
+native admission/work/retention ledger, rather than one independent read per item.
+
+The default request ceiling is 1,024. Limits are inclusive. Invalid configurations,
+resource exhaustion and cancellation throw without publishing a partial array.
+Returned JSON values retain ordinary native value semantics. Logical alias bytes
+are finite accounting policy, not a process RSS or caller-instruction guarantee.
+
+## Declare a read-only store
+
+``JSONStoreBuilder`` promotes a source to ``JSONConfiguredStore`` only after the
+existing complete eight-phase checked construction. Names, ID fields, paths and
+reference fields are explicit. There is no id/_id inference, Unicode normalization,
+database, file loader or network resolution. This API is a candidate for 1.8;
+released 1.7 does not contain it.
+
+```swift
+import Foundation
+import JSON
+import JSONInspection
+
+func boundedStoreExample() throws -> [JSON] {
+ let document = try JSONParser().parseDocument(Data(#"{"users":[{"id":"a","friend":{"collection":"users","id":"b"}},{"id":"b","friend":null}]}"#.utf8))
+ let profile = JSONStoreProfile(version: .v1, collectionsAt: [], collections: [
+ .init(name: "users", recordsAt: [.key("users")], idKey: "id", referenceFields: [
+ .init(key: "friend", presence: .required, nullable: true)])
+ ], references: .strict(marker: .v1, validateTargets: true))
+ let store = try JSONStoreBuilder(document: document, profile: profile).build().result
+ guard let first = try store.lookup(collection: "users", id: "a").result else { return [] }
+ let closure = try store.follow(from: first)
+ // Construction, lookup, follow and value reads are separate reported operations.
+ // Every selection within this one batch shares its original native ledger.
+ let values = try store.values(at: closure.result)
+ return values.result
+}
+```
+
+The runnable [public consumer](https://github.com/ThorFuchs/Swift-JSON/tree/main/Examples/BoundedWorkflows)
+exercises exact scalar IDs, missing versus null, repeated selections, cycles,
+owner refusals and cumulative reports. Use this feature branch for the 1.8 API;
+a compatible dependency on 1.7 cannot supply it.
+
+## Understand ownership and errors
+
+Positions are store-instance-local tokens. They cannot be constructed publicly
+or reused against another build, even if that build has byte-identical input.
+Fixed collection/record/reference metadata access is ordinary trusted caller work;
+scanning lookup and value/follow operations have explicit reports and finite quotas.
+A missing ID in a known collection returns nil. An unknown collection throws
+``JSONStoreFailure`` with ``JSONStoreFailureCode/unknownCollection``. Declared
+reference slots distinguish absent, null and validated links. Marker-shaped values
+outside declared fields remain ordinary JSON.
+
+``JSONStoreFollowLimits`` defaults to depth64 and frontier/visited/results10,000.
+Follow includes the start exactly once and visits new positions breadth-first in
+configured field order. A self-cycle does not consume another depth. Any reachable
+new node beyond a bound throws: depth is a complete-search capacity, not permission
+to return a truncated closure. Cancellation invalidates the invocation; another
+operation on the same immutable store starts with independent health.
+
+The original twelve store construction ceilings and 49 core metrics are unchanged.
+They are finite policies, not a demonstrated joint capacity or heap bound. No
+persistence, mutable store, parallel batches or new performance claim is supplied.
diff --git a/Sources/JSONInspection/Documentation.docc/JSONInspection.md b/Sources/JSONInspection/Documentation.docc/JSONInspection.md
index e4a99f4..ad9572a 100644
--- a/Sources/JSONInspection/Documentation.docc/JSONInspection.md
+++ b/Sources/JSONInspection/Documentation.docc/JSONInspection.md
@@ -57,3 +57,16 @@ or numeric matrix. Sampling does not exempt any source from admission.
- ``JSONGraphProfile``
- ``JSONGraphProjection``
- ``JSONGraphFailure``
+
+### Bounded structural workflows (1.8 candidate)
+
+-
+- ``JSONInspectionBatch``
+- ``JSONInspectionBatchFailure``
+- ``JSONStoreBuilder``
+- ``JSONConfiguredStore``
+- ``JSONStoreProfile``
+- ``JSONStoreCaps``
+- ``JSONStoreFollowLimits``
+- ``JSONStorePosition``
+- ``JSONStoreFailure``
diff --git a/Sources/JSONInspection/JSONInspectionBatch.swift b/Sources/JSONInspection/JSONInspectionBatch.swift
new file mode 100644
index 0000000..5ccbd63
--- /dev/null
+++ b/Sources/JSONInspection/JSONInspectionBatch.swift
@@ -0,0 +1,153 @@
+// SPDX-License-Identifier: Apache-2.0
+import JSON
+import JSONNavigation
+
+/// A finite batch request policy refusal; ordinary selection errors retain core paths.
+public enum JSONInspectionBatchFailure: Error, Sendable, Equatable {
+ /// The requested result count exceeds the inclusive batch bound.
+ /// - Parameters:
+ /// - limit: Configured finite maximum.
+ /// - actual: Number of caller requests, including repeated requests.
+ case requestLimitExceeded(limit: Int, actual: Int)
+}
+
+internal func checkInspectionBatch(_ count: Int, maximum: Int) throws {
+ try Task.checkCancellation()
+ guard maximum >= 0, maximum < Int.max else { throw JSONFailure(.configuration, code: .invalidLimits) }
+ guard count <= maximum else { throw JSONInspectionBatchFailure.requestLimitExceeded(limit: maximum, actual: count) }
+}
+
+/// Ordered exact value reads with one complete source admission and cumulative ledger.
+/// Caller-authored paths and later caller copies retain their ordinary value semantics.
+public struct JSONInspectionBatch: Sendable {
+ /// Canonical immutable source used by every request.
+ public let document: JSONDocument
+ /// Finite original native quotas for each batch, not per selection.
+ public let limits: JSONLimits
+ package var checkpointObserver: (@Sendable (UInt64) -> Void)?
+ package var outputObserver: (@Sendable (Int) -> Void)?
+ package var lifetimeObserver: (@Sendable (Bool, Bool) -> Void)?
+
+ /// Captures immutable input without opening an operation or scanning it.
+ /// - Parameters:
+ /// - document: Canonical source, optionally with coherent source mapping.
+ /// - limits: Finite cumulative admission, work, path and retention quotas.
+ public init(document: JSONDocument, limits: JSONLimits = .balanced) {
+ self.document = document; self.limits = limits
+ }
+
+ /// Reads all requested values or throws, preserving caller order and repeats.
+ /// Even an empty batch admits the entire source. JSON null is a value; missing
+ /// paths throw. The result owns32 logical header bytes and32 per alias slot;
+ /// JSON aliases follow existing read semantics, not an RSS guarantee.
+ /// - Parameters:
+ /// - paths: Absolute typed selections; an empty path selects the root.
+ /// - maximumRequests: Inclusive request cap, default1024; zero admits only [].
+ /// - Returns: All exact values and one cumulative operation report.
+ /// - Throws: Batch capacity, core access/resource/configuration errors or cancellation.
+ public func values(at paths: [[JSONLocationComponent]], maximumRequests: Int = 1_024)
+ throws -> JSONOperationReport<[JSON]> {
+ try checkInspectionBatch(paths.count, maximum: maximumRequests)
+ var reducer = PathBatchReducer(outputObserver: outputObserver)
+ var traversal = JSONTraversal(document: document, limits: limits)
+ traversal.checkpointObserver = checkpointObserver
+ traversal.lifetimeObserver = lifetimeObserver
+ return try traversal.inspectPreparedOwned(scope: .virtual(roots: paths, overlap: .orderedOccurrences),
+ options: .init(depthLimit: 0), reducer: &reducer)
+ }
+}
+
+private struct PathBatchReducer: JSONReadPreparation, JSONTraversalOwnedReducer {
+ var values: [JSON] = []
+ let outputObserver: (@Sendable (Int) -> Void)?
+ mutating func prepare(context: inout JSONReadPreparationContext) throws { try context.retain(32) }
+ mutating func observe(_ hit: JSONTraversalHit, value: JSON,
+ context: inout JSONTraversalContext) throws -> JSONTraversalAction {
+ try context.work(); try context.retain(32); values.append(value)
+ outputObserver?(values.count)
+ return .skipDescendants
+ }
+ mutating func finish(_ summary: JSONTraversalSummary, context: inout JSONTraversalContext) throws -> [JSON] {
+ try context.checkpoint()
+ let result = values; values = []; return result
+ }
+}
+
+extension JSONStructuralSnapshot {
+ /// Resolves ordered snapshot positions with one source admission and ledger.
+ /// This does not invoke singular public reads. Invalid positions throw before
+ /// admission; repeats are preserved, billed and never silently deduplicated.
+ /// - Parameters:
+ /// - ordinals: Positions local to this complete immutable snapshot.
+ /// - maximumRequests: Inclusive finite request/output count, default1024.
+ /// - limits: Cumulative native quotas for this entire independent batch.
+ /// - Returns: Exact values in request order and cumulative statistics.
+ /// - Throws: Position/capacity, core resource/access/configuration errors or cancellation.
+ public func values(for ordinals: [Int], maximumRequests: Int = 1_024,
+ limits: JSONLimits = .balanced) throws -> JSONOperationReport<[JSON]> {
+ try checkInspectionBatch(ordinals.count, maximum: maximumRequests)
+ return try readValues(for: ordinals, limits: limits)
+ }
+ package func valuesForQualification(for ordinals: [Int], limits: JSONLimits,
+ outputObserver: @Sendable @escaping (Int) -> Void,
+ lifetimeObserver: @Sendable @escaping (Bool, Bool) -> Void) throws -> JSONOperationReport<[JSON]> {
+ try checkInspectionBatch(ordinals.count, maximum: 1_024)
+ return try readValues(for: ordinals, limits: limits, outputObserver: outputObserver, lifetimeObserver: lifetimeObserver)
+ }
+ private func readValues(for ordinals: [Int], limits: JSONLimits,
+ outputObserver: (@Sendable (Int) -> Void)? = nil,
+ lifetimeObserver: (@Sendable (Bool, Bool) -> Void)? = nil) throws -> JSONOperationReport<[JSON]> {
+ var reducer = SnapshotBatchReducer(snapshot: self, ordinals: ordinals, outputObserver: outputObserver)
+ var traversal = JSONTraversal(document: document, limits: limits)
+ traversal.lifetimeObserver = lifetimeObserver
+ return try traversal.inspectPreparedOwned(options: .init(depthLimit: 0), reducer: &reducer)
+ }
+}
+
+private struct SnapshotBatchReducer: JSONReadPreparation, JSONTraversalOwnedReducer {
+ let snapshot: JSONStructuralSnapshot
+ let ordinals: [Int]
+ let outputObserver: (@Sendable (Int) -> Void)?
+ private var requestAliases: UInt64 = 0
+ init(snapshot: JSONStructuralSnapshot, ordinals: [Int], outputObserver: (@Sendable (Int) -> Void)?) {
+ self.snapshot = snapshot; self.ordinals = ordinals; self.outputObserver = outputObserver
+ }
+ mutating func prepare(context: inout JSONReadPreparationContext) throws {
+ try context.retain(32)
+ for ordinal in ordinals {
+ try context.work(); _ = try snapshot.record(at: ordinal)
+ try context.retain(8); requestAliases += 8 // count was capped before preparation
+ }
+ }
+ mutating func observe(_ hit: JSONTraversalHit, value: JSON,
+ context: inout JSONTraversalContext) throws -> JSONTraversalAction {
+ try context.work(); return .skipDescendants
+ }
+ mutating func finish(_ summary: JSONTraversalSummary, context: inout JSONTraversalContext) throws -> [JSON] {
+ var result: [JSON] = []
+ for ordinal in ordinals {
+ try context.work()
+ var reversed: [JSONLocationComponent] = [], current = ordinal, scratch: UInt64 = 0
+ while let parent = try snapshot.record(at: current).parentOrdinal {
+ try context.work()
+ guard parent >= 0, parent < current,
+ let component = try snapshot.record(at: current).componentFromParent else {
+ throw JSONFailure(.configuration, code: .invalidOperation)
+ }
+ try context.retain(32); scratch += 32
+ if case .key(let key) = component {
+ for _ in key.string.utf8 { try context.work(); try context.retain(1); scratch += 1 }
+ }
+ reversed.append(component); current = parent
+ }
+ // Reuse the reserved reverse buffer; do not accumulate owned prefix copies.
+ for index in 0..<(reversed.count / 2) {
+ try context.work(); reversed.swapAt(index, reversed.count - index - 1)
+ }
+ let value = try context.selectedValue(at: reversed)
+ reversed = []; try context.release(scratch)
+ try context.retain(32); result.append(value); outputObserver?(result.count)
+ }
+ try context.checkpoint(); try context.release(requestAliases); return result
+ }
+}
diff --git a/Sources/JSONInspection/Store/JSONConfiguredStore.swift b/Sources/JSONInspection/Store/JSONConfiguredStore.swift
index b58a220..5c0428b 100644
--- a/Sources/JSONInspection/Store/JSONConfiguredStore.swift
+++ b/Sources/JSONInspection/Store/JSONConfiguredStore.swift
@@ -3,27 +3,53 @@ import JSON
import JSONNavigation
private final class JSONStoreOwner: Sendable {}
-package struct JSONStorePosition: Sendable {
+/// Unforgeable position bound to one immutable store instance, not a persistent ID.
+public struct JSONStorePosition: Sendable, Equatable {
fileprivate let owner: JSONStoreOwner
- package let collection: Int
- package let row: Int
+ /// Zero-based physical source collection position.
+ public let collection: Int
+ /// Zero-based physical row in that collection.
+ public let row: Int
+ /// Compares exact owner identity and fixed local ordinals; no JSON scan.
+ /// - Parameters:
+ /// - lhs: First owner-bound position.
+ /// - rhs: Second owner-bound position.
+ /// - Returns: True only for the same source instance and physical record.
+ public static func == (lhs: Self, rhs: Self) -> Bool {
+ lhs.owner === rhs.owner && lhs.collection == rhs.collection && lhs.row == rhs.row
+ }
}
-package enum JSONStoreReferenceState: Sendable {
+/// A declared reference slot; missing and explicit null remain distinct.
+public enum JSONStoreReferenceState: Sendable {
+ /// Optional field absent from the original record.
case absent
+ /// Explicit JSON null accepted by the field policy.
case null
+ /// A validated existing target in this same store.
+ /// - Parameter position: Exact owner-bound target position.
case link(JSONStorePosition)
}
-package struct JSONStoreRecordMetadata: Sendable {
- package let position: JSONStorePosition
- package let id: String
- package let path: [JSONLocationComponent]
- package let idPath: [JSONLocationComponent]
- package let referenceRange: Range
+/// Immutable source metadata; value resolution is a separately metered operation.
+public struct JSONStoreRecordMetadata: Sendable {
+ /// Exact instance-bound physical position.
+ public let position: JSONStorePosition
+ /// Original decoded-scalar string ID.
+ public let id: String
+ /// Absolute physical path of the record.
+ public let path: [JSONLocationComponent]
+ /// Absolute path of its configured ID member.
+ public let idPath: [JSONLocationComponent]
+ /// Half-open range into the containing store references array.
+ public let referenceRange: Range
}
-package struct JSONStoreReferenceMetadata: Sendable {
- package let field: Int
- package let path: [JSONLocationComponent]
- package let state: JSONStoreReferenceState
+/// Immutable configured-slot metadata in physical record and declaration order.
+public struct JSONStoreReferenceMetadata: Sendable {
+ /// Zero-based field declaration in the record collection profile.
+ public let field: Int
+ /// Exact absolute field path, including absent fields.
+ public let path: [JSONLocationComponent]
+ /// Absent, explicit null or a validated same-store link.
+ public let state: JSONStoreReferenceState
}
internal struct StoreCollectionMetadata: Sendable {
let profile: Int
@@ -31,25 +57,46 @@ internal struct StoreCollectionMetadata: Sendable {
let start: Int
let count: Int
}
-/// Package-private immutable complete view. No independent resolver or follow API.
-package struct JSONConfiguredStore: Sendable {
- package let document: JSONDocument
- package let profile: JSONStoreProfile
+/// Immutable completely admitted source, records and explicitly validated references.
+/// Construct through JSONStoreBuilder. No mutable store, I/O or persistent identity.
+public struct JSONConfiguredStore: Sendable {
+ /// Original complete canonical source, optionally with coherent source mapping.
+ public let document: JSONDocument
+ /// Immutable explicit collection/reference interpretation.
+ public let profile: JSONStoreProfile
private let owner: JSONStoreOwner
private let collections: [StoreCollectionMetadata]
private let records: [JSONStoreRecordMetadata]
- package let references: [JSONStoreReferenceMetadata]
+ /// All immutable configured slots, in physical record and field order.
+ public let references: [JSONStoreReferenceMetadata]
package let overlayLogicalBytes: UInt64
package let identityAliasBytes: UInt64
package let profileTextBytes: UInt64
- package var collectionCount: Int { collections.count }
- package var recordCount: Int { records.count }
- package func recordCount(in collection: Int) throws -> Int {
+ /// Number of complete physical collections.
+ public var collectionCount: Int { collections.count }
+ /// Total complete physical records.
+ public var recordCount: Int { records.count }
+ /// Fixed metadata access without a new owned operation.
+ /// - Parameter collection: Zero-based physical collection position.
+ /// - Returns: Complete record count in that collection.
+ /// - Throws: Invalid local collection position.
+ public func recordCount(in collection: Int) throws -> Int {
guard collection >= 0, collection < collections.count else {
throw JSONStoreFailure(.invalidPosition, phase: 0, domain: .operation)
}
return collections[collection].count
}
+ /// Reads a physical collection's exact name without scanning or opening a ledger.
+ /// - Parameter collection: Zero-based physical source collection position.
+ /// - Returns: Exact configured source name.
+ /// - Throws: Invalid collection position.
+ public func collectionName(at collection: Int) throws -> JSONKey {
+ guard collection >= 0, collection < collections.count else {
+ throw JSONStoreFailure(.invalidPosition, phase: 0, domain: .operation)
+ }
+ return profile.collections[collections[collection].profile].name
+ }
+
fileprivate init(document: JSONDocument, profile: JSONStoreProfile, owner: JSONStoreOwner,
collections: [StoreCollectionMetadata], records: [JSONStoreRecordMetadata],
references: [JSONStoreReferenceMetadata], overlay: UInt64, aliases: UInt64,
@@ -60,14 +107,23 @@ package struct JSONConfiguredStore: Sendable {
profileTextBytes = profileText
}
/// Checked fixed metadata access; positions are bound to this exact owner instance.
- package func record(collection: Int, row: Int) throws -> JSONStoreRecordMetadata {
+ /// - Parameters:
+ /// - collection: Physical collection position.
+ /// - row: Physical row position.
+ /// - Returns: Immutable metadata from this exact store.
+ /// - Throws: Invalid local position.
+ public func record(collection: Int, row: Int) throws -> JSONStoreRecordMetadata {
guard collection >= 0, collection < collections.count, row >= 0,
row < collections[collection].count else {
throw JSONStoreFailure(.invalidPosition, phase: 0, domain: .operation)
}
return records[collections[collection].start + row]
}
- package func record(at position: JSONStorePosition) throws -> JSONStoreRecordMetadata {
+ /// Reads fixed metadata with exact owner validation, without scanning JSON.
+ /// - Parameter position: Unforgeable position from this exact store instance.
+ /// - Returns: Original immutable record metadata.
+ /// - Throws: Foreign-owner or out-of-bounds position; never a null substitute.
+ public func record(at position: JSONStorePosition) throws -> JSONStoreRecordMetadata {
guard position.owner === owner else {
throw JSONStoreFailure(.invalidPosition, phase: 0, domain: .operation)
}
@@ -75,20 +131,35 @@ package struct JSONConfiguredStore: Sendable {
}
}
-package struct JSONStoreBuilder: Sendable {
- package let document: JSONDocument
- package let profile: JSONStoreProfile
- package let limits: JSONLimits
- package let caps: JSONStoreCaps
+/// Builds a complete immutable configured store using the original native ledger.
+/// Profile preparation, admission, all construction phases and publication share it.
+public struct JSONStoreBuilder: Sendable {
+ /// Original canonical source; admitted once during build.
+ public let document: JSONDocument
+ /// Closed typed configuration checked before source admission.
+ public let profile: JSONStoreProfile
+ /// Native cumulative operation quotas.
+ public let limits: JSONLimits
+ /// Additional inclusive store construction policies.
+ public let caps: JSONStoreCaps
package var checkpointObserver: (@Sendable (UInt64) -> Void)?
package var lifetimeObserver: (@Sendable (Bool, Bool) -> Void)?
/// Qualification-only phase rendezvous; arbitrary observer work is not library work.
package var phaseObserver: (@Sendable (Int) -> Void)?
- package init(document: JSONDocument, profile: JSONStoreProfile,
+ /// Captures trusted immutable inputs; validation occurs inside build.
+ /// - Parameters:
+ /// - document: Complete canonical source, optionally with mapping.
+ /// - profile: Explicit exact collection/ID/reference declarations.
+ /// - limits: Finite cumulative native quotas.
+ /// - caps: Additional finite store policies, not physical capacity promises.
+ public init(document: JSONDocument, profile: JSONStoreProfile,
limits: JSONLimits = .balanced, caps: JSONStoreCaps = .init()) {
self.document = document; self.profile = profile; self.limits = limits; self.caps = caps
}
- package func build() throws -> JSONOperationReport {
+ /// Completes all construction phases atomically or throws, with no partial store.
+ /// - Returns: Immutable admitted store and the one original operation report.
+ /// - Throws: Profile/source/capacity/core resource errors or cancellation.
+ public func build() throws -> JSONOperationReport {
try Task.checkCancellation()
try limits.validate()
try caps.validate()
diff --git a/Sources/JSONInspection/Store/JSONStoreProfile.swift b/Sources/JSONInspection/Store/JSONStoreProfile.swift
index d581aa3..0d5ffef 100644
--- a/Sources/JSONInspection/Store/JSONStoreProfile.swift
+++ b/Sources/JSONInspection/Store/JSONStoreProfile.swift
@@ -1,62 +1,126 @@
// SPDX-License-Identifier: Apache-2.0
import JSON
-package enum JSONStoreProfileVersion: String, Sendable { case v1 = "nav12.store-profile/1" }
-package enum JSONStoreMarkerVersion: String, Sendable { case v1 = "collection-id/1" }
-package enum JSONStoreFieldPresence: String, Sendable { case required, optional }
-package enum JSONStoreReferences: Sendable {
+/// Closed version of the typed configured-store construction contract.
+public enum JSONStoreProfileVersion: String, Sendable {
+ /// Exact collection arrays, explicit string identifiers and declared reference slots.
+ case v1 = "nav12.store-profile/1"
+}
+/// Closed reference-marker interpretation, independent of JSON Schema $ref.
+public enum JSONStoreMarkerVersion: String, Sendable {
+ /// An object containing exactly the string members collection and id.
+ case v1 = "collection-id/1"
+}
+/// Requiredness of one explicitly declared reference field.
+public enum JSONStoreFieldPresence: String, Sendable {
+ /// A missing member is a construction error.
+ case required
+ /// A missing member is retained as an absent slot.
+ case optional
+}
+/// Explicit reference interpretation; undeclared fields always remain ordinary JSON.
+public enum JSONStoreReferences: Sendable {
+ /// Interpret no fields as references.
case disabled
+ /// Validate exact markers and their existing targets before publishing a store.
+ /// - Parameters:
+ /// - marker: Closed version of the two-member marker grammar.
+ /// - validateTargets: Must be true; false is a typed profile refusal.
case strict(marker: JSONStoreMarkerVersion, validateTargets: Bool)
}
-package struct JSONStoreReferenceField: Sendable {
- package let key: JSONKey
- package let presence: JSONStoreFieldPresence
- package let nullable: Bool
- package init(key: JSONKey, presence: JSONStoreFieldPresence, nullable: Bool) {
+/// One configured direct reference field; no name or null policy is inferred.
+public struct JSONStoreReferenceField: Sendable {
+ /// Exact decoded-scalar field name.
+ public let key: JSONKey
+ /// Whether a missing member is accepted as an absent slot.
+ public let presence: JSONStoreFieldPresence
+ /// Whether explicit JSON null is accepted as a null slot.
+ public let nullable: Bool
+ /// Captures trusted typed configuration, checked on construction's one ledger.
+ /// - Parameters:
+ /// - key: Literal exact member name.
+ /// - presence: Required or optional field policy.
+ /// - nullable: Explicit null policy, independent of absence.
+ public init(key: JSONKey, presence: JSONStoreFieldPresence, nullable: Bool) {
self.key = key; self.presence = presence; self.nullable = nullable
}
}
-package struct JSONStoreCollectionProfile: Sendable {
- package let name: JSONKey
- package let recordsAt: [JSONLocationComponent]
- package let idKey: JSONKey
- package let referenceFields: [JSONStoreReferenceField]
- package init(name: JSONKey, recordsAt: [JSONLocationComponent], idKey: JSONKey,
- referenceFields: [JSONStoreReferenceField]) {
+/// A named direct array collection with an explicit exact string-ID field.
+public struct JSONStoreCollectionProfile: Sendable {
+ /// Exact collection name; canonical Unicode equivalence does not merge it.
+ public let name: JSONKey
+ /// Must be exactly one key component matching name inside collectionsAt.
+ public let recordsAt: [JSONLocationComponent]
+ /// Exact direct record member holding its string ID; no id/_id fallback.
+ public let idKey: JSONKey
+ /// Declared reference slots, interpreted in this order.
+ public let referenceFields: [JSONStoreReferenceField]
+ /// Captures an explicit collection declaration without scanning source data.
+ /// - Parameters:
+ /// - name: Exact direct collection name.
+ /// - recordsAt: One direct key equal to name; checked during preparation.
+ /// - idKey: Required exact string identifier member.
+ /// - referenceFields: Ordered explicit reference declarations, or [].
+ public init(name: JSONKey, recordsAt: [JSONLocationComponent], idKey: JSONKey,
+ referenceFields: [JSONStoreReferenceField]) {
self.name = name; self.recordsAt = recordsAt; self.idKey = idKey
self.referenceFields = referenceFields
}
}
-/// Closed native input, not a new JSON wire decoder or public configuration API.
-package struct JSONStoreProfile: Sendable {
- package let version: JSONStoreProfileVersion
- package let collectionsAt: [JSONLocationComponent]
- package let collections: [JSONStoreCollectionProfile]
- package let references: JSONStoreReferences
- package init(version: JSONStoreProfileVersion, collectionsAt: [JSONLocationComponent],
- collections: [JSONStoreCollectionProfile], references: JSONStoreReferences) {
+/// Closed native configuration, checked before source admission on the same ledger.
+/// It is neither a JSON wire-profile decoder nor an online resource resolver.
+public struct JSONStoreProfile: Sendable {
+ /// Closed construction contract version.
+ public let version: JSONStoreProfileVersion
+ /// Absolute exact typed selection of the collections object.
+ public let collectionsAt: [JSONLocationComponent]
+ /// Explicit collection declarations; unconfigured source collections are refused.
+ public let collections: [JSONStoreCollectionProfile]
+ /// Disabled or strict reference interpretation; no ambient reference behavior.
+ public let references: JSONStoreReferences
+ /// Captures immutable typed policy; construction performs bounded validation.
+ /// - Parameters:
+ /// - version: Explicit closed profile version.
+ /// - collectionsAt: Absolute collections-object selection, including [].
+ /// - collections: Exact declared direct collections.
+ /// - references: Explicit reference mode.
+ public init(version: JSONStoreProfileVersion, collectionsAt: [JSONLocationComponent],
+ collections: [JSONStoreCollectionProfile], references: JSONStoreReferences) {
self.version = version; self.collectionsAt = collectionsAt
self.collections = collections; self.references = references
}
}
-/// Finite prototype policy only. All maxima inclusive, inside unchanged core limits.
-package struct JSONStoreCaps: Sendable {
- package var collections = 256
- package var recordsPerCollection = 10_000
- package var records = 10_000
- package var referenceFieldsPerCollection = 16
- package var referenceFields = 4_096
- package var referenceSlots = 100_000
- package var links = 100_000
- package var identityBytes = 4_096
- package var profileTextBytes = 1_048_576
- package var aliasBytes = 16_777_216
- package var pathComponents = 128
- package var overlayBytes = 33_554_432
- package init() {}
+/// Finite inclusive construction policies within unchanged native limits.
+/// Maxima are not jointly achievable capacity or RSS/throughput guarantees.
+public struct JSONStoreCaps: Sendable {
+ /// Maximum configured/source collections.
+ public var collections = 256
+ /// Maximum records in each complete collection.
+ public var recordsPerCollection = 10_000
+ /// Maximum records across all collections.
+ public var records = 10_000
+ /// Maximum configured reference fields in each collection.
+ public var referenceFieldsPerCollection = 16
+ /// Maximum configured fields across all collections.
+ public var referenceFields = 4_096
+ /// Maximum interpreted slots, including absent and null slots.
+ public var referenceSlots = 100_000
+ /// Maximum staged links; repeated targets remain distinct slots.
+ public var links = 100_000
+ /// Maximum UTF-8 bytes in a collection or record identity.
+ public var identityBytes = 4_096
+ /// Maximum cumulative profile UTF-8 text bytes.
+ public var profileTextBytes = 1_048_576
+ /// Maximum live logical ID/marker text-alias bytes.
+ public var aliasBytes = 16_777_216
+ /// Maximum typed components in generated/configured paths.
+ public var pathComponents = 128
+ /// Maximum conservative logical metadata/scratch overlay bytes.
+ public var overlayBytes = 33_554_432
+ /// Uses finite policy defaults; zero is finite, negatives and Int.max invalid.
+ public init() {}
internal func validate() throws {
- // Fixed bootstrap size; no source/profile text scanned before native health.
for value in [collections, recordsPerCollection, records, referenceFieldsPerCollection,
referenceFields, referenceSlots, links, identityBytes, profileTextBytes,
aliasBytes, pathComponents, overlayBytes] {
@@ -65,28 +129,98 @@ package struct JSONStoreCaps: Sendable {
}
}
-package enum JSONStoreFailureCode: String, Sendable {
- case invalidTypedComponent, duplicateCollectionDeclaration, invalidRecordsAt
- case duplicateReferenceField, idReferenceFieldCollision, disabledReferencesConflict
- case invalidReferenceMode, collectionsNotObject, unconfiguredCollection, collectionNotArray
- case missingConfiguredCollection, recordWrongKind, recordIDMissing, recordIDNull, recordIDWrongKind
- case duplicateRecordID, referenceMissing, referenceNull, referenceMarkerWrongKind
- case referenceMarkerMissing, referenceMarkerMemberWrongKind, referenceMarkerExtra
- case referenceUnknownCollection, referenceDanglingID, selectionMissing, selectionWrongKind
- case capacityExceeded, counterOverflow, invalidPosition
+/// Stable typed construction/selection refusals, separate from native quota errors.
+public enum JSONStoreFailureCode: String, Sendable {
+ /// A configured typed index is negative.
+ case invalidTypedComponent
+ /// Two profile collection names have identical decoded scalars.
+ case duplicateCollectionDeclaration
+ /// A collection selector is not its one exact direct name.
+ case invalidRecordsAt
+ /// Two configured fields have identical decoded scalars.
+ case duplicateReferenceField
+ /// An ID member is also declared as a reference.
+ case idReferenceFieldCollision
+ /// Reference declarations conflict with disabled references.
+ case disabledReferencesConflict
+ /// Strict mode attempted to disable target validation.
+ case invalidReferenceMode
+ /// The selected collections value is not an object.
+ case collectionsNotObject
+ /// A source collection has no explicit declaration.
+ case unconfiguredCollection
+ /// A declared collection value is not an array.
+ case collectionNotArray
+ /// A declared collection is absent from the source.
+ case missingConfiguredCollection
+ /// A physical record is not an object.
+ case recordWrongKind
+ /// The required ID member is absent.
+ case recordIDMissing
+ /// The required ID is explicit null.
+ case recordIDNull
+ /// The ID is present with a non-string kind.
+ case recordIDWrongKind
+ /// The second exact duplicate ID retains its first ID path.
+ case duplicateRecordID
+ /// A required declared reference field is absent.
+ case referenceMissing
+ /// A declared reference field forbids explicit null.
+ case referenceNull
+ /// A non-null marker is not an object.
+ case referenceMarkerWrongKind
+ /// An exact collection/id marker member is missing.
+ case referenceMarkerMissing
+ /// A marker member has a non-string kind.
+ case referenceMarkerMemberWrongKind
+ /// A marker has an undeclared extra member.
+ case referenceMarkerExtra
+ /// A marker names an unknown exact collection.
+ case referenceUnknownCollection
+ /// A marker names a missing ID in an existing collection.
+ case referenceDanglingID
+ /// The collections selection is missing; coreCause retains the original error.
+ case selectionMissing
+ /// The collections selection crosses a wrong kind.
+ case selectionWrongKind
+ /// A finite store construction or follow policy was exceeded.
+ case capacityExceeded
+ /// Checked size/counter arithmetic overflowed.
+ case counterOverflow
+ /// A local position is out of bounds or belongs to another store instance.
+ case invalidPosition
+ /// A lookup request names no declared physical collection.
+ case unknownCollection
+}
+/// The authority that refused construction or a read-only operation.
+public enum JSONStoreFailureDomain: String, Sendable {
+ /// Invalid typed configuration, before source admission.
+ case profile
+ /// Invalid source structure or declared references.
+ case source
+ /// Invalid local position, query policy or checked capacity.
+ case operation
}
-package enum JSONStoreFailureDomain: String, Sendable { case profile, source, operation }
-/// Source path and core cause are separate; missing targets have no invented span.
-package struct JSONStoreFailure: Error, Sendable {
- package let code: JSONStoreFailureCode
- package let phase: Int
- package let domain: JSONStoreFailureDomain
- package let path: [JSONLocationComponent]
- package let firstPath: [JSONLocationComponent]?
- package let sourceSpan: JSONSourceSpan?
- package let coreCause: JSONFailure?
- package let capacity: String?
- package let limit: Int?
+/// Precise checked refusal; no partial store or null substitute is published.
+public struct JSONStoreFailure: Error, Sendable {
+ /// Stable refusal identity.
+ public let code: JSONStoreFailureCode
+ /// Construction phases1...8; ordinary metadata0 and owned queries9.
+ public let phase: Int
+ /// Profile, source or operation authority.
+ public let domain: JSONStoreFailureDomain
+ /// Exact source/request path, empty when no location applies.
+ public let path: [JSONLocationComponent]
+ /// First occurrence for a duplicate ID; never a normalized key.
+ public let firstPath: [JSONLocationComponent]?
+ /// Coherent original span, or nil when unavailable.
+ public let sourceSpan: JSONSourceSpan?
+ /// Original core selection failure without loss of its resource/location metadata.
+ public let coreCause: JSONFailure?
+ /// Named construction/follow capacity for a capacity refusal.
+ public let capacity: String?
+ /// Inclusive configured capacity, distinct from physical memory.
+ public let limit: Int?
internal init(_ code: JSONStoreFailureCode, phase: Int, domain: JSONStoreFailureDomain,
path: [JSONLocationComponent] = [], firstPath: [JSONLocationComponent]? = nil,
sourceSpan: JSONSourceSpan? = nil, coreCause: JSONFailure? = nil,
diff --git a/Sources/JSONInspection/Store/JSONStoreQueries.swift b/Sources/JSONInspection/Store/JSONStoreQueries.swift
new file mode 100644
index 0000000..0055990
--- /dev/null
+++ b/Sources/JSONInspection/Store/JSONStoreQueries.swift
@@ -0,0 +1,234 @@
+// SPDX-License-Identifier: Apache-2.0
+import JSON
+import JSONNavigation
+
+/// Inclusive finite policies for a complete reachable closure, not truncation.
+public struct JSONStoreFollowLimits: Sendable {
+ /// Maximum shortest link distance from the start; start has depth zero.
+ public var maximumDepth: Int = 64
+ /// Maximum pending BFS positions, excluding already consumed queue entries.
+ public var maximumFrontier: Int = 10_000
+ /// Maximum distinct visited records, including the start.
+ public var maximumVisited: Int = 10_000
+ /// Maximum returned positions, including the start.
+ public var maximumResults: Int = 10_000
+ /// Uses finite defaults; zero is valid, negative values and Int.max are invalid.
+ public init() {}
+ internal func validate() throws {
+ for value in [maximumDepth, maximumFrontier, maximumVisited, maximumResults] {
+ guard value >= 0, value < Int.max else { throw JSONFailure(.configuration, code: .invalidLimits) }
+ }
+ }
+}
+
+extension JSONConfiguredStore {
+ /// Looks up an exact collection/string ID in one independent admitted operation.
+ /// No canonical normalization or id/_id fallback. Missing IDs return nil;
+ /// an unknown collection is a distinct typed refusal. Metadata scans and text
+ /// comparisons use the same source-admission ledger; no index speed is promised.
+ /// - Parameters:
+ /// - collection: Exact decoded-scalar configured collection name.
+ /// - id: Exact decoded-scalar string record identifier.
+ /// - limits: Finite cumulative native quotas for this lookup.
+ /// - Returns: Owner-bound position or nil, plus cumulative statistics.
+ /// - Throws: Unknown collection, native configuration/resource failure or cancellation.
+ public func lookup(collection: JSONKey, id: String, limits: JSONLimits = .balanced)
+ throws -> JSONOperationReport {
+ let report = try query(.lookup(collection, id), limits: limits)
+ guard case .position(let result) = report.result else { throw JSONFailure(.configuration, code: .invalidOperation) }
+ return .init(result: result, statistics: report.statistics)
+ }
+
+ /// Reads exact values in request order, preserving repeated positions.
+ /// One source admission, metadata reservation and ledger cover the entire batch.
+ /// Empty requests still admit the whole source; foreign tokens never become null.
+ /// Result aliases follow ordinary native read semantics and caller-copy boundaries.
+ /// - Parameters:
+ /// - positions: Unforgeable same-store record positions.
+ /// - maximumRequests: Inclusive finite request/result cap, default1024.
+ /// - limits: Finite native quotas for the whole independent batch.
+ /// - Returns: All canonical record objects and cumulative statistics.
+ /// - Throws: Position/batch/core resource/configuration failure or cancellation.
+ public func values(at positions: [JSONStorePosition], maximumRequests: Int = 1_024,
+ limits: JSONLimits = .balanced) throws -> JSONOperationReport<[JSON]> {
+ try checkInspectionBatch(positions.count, maximum: maximumRequests)
+ let report = try query(.values(positions), limits: limits)
+ guard case .values(let result) = report.result else { throw JSONFailure(.configuration, code: .invalidOperation) }
+ return .init(result: result, statistics: report.statistics)
+ }
+
+ /// Resolves one same-store position through the one-request batch delegate.
+ /// - Parameters:
+ /// - position: Unforgeable same-store physical record position.
+ /// - limits: Finite native quotas for this independent admitted read.
+ /// - Returns: The original canonical record and actual operation statistics.
+ /// - Throws: Invalid position, core resource/configuration failure or cancellation.
+ public func value(at position: JSONStorePosition, limits: JSONLimits = .balanced)
+ throws -> JSONOperationReport {
+ let report = try values(at: [position], maximumRequests: 1, limits: limits)
+ return .init(result: report.result[0], statistics: report.statistics)
+ }
+
+ /// Returns the complete BFS reachable closure, including start exactly once.
+ /// Only declared validated link slots are edges. Absent/null slots add no edge;
+ /// cycles are deduplicated by physical positions. Any exceeded depth/frontier/
+ /// visited/result cap throws instead of publishing a truncated closure. No I/O.
+ /// - Parameters:
+ /// - start: Owner-bound starting record in this immutable store.
+ /// - bounds: Inclusive finite BFS policies, including a finite depth ceiling.
+ /// - limits: Finite native quotas shared by admission, aliases, scans and output.
+ /// - Returns: Deterministic positions in configured-field BFS order and statistics.
+ /// - Throws: Invalid position, typed capacity/native resource failure or cancellation.
+ public func follow(from start: JSONStorePosition, bounds: JSONStoreFollowLimits = .init(),
+ limits: JSONLimits = .balanced) throws -> JSONOperationReport<[JSONStorePosition]> {
+ try Task.checkCancellation(); try bounds.validate()
+ let report = try query(.follow(start, bounds), limits: limits)
+ guard case .positions(let result) = report.result else { throw JSONFailure(.configuration, code: .invalidOperation) }
+ return .init(result: result, statistics: report.statistics)
+ }
+
+ package func followForQualification(from start: JSONStorePosition, bounds: JSONStoreFollowLimits,
+ limits: JSONLimits, checkpointObserver: @Sendable @escaping (UInt64) -> Void,
+ lifetimeObserver: @Sendable @escaping (Bool, Bool) -> Void,
+ edgeObserver: (@Sendable (Int) -> Void)? = nil) throws -> JSONOperationReport<[JSONStorePosition]> {
+ try bounds.validate()
+ let report = try query(.follow(start, bounds), limits: limits,
+ checkpointObserver: checkpointObserver, lifetimeObserver: lifetimeObserver, edgeObserver: edgeObserver)
+ guard case .positions(let result) = report.result else { throw JSONFailure(.configuration, code: .invalidOperation) }
+ return .init(result: result, statistics: report.statistics)
+ }
+
+ private func query(_ operation: StoreReadOperation, limits: JSONLimits,
+ checkpointObserver: (@Sendable (UInt64) -> Void)? = nil,
+ lifetimeObserver: (@Sendable (Bool, Bool) -> Void)? = nil,
+ edgeObserver: (@Sendable (Int) -> Void)? = nil)
+ throws -> JSONOperationReport {
+ var reducer = StoreReadReducer(store: self, operation: operation, edgeObserver: edgeObserver)
+ var traversal = JSONTraversal(document: document, limits: limits)
+ traversal.checkpointObserver = checkpointObserver
+ traversal.lifetimeObserver = lifetimeObserver
+ return try traversal.inspectPreparedOwned(options: .init(depthLimit: 0), reducer: &reducer)
+ }
+}
+
+private enum StoreReadOperation {
+ case lookup(JSONKey, String)
+ case values([JSONStorePosition])
+ case follow(JSONStorePosition, JSONStoreFollowLimits)
+}
+private enum StoreReadResult {
+ case position(JSONStorePosition?)
+ case values([JSON])
+ case positions([JSONStorePosition])
+}
+
+private struct StoreReadReducer: JSONReadPreparation, JSONTraversalOwnedReducer {
+ let store: JSONConfiguredStore
+ let operation: StoreReadOperation
+ let edgeObserver: (@Sendable (Int) -> Void)?
+ private var inputAliases: UInt64 = 0
+ init(store: JSONConfiguredStore, operation: StoreReadOperation, edgeObserver: (@Sendable (Int) -> Void)?) {
+ self.store = store; self.operation = operation; self.edgeObserver = edgeObserver
+ }
+ mutating func prepare(context: inout JSONReadPreparationContext) throws {
+ inputAliases = try JSONStoreAccounting.sum(store.overlayLogicalBytes, 128)
+ try context.retain(inputAliases)
+ switch operation {
+ case .lookup(let collection, let id):
+ try context.admitText(collection.string, key: true); try context.admitText(id)
+ for text in [collection.string, id] {
+ for _ in text.utf8 { try context.work(); inputAliases = try JSONStoreAccounting.sum(inputAliases, 1) }
+ }
+ case .values(let positions):
+ for position in positions {
+ try context.work(); _ = try store.record(at: position)
+ try context.retain(8); inputAliases = try JSONStoreAccounting.sum(inputAliases, 8)
+ }
+ case .follow(let start, _):
+ try context.work(); _ = try store.record(at: start)
+ }
+ }
+ mutating func observe(_ hit: JSONTraversalHit, value: JSON,
+ context: inout JSONTraversalContext) throws -> JSONTraversalAction {
+ try context.work(); return .skipDescendants
+ }
+ mutating func finish(_ summary: JSONTraversalSummary, context: inout JSONTraversalContext) throws -> StoreReadResult {
+ let result: StoreReadResult
+ switch operation {
+ case .lookup(let collection, let id): result = .position(try lookup(collection, id: id, context: &context))
+ case .values(let positions):
+ var values: [JSON] = []; try context.retain(32)
+ for position in positions {
+ try context.work()
+ let record = try store.record(at: position)
+ let value = try context.selectedValue(at: record.path)
+ try context.retain(32); values.append(value)
+ }
+ result = .values(values)
+ case .follow(let start, let bounds): result = .positions(try follow(start, bounds: bounds, context: &context))
+ }
+ try context.checkpoint(); try context.release(inputAliases)
+ return result
+ }
+ private func lookup(_ name: JSONKey, id: String, context: inout JSONTraversalContext) throws -> JSONStorePosition? {
+ try context.retain(32)
+ for collection in 0.. Int {
+ try context.work(); _ = try store.record(at: position)
+ var offset: UInt64 = 0
+ for collection in 0.. [JSONStorePosition] {
+ var visited: [Bool] = [], result: [JSONStorePosition] = [], depths: [Int] = []
+ try context.retain(32)
+ for _ in 0.. JSONStoreFailure {
+ .init(.capacityExceeded, phase: 9, domain: .operation, capacity: name, limit: limit)
+ }
+ func enqueue(_ position: JSONStorePosition, depth: Int, context: inout JSONTraversalContext) throws {
+ try context.work()
+ guard depth <= bounds.maximumDepth else { throw refuse("maximumDepth", bounds.maximumDepth) }
+ guard result.count - head < bounds.maximumFrontier else { throw refuse("maximumFrontier", bounds.maximumFrontier) }
+ guard result.count < bounds.maximumVisited else { throw refuse("maximumVisited", bounds.maximumVisited) }
+ guard result.count < bounds.maximumResults else { throw refuse("maximumResults", bounds.maximumResults) }
+ let index = try flatIndex(position, context: &context)
+ try context.retain(40); visited[index] = true; result.append(position); depths.append(depth)
+ }
+ try enqueue(start, depth: 0, context: &context)
+ while head < result.count {
+ try context.work()
+ let record = try store.record(at: result[head]), depth = depths[head]; head += 1
+ for slot in record.referenceRange {
+ try context.work()
+ if case .link(let target) = store.references[slot].state {
+ let index = try flatIndex(target, context: &context)
+ if !visited[index] {
+ guard depth < bounds.maximumDepth else { throw refuse("maximumDepth", bounds.maximumDepth) }
+ try enqueue(target, depth: depth + 1, context: &context)
+ }
+ }
+ edges += 1; edgeObserver?(edges)
+ try context.checkpoint()
+ }
+ }
+ let scratch = try JSONStoreAccounting.sum(UInt64(visited.count), JSONStoreAccounting.product(UInt64(depths.count), 8))
+ visited = []; depths = []; try context.release(scratch)
+ return result
+ }
+}
diff --git a/Sources/JSONNavigation/JSONTraversalContext.swift b/Sources/JSONNavigation/JSONTraversalContext.swift
index d1c986e..7bf4fff 100644
--- a/Sources/JSONNavigation/JSONTraversalContext.swift
+++ b/Sources/JSONNavigation/JSONTraversalContext.swift
@@ -124,6 +124,21 @@ package struct JSONTraversalContext: ~Copyable {
try scope.document?.sourceMap?.span(at: path, kind: .value, meter: &state.meter)
}
}
+ /// Resolve a typed selection on this original admitted scope; no new ledger.
+ /// The value is an ephemeral native alias; returned owned slots need reservation.
+ package mutating func selectedValue(at path: [JSONLocationComponent]) throws -> JSON {
+ try perform(at: path) { state in
+ // A metadata path is not automatically admitted by this invocation.
+ // Use the original scope's checked path constructor before selection.
+ var checked: [JSONLocationComponent] = []
+ for component in path {
+ checked = try scope.append(component, to: checked, state: state).components
+ }
+ try state.meter.work()
+ return try scope.select(checked, state: state)
+ }
+ }
+
/// Transfer already-admitted input ownership, without exposing its scope.
package mutating func transferDocumentOwnership() throws -> JSONDocument {
try perform { _ in try scope.transferDocumentOwnership() }
diff --git a/Tests/JSONCLITests/IntegrationTests.swift b/Tests/JSONCLITests/IntegrationTests.swift
index 5157667..12ac2a7 100644
--- a/Tests/JSONCLITests/IntegrationTests.swift
+++ b/Tests/JSONCLITests/IntegrationTests.swift
@@ -146,7 +146,7 @@ private func integrationDirectory() throws -> URL {
@Test func cliVersionIsDeterministicAndNeedsNoRepositoryOrInput() throws {
let metadata = try JSONParser().parse(CLIHelp.versionJSON)
- #expect(try metadata.requiredMember("version") == .string("1.7.0"))
+ #expect(try metadata.requiredMember("version") == .string("1.8.0-dev"))
#expect(try metadata.requiredMember("revisionKind") == .string("source-sha256"))
let revision = try #require(try metadata.requiredMember("revision").stringValue)
#expect(revision.hasPrefix("sha256:") && revision.utf8.count == 71)
diff --git a/Tests/JSONInspectionTests/WorkflowTests.swift b/Tests/JSONInspectionTests/WorkflowTests.swift
new file mode 100644
index 0000000..82bf917
--- /dev/null
+++ b/Tests/JSONInspectionTests/WorkflowTests.swift
@@ -0,0 +1,309 @@
+// SPDX-License-Identifier: Apache-2.0
+import Foundation
+import Synchronization
+import Testing
+import JSON
+import JSONNavigation
+@testable import JSONInspection
+
+private enum WorkflowTestError: Error { case expectedFailure }
+private func wfCore(_ body: () throws -> Void) throws -> JSONFailure {
+ do { try body(); Issue.record("Expected core failure"); throw WorkflowTestError.expectedFailure }
+ catch let error as JSONFailure { return error }
+}
+private func wfStore(_ body: () throws -> Void) throws -> JSONStoreFailure {
+ do { try body(); Issue.record("Expected store failure"); throw WorkflowTestError.expectedFailure }
+ catch let error as JSONStoreFailure { return error }
+}
+private func wfDocument(_ text: String) throws -> JSONDocument {
+ try JSONParser().parseDocument(Data(text.utf8))
+}
+private func wfStoreFixture() throws -> JSONConfiguredStore {
+ let document = try wfDocument(#"{"collections":{"users":[{"id":"é","count":9007199254740993,"friend":{"collection":"users","id":"e\u0301"},"optional":null},{"id":"e\u0301","friend":{"collection":"users","id":"é"}}],"items":[{"id":"x","owner":{"collection":"users","id":"é"}}]}}"#)
+ let profile = JSONStoreProfile(version: .v1, collectionsAt: [.key("collections")], collections: [
+ .init(name: "items", recordsAt: [.key("items")], idKey: "id", referenceFields: [.init(key: "owner", presence: .required, nullable: false)]),
+ .init(name: "users", recordsAt: [.key("users")], idKey: "id", referenceFields: [
+ .init(key: "friend", presence: .required, nullable: false), .init(key: "optional", presence: .optional, nullable: true)])],
+ references: .strict(marker: .v1, validateTargets: true))
+ return try JSONStoreBuilder(document: document, profile: profile).build().result
+}
+private func wfStar() throws -> JSONConfiguredStore {
+ let document = try wfDocument(#"{"u":[{"id":"0","left":{"collection":"u","id":"1"},"right":{"collection":"u","id":"2"}},{"id":"1"},{"id":"2"}]}"#)
+ return try JSONStoreBuilder(document: document, profile: .init(version: .v1, collectionsAt: [], collections: [
+ .init(name: "u", recordsAt: [.key("u")], idKey: "id", referenceFields: [
+ .init(key: "left", presence: .optional, nullable: true), .init(key: "right", presence: .optional, nullable: true)])],
+ references: .strict(marker: .v1, validateTargets: true))).build().result
+}
+
+@Test func workflowB01PathsPreserveOrderRepeatsNullAndExactScalars() throws {
+ let document = try wfDocument(#"{"é":9007199254740993,"e\u0301":null,"a/b~":0.1}"#)
+ let report = try JSONInspectionBatch(document: document).values(at: [[.key("e\u{301}")], [.key("é")], [.key("e\u{301}")], [.key("a/b~")]])
+ #expect(report.result == [.null, .number(try JSONNumber(parsing: "9007199254740993")), .null, .number(try JSONNumber(parsing: "0.1"))])
+ #expect(report.statistics[.workUnits] > 0)
+}
+@Test func workflowB02OneCompleteAdmissionAndEmptyBatch() throws {
+ let document = try wfDocument("[1,2,3]")
+ let base = try JSONTraversal(document: document).visit { _,_ in .skipDescendants }
+ let batch = try JSONInspectionBatch(document: document).values(at: [[.index(1)], [.index(1)]])
+ #expect(batch.statistics[.nodes] == base.statistics[.nodes])
+ #expect(batch.statistics[.sourceMapEntries] == base.statistics[.sourceMapEntries])
+ let empty = try JSONInspectionBatch(document: document).values(at: [], maximumRequests: 0)
+ #expect(empty.result.isEmpty && empty.statistics[.nodes] == base.statistics[.nodes])
+ var limits = JSONLimits.balanced; limits[.nodes] = 0
+ #expect(try wfCore { _ = try JSONInspectionBatch(document: document, limits: limits).values(at: []) }.resource?.metric == .nodes)
+}
+@Test func workflowB03RequestCapacityConfigurationAndExactBoundary() throws {
+ let batch = JSONInspectionBatch(document: .init(root: .null))
+ #expect(try batch.values(at: [[],[]], maximumRequests: 2).result == [.null,.null])
+ do { _ = try batch.values(at: [[],[]], maximumRequests: 1); Issue.record("Expected batch refusal") }
+ catch let error as JSONInspectionBatchFailure { #expect(error == .requestLimitExceeded(limit: 1, actual: 2)) }
+ for invalid in [-1, Int.max] {
+ #expect(try wfCore { _ = try batch.values(at: [], maximumRequests: invalid) }.code == .invalidLimits)
+ }
+}
+@Test func workflowB04MissingAndWrongKindRetainTypedPaths() throws {
+ let batch = JSONInspectionBatch(document: try wfDocument(#"{"a":null}"#))
+ for (path, category, code) in [([JSONLocationComponent.key("missing")], JSONFailure.Category.missingValue, JSONFailure.Code.missingMember),
+ ([.key("a"),.index(0)], .typeMismatch, .wrongKind)] {
+ do { _ = try batch.values(at: [[],path]); Issue.record("Expected atomic access refusal") }
+ catch let error as JSONAccessFailure {
+ #expect(error.path == path && error.cause.category == category && error.cause.code == code)
+ #expect(error.cause.location != nil)
+ }
+ }
+}
+@Test func workflowB05UnselectedSuffixAndCumulativeWorkCannotReset() throws {
+ let document = try wfDocument(#"{"selected":null,"outside":[1,2,3,4]}"#)
+ var limits = JSONLimits.balanced; limits[.nodes] = 2
+ #expect(try wfCore { _ = try JSONInspectionBatch(document: document, limits: limits).values(at: [[.key("selected")]]) }.resource?.metric == .nodes)
+ let paths = Array(repeating: [JSONLocationComponent.key("selected")], count: 8)
+ let measured = try JSONInspectionBatch(document: document).values(at: paths)
+ limits = .balanced; limits[.workUnits] = measured.statistics[.workUnits]
+ #expect(try JSONInspectionBatch(document: document, limits: limits).values(at: paths).result.count == 8)
+ limits[.workUnits] -= 1
+ let failure = try wfCore { _ = try JSONInspectionBatch(document: document, limits: limits).values(at: paths) }
+ #expect(failure.resource?.metric == .workUnits && failure.resource?.limit == limits[.workUnits])
+}
+@Test func workflowB06SnapshotBatchKeepsOneSourceAndOrdinalMeaning() throws {
+ let snapshot = try JSONStructuralSnapshotBuilder(document: wfDocument("[9,8,null]")).build().result
+ let batch = try snapshot.values(for: [2,1,3,2])
+ #expect(batch.result == [8,9,.null,8])
+ let one = try snapshot.value(for: 2)
+ #expect(batch.statistics[.nodes] == one.statistics[.nodes])
+ var limits = JSONLimits.balanced; limits[.nodes] = 0
+ do { _ = try snapshot.values(for: [1,99], limits: limits); Issue.record("Expected bad ordinal") }
+ catch let error as JSONStructuralSnapshotFailure { #expect(error == .ordinalOutOfBounds(99)) }
+}
+@Test func workflowB07InnerCancellationInvalidatesOriginalScope() async throws {
+ let document = try wfDocument("[1,2,3]")
+ let lifetime = Mutex(false), outputs = Mutex(0)
+ let task = Task {
+ var limits = JSONLimits.balanced; limits[.cancellationQuantum] = 1
+ var batch = JSONInspectionBatch(document: document, limits: limits)
+ batch.lifetimeObserver = { root, state in lifetime.withLock { $0 = root && state } }
+ batch.outputObserver = { count in
+ outputs.withLock { $0 = count }
+ if count == 2 { withUnsafeCurrentTask { $0?.cancel() } }
+ }
+ return try batch.values(at: Array(repeating: [.index(0)], count: 20))
+ }
+ do { _ = try await task.value; Issue.record("Expected cancellation") } catch is CancellationError {}
+ #expect(lifetime.withLock { $0 })
+ #expect(outputs.withLock { $0 } == 2)
+}
+@Test func workflowB08RetentionBoundaryIsFiniteAndSourceIsUnchanged() throws {
+ let document = try wfDocument("[1,2]")
+ let report = try JSONInspectionBatch(document: document).values(at: [[],[.index(1)]])
+ var limits = JSONLimits.balanced; limits[.retainedBytes] = report.statistics.peakLogicalRetainedBytes
+ _ = try JSONInspectionBatch(document: document, limits: limits).values(at: [[],[.index(1)]])
+ limits[.retainedBytes] -= 1
+ #expect(try wfCore { _ = try JSONInspectionBatch(document: document, limits: limits).values(at: [[],[.index(1)]]) }.resource?.metric == .retainedBytes)
+ #expect(document.root == [1,2] && document.sourceMap != nil)
+}
+@Test func workflowS01PublicStorePreservesPhysicalAndDeclaredOrder() throws {
+ let store = try wfStoreFixture()
+ #expect(store.collectionCount == 2 && store.recordCount == 3)
+ #expect(try store.collectionName(at: 0) == "users")
+ #expect(try store.collectionName(at: 1) == "items")
+ #expect(try store.recordCount(in: 0) == 2 && store.references.count == 5)
+ #expect(try store.record(collection: 0, row: 0).idPath == [.key("collections"),.key("users"),.index(0),.key("id")])
+}
+@Test func workflowS02LookupUsesDecodedScalarIDsAndDistinguishesMissing() throws {
+ let store = try wfStoreFixture()
+ let composed = try #require(store.lookup(collection: "users", id: "é").result)
+ let decomposed = try #require(store.lookup(collection: "users", id: "e\u{301}").result)
+ #expect(composed.row == 0 && decomposed.row == 1 && composed != decomposed)
+ #expect(try store.lookup(collection: "users", id: "absent").result == nil)
+ let error = try wfStore { _ = try store.lookup(collection: "unknown", id: "é") }
+ #expect(error.code == .unknownCollection && error.domain == .operation && error.phase == 9)
+}
+@Test func workflowS03OwnerTokensCannotRebindAnotherInstance() throws {
+ let store = try wfStoreFixture(), other = try JSONStoreBuilder(document: store.document, profile: store.profile).build().result
+ let position = try store.record(collection: 0, row: 0).position
+ #expect(position != (try other.record(collection: 0, row: 0).position))
+ #expect(try wfStore { _ = try other.values(at: [position]) }.code == .invalidPosition)
+ #expect(try wfStore { _ = try other.follow(from: position) }.code == .invalidPosition)
+}
+@Test func workflowS04StoreBatchValuesPreserveOrderExactnessAndOneAdmission() throws {
+ let store = try wfStoreFixture(), a = try store.record(collection: 0, row: 0).position, b = try store.record(collection: 0, row: 1).position
+ let report = try store.values(at: [b,a,b])
+ #expect(try report.result[0].requiredMember("id") == .string("e\u{301}"))
+ #expect(try report.result[1].requiredMember("count") == .number(JSONNumber(parsing: "9007199254740993")))
+ #expect(report.result[0] == report.result[2])
+ let source = try JSONTraversal(document: store.document).visit { _,_ in .skipDescendants }
+ #expect(report.statistics[.nodes] == source.statistics[.nodes])
+ #expect(try store.value(at: a).result == report.result[1])
+ #expect(try store.values(at: [], maximumRequests: 0).statistics[.nodes] == source.statistics[.nodes])
+}
+@Test func workflowS05ReferenceSlotsKeepAbsentNullAndLinkDistinct() throws {
+ let store = try wfStoreFixture()
+ if case .link(let p) = store.references[0].state { #expect(p.row == 1) } else { Issue.record("Expected link") }
+ if case .null = store.references[1].state {} else { Issue.record("Expected null") }
+ if case .absent = store.references[3].state {} else { Issue.record("Expected absence") }
+}
+@Test func workflowS06FollowCompletesCyclesAndPreservesBFSOrder() throws {
+ let store = try wfStoreFixture(), start = try store.record(collection: 1, row: 0).position
+ let report = try store.follow(from: start)
+ #expect(report.result.map(\.collection) == [1,0,0] && report.result.map(\.row) == [0,0,1])
+ let star = try wfStar(), root = try star.record(collection: 0, row: 0).position
+ #expect(try star.follow(from: root).result.map(\.row) == [0,1,2])
+}
+@Test func workflowS07FollowAllFourInclusiveBoundsThrowInsteadOfTruncate() throws {
+ let store = try wfStar(), start = try store.record(collection: 0, row: 0).position
+ let cases: [(String, WritableKeyPath, Int)] = [
+ ("maximumDepth",\JSONStoreFollowLimits.maximumDepth,1), ("maximumFrontier",\JSONStoreFollowLimits.maximumFrontier,2),
+ ("maximumVisited",\JSONStoreFollowLimits.maximumVisited,3), ("maximumResults",\JSONStoreFollowLimits.maximumResults,3)]
+ for (name,key,needed) in cases {
+ var bounds = JSONStoreFollowLimits(); bounds[keyPath: key] = needed
+ #expect(try store.follow(from: start, bounds: bounds).result.count == 3)
+ bounds[keyPath: key] = needed - 1
+ let failure = try wfStore { _ = try store.follow(from: start, bounds: bounds) }
+ #expect(failure.code == .capacityExceeded && failure.capacity == name && failure.limit == needed - 1)
+ for invalid in [-1, Int.max] {
+ bounds[keyPath: key] = invalid
+ #expect(try wfCore { _ = try store.follow(from: start, bounds: bounds) }.code == .invalidLimits)
+ }
+ }
+}
+@Test func workflowS08FollowAndLookupHaveCumulativeQuotaAndNoSecondAdmission() throws {
+ let store = try wfStar(), start = try store.record(collection: 0, row: 0).position
+ let report = try store.follow(from: start)
+ let lookup = try store.lookup(collection: "u", id: "2")
+ #expect(report.statistics[.nodes] == lookup.statistics[.nodes])
+ for work in [report.statistics[.workUnits], lookup.statistics[.workUnits]] { #expect(work > 1) }
+ var limits = JSONLimits.balanced; limits[.workUnits] = report.statistics[.workUnits]
+ #expect(try store.follow(from: start, limits: limits).result.count == 3)
+ limits[.workUnits] -= 1
+ #expect(try wfCore { _ = try store.follow(from: start, limits: limits) }.resource?.metric == .workUnits)
+}
+@Test func workflowS09FollowInnerCancellationAndInvalidation() async throws {
+ let store = try wfStar(), start = try store.record(collection: 0, row: 0).position
+ let lifetime = Mutex(false), edges = Mutex(0)
+ let task = Task {
+ var limits = JSONLimits.balanced; limits[.cancellationQuantum] = 1
+ return try store.followForQualification(from: start, bounds: .init(), limits: limits,
+ checkpointObserver: { _ in }, lifetimeObserver: { root, state in lifetime.withLock { $0 = root && state } },
+ edgeObserver: { count in
+ edges.withLock { $0 = count }
+ if count == 1 { withUnsafeCurrentTask { $0?.cancel() } }
+ })
+ }
+ do { _ = try await task.value; Issue.record("Expected inner cancellation") } catch is CancellationError {}
+ #expect(edges.withLock { $0 } == 1 && lifetime.withLock { $0 })
+ #expect(try store.follow(from: start).result.count == 3)
+}
+@Test func workflowS10SourceReplacementCannotInventSpansOrRebindPositions() throws {
+ let store = try wfStoreFixture(), position = try store.record(collection: 0, row: 0).position
+ let copy = try JSONStoreBuilder(document: store.document.replacingRoot(store.document.root), profile: store.profile).build().result
+ #expect(store.document.sourceMap != nil && copy.document.sourceMap == nil)
+ #expect(try wfStore { _ = try copy.value(at: position) }.code == .invalidPosition)
+ let original = try store.record(at: position)
+ #expect(try store.document.sourceMap?.span(at: original.path) != nil)
+}
+@Test func workflowS11DisabledReferencesRemainOrdinaryData() throws {
+ let document = try wfDocument(#"{"u":[{"id":"a","arbitrary":{"collection":"missing","id":"not-a-link"}}]}"#)
+ let store = try JSONStoreBuilder(document: document, profile: .init(version: .v1, collectionsAt: [], collections: [
+ .init(name: "u", recordsAt: [.key("u")], idKey: "id", referenceFields: [])], references: .disabled)).build().result
+ let start = try store.record(collection: 0, row: 0).position
+ var bounds = JSONStoreFollowLimits(); bounds.maximumDepth = 0
+ #expect(try store.follow(from: start, bounds: bounds).result == [start] && store.references.isEmpty)
+}
+@Test func workflowSharedImmutablePlansHaveIndependentBatchAndStoreHealth() async throws {
+ let store = try wfStar(), start = try store.record(collection: 0, row: 0).position
+ let batch = JSONInspectionBatch(document: store.document)
+ try await withThrowingTaskGroup(of: Bool.self) { group in
+ for i in 0..<8 { group.addTask {
+ if i % 2 == 0 {
+ var limits = JSONLimits.balanced; limits[.workUnits] = 0
+ do { _ = try store.follow(from: start, limits: limits); return false }
+ catch let error as JSONFailure { return error.resource?.metric == .workUnits }
+ }
+ let read = try batch.values(at: [[]])
+ return try store.follow(from: start).result.count == 3 && read.result == [store.document.root]
+ } }
+ for try await ok in group { #expect(ok) }
+ }
+}
+
+@Test func workflowS12StorePathsHonorInvocationPointerAndDiagnosticLimits() throws {
+ let store = try wfStar(), position = try store.record(collection: 0, row: 0).position
+ for (metric, boundary) in [(JSONResourceMetric.pointerTokens, UInt64(2)), (.pointerBytes, 4), (.diagnosticBytes, 4)] {
+ var limits = JSONLimits.balanced; limits[metric] = boundary
+ #expect(try store.value(at: position, limits: limits).result == store.value(at: position).result)
+ limits[metric] = boundary - 1
+ #expect(try wfCore { _ = try store.value(at: position, limits: limits) }.resource?.metric == metric)
+ }
+}
+@Test func workflowS13FollowSelfCycleAndLongNarrowChain() throws {
+ let profile = JSONStoreProfile(version: .v1, collectionsAt: [], collections: [
+ .init(name: "u", recordsAt: [.key("u")], idKey: "id", referenceFields: [.init(key: "next", presence: .required, nullable: false)])],
+ references: .strict(marker: .v1, validateTargets: true))
+ func store(_ text: String) throws -> JSONConfiguredStore { try JSONStoreBuilder(document: wfDocument(text), profile: profile).build().result }
+ let single = try store(#"{"u":[{"id":"0","next":{"collection":"u","id":"0"}}]}"#)
+ let start = try single.record(collection: 0, row: 0).position
+ var bounds = JSONStoreFollowLimits(); bounds.maximumDepth = 0; bounds.maximumFrontier = 1
+ #expect(try single.follow(from: start, bounds: bounds).result == [start])
+ let chain = try store(#"{"u":[{"id":"0","next":{"collection":"u","id":"1"}},{"id":"1","next":{"collection":"u","id":"2"}},{"id":"2","next":{"collection":"u","id":"3"}},{"id":"3","next":{"collection":"u","id":"3"}}]}"#)
+ let first = try chain.record(collection: 0, row: 0).position
+ bounds.maximumDepth = 3
+ #expect(try chain.follow(from: first, bounds: bounds).result.map(\.row) == [0,1,2,3])
+ bounds.maximumDepth = 2
+ let failure = try wfStore { _ = try chain.follow(from: first, bounds: bounds) }
+ #expect(failure.code == .capacityExceeded && failure.capacity == "maximumDepth" && failure.limit == 2)
+}
+
+@Test func workflowB09NestedSnapshotRepeatsQuotasAndLateCancellation() async throws {
+ let document = try wfDocument(#"{"a/b~":{"":{"é":[1,2],"e\u0301":null}}}"#)
+ let snapshot = try JSONStructuralSnapshotBuilder(document: document).build().result
+ // DFS: root0, a/b~1, empty2, é3, 1 at4, 2 at5, decomposed null6.
+ let ordinals = [5,4,5,6], report = try snapshot.values(for: [5,4,5,6])
+ #expect(report.result == [2,1,2,.null] && report.statistics[.retainedBytes] == 160)
+ var limits = JSONLimits.balanced; limits[.workUnits] = report.statistics[.workUnits]
+ #expect(try snapshot.values(for: ordinals, limits: limits).result == report.result)
+ limits[.workUnits] -= 1
+ #expect(try wfCore { _ = try snapshot.values(for: ordinals, limits: limits) }.resource?.metric == .workUnits)
+ let outputs = Mutex(0), lifetime = Mutex(false)
+ let task = Task {
+ var inner = JSONLimits.balanced; inner[.cancellationQuantum] = 1
+ return try snapshot.valuesForQualification(for: ordinals, limits: inner, outputObserver: { count in
+ outputs.withLock { $0 = count }; if count == 2 { withUnsafeCurrentTask { $0?.cancel() } }
+ }, lifetimeObserver: { root, state in lifetime.withLock { $0 = root && state } })
+ }
+ do { _ = try await task.value; Issue.record("Expected late cancellation") } catch is CancellationError {}
+ #expect(outputs.withLock { $0 } == 2 && lifetime.withLock { $0 })
+ #expect(try snapshot.values(for: [5]).result == [2])
+}
+@Test func workflowS14MetadataAliasesAndScratchHaveExactRetentionDisposition() throws {
+ let store = try wfStar(), start = try store.record(collection: 0, row: 0).position
+ let lookup = try store.lookup(collection: "u", id: "0")
+ #expect(lookup.statistics[.retainedBytes] == 32)
+ let follow = try store.follow(from: start)
+ #expect(follow.statistics[.retainedBytes] == 128)
+ let values = try store.values(at: [start,start])
+ #expect(values.statistics[.retainedBytes] == 96)
+ #expect(follow.statistics.peakLogicalRetainedBytes >= store.overlayLogicalBytes)
+ var limits = JSONLimits.balanced; limits[.retainedBytes] = follow.statistics.peakLogicalRetainedBytes
+ #expect(try store.follow(from: start, limits: limits).result == follow.result)
+ limits[.retainedBytes] -= 1
+ #expect(try wfCore { _ = try store.follow(from: start, limits: limits) }.resource?.metric == .retainedBytes)
+}
From 5e27899230262a8d61cf0a5fd7d71fed18e105ac Mon Sep 17 00:00:00 2001
From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com>
Date: Wed, 28 Oct 2026 17:16:05 +0100
Subject: [PATCH 2/6] Qualify cross-product workflows and bounded structural
capacity
---
CHANGELOG.md | 6 +
CONTRIBUTING.md | 5 +-
.../BoundedWorkflows1.8.md | 17 ++
Documentation/Qualification/Release1.8.md | 64 ++++++
.../StructuralWorkflows1.8Matrix.json | 88 ++++++++
.../Qualification/TestDiscovery.json | 66 +++++-
Examples/BoundedWorkflows/Package.swift | 3 +
Examples/BoundedWorkflows/README.md | 50 +++++
.../Sources/BoundedWorkflows/Acceptance.swift | 57 +++++
.../Sources/BoundedWorkflows/Capacity.swift | 157 +++++++++++++
.../Sources/BoundedWorkflows/Main.swift | 7 +
README.md | 7 +
Scripts/ci.py | 44 +++-
Scripts/qualify_structural_acceptance.py | 134 +++++++++++
Scripts/test_qualification_harness.py | 56 ++++-
Sources/JSONCLIEngine/CLIBuildIdentity.swift | 2 +-
.../BoundedWorkflowRecipes.md | 67 ++++++
.../Documentation.docc/JSONInspection.md | 1 +
.../CompositionContractTests.swift | 4 +-
.../PublicWorkflowTests.swift | 4 +-
.../StagedImportTests.swift | 4 +-
.../StructuralAcceptanceTests.swift | 209 ++++++++++++++++++
22 files changed, 1034 insertions(+), 18 deletions(-)
create mode 100644 Documentation/Qualification/StructuralWorkflows1.8Matrix.json
create mode 100644 Examples/BoundedWorkflows/README.md
create mode 100644 Examples/BoundedWorkflows/Sources/BoundedWorkflows/Acceptance.swift
create mode 100644 Examples/BoundedWorkflows/Sources/BoundedWorkflows/Capacity.swift
create mode 100644 Scripts/qualify_structural_acceptance.py
create mode 100644 Sources/JSONInspection/Documentation.docc/BoundedWorkflowRecipes.md
create mode 100644 Tests/JSONIntegrationTests/StructuralAcceptanceTests.swift
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4f4f047..630926e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,12 @@
## Unreleased — 1.8.0-dev
+- Extend public acceptance across schema validation, store/transform reads and
+ checked Patch rebuilds, with exact numeric/scalar identity and lifecycle cases.
+- Add a frozen capacity/owned-counter matrix and compiled cross-product recipes;
+ distinguish diagnostic refusals from completed cells and logical retention from
+ timing, RSS and allocations. Library APIs/algorithms and default policies unchanged.
+
- Add ordered typed-path, snapshot-ordinal and store-position value batches using
one cumulative native ledger per batch, complete source admission and atomic results.
- Promote the original checked configured-store profile/construction inside
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index ba2262c..9171704 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -2,7 +2,10 @@
This isolated development source is 1.8.0-dev from published 1.7.0.
Run `python3 Scripts/ci.py --lane structural-workflows` for exact Debug/Release
-behavior, preserved store cases and public compiler/consumer checks. Run
+behavior, preserved store cases and public compiler/consumer checks. M2 adds
+`--lane structural-acceptance` and `--lane structural-acceptance-safety` for exact
+public cross-product cases, deterministic capacity observations and their safety
+checks. These extend the existing gates. Run
`--lane structural-workflows-safety` for scoped ASan, shared-state TSan and strict
memory checks; complete native/Linux/docs/WASI gates still apply before release.
Do not infer a pass when storage or an actual supported runtime is unavailable.
diff --git a/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md b/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
index a22601d..77c6890 100644
--- a/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
+++ b/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
@@ -61,3 +61,20 @@ cases. Keep partial profiling/G4 HOLD and the1.7-only P4 exception historical;
No persistence, public mutable store, online lookup, new query grammar, lower
platform floor, parallel batch execution or wider retrieval belongs to this wave.
+
+## M2 application acceptance and counter observations
+
+Schema validation, store construction, reads, transformation and Patch remain
+separate public operations with independent reports. A recipe does not attach a
+replacement ledger or claim a pipeline-wide budget. Ordinary schema invalidity,
+operational quota failure and root removal remain distinct. Patches require a
+new source wrapper/store; old positions and source spans do not silently migrate.
+Accepted bytes of an awaited sink cannot be recalled after cancellation.
+
+The authored public examples and Integration behavior cases qualify those
+boundaries without changing any engine API. The capacity matrix in
+Documentation/Qualification/StructuralWorkflows1.8Matrix.json records fixed
+fixtures, exact boundaries and separately reported diagnostic refusals. Its
+counter comparisons are not elapsed-time, allocation or RSS evidence. Existing
+49 core metrics, twelve construction caps, profile defaults, fixture identities,
+performance review trigger and controlled-performance holds are unchanged.
diff --git a/Documentation/Qualification/Release1.8.md b/Documentation/Qualification/Release1.8.md
index 9768c20..daff11a 100644
--- a/Documentation/Qualification/Release1.8.md
+++ b/Documentation/Qualification/Release1.8.md
@@ -1,3 +1,67 @@
+# Milestone 2 — cross-product acceptance and capacity observations
+
+M2 is an additive qualification increment from local checkpoint
+`07ed28095dde5e8063570aa6a2436039b5bd43ca`, tree
+`5e0961a13fa4b8746854beca23a424db3075ff96`. The sealed M1 archive/receipts remain
+unchanged. No public library signature, algorithm, root manifest, default profile
+or fixture comparator is changed by this increment. A new guide updates the
+existing catalog and therefore requires regenerated CLI source identity.
+
+The completed local M2 receipts pass all43 Integration functions in Debug/Release
+and ASan, the new sharing TSan function and strict test-target compilation. The
+existing49-cap public test passes every exact expected probe. Six new public
+consumer witnesses/configuration, the original twelve and their ordinary compiler
+refusals pass; all16 counter/capacity cells complete with exact boundary proofs.
+All13 native DocC catalogs compile with warnings as errors, and1,639 authored
+public declarations/parameters are documented. Compared to published1.7.0, only
+Inspection has139 additive declarations from M1; no existing declaration fragment
+or identity changes in the complete package graph. Native pinned qualification
+observes318 syntax cases (287 evaluated successes,31 policy rejections) and2,337
+schema cases (2,300 evaluated successes,37 policy exclusions), with82 authored
+semantics,9 budget cases and4,396 deterministic malformed replays, no failures.
+The full native/safety/docs CI lanes and platform/runtime/performance gates remain
+separate; these local results do not publish or admit1.8.
+
+This completion note is a documentary successor to the raw receipts. Final private
+binding records its one-path delta; the earlier safety receipt also precedes the
+ci.py addition of the49-cap native command. Its three safety commands, product/test/
+fixture bytes and flags are unchanged and rechecked by the current exact harness.
+Original failures and M1 evidence remain intact. This is scoped agent review and
+native qualification, not independent human release approval.
+
+Nine named public-import Integration tests cover schema/store/transform/Patch
+composition, exact bytes/scalar collection names, source-map/owner replacement,
+dangling-link refusal, terminal accounting, accepted sink prefixes after cancellation
+and independent shared-plan health. Native/portable discovery is now
+1,008/992 functions,390 parameter invocations; WASI expects904/353. Integration
+has43 functions and12 sharing cases; combined sharing62/61. These are discovery
+requirements, not claims that all platform lanes have run on this source.
+
+The expanded public SwiftPM example adds six cross-product witnesses per
+configuration, preserving all twelve M1 witnesses and compiler refusals. The
+[capacity matrix](StructuralWorkflows1.8Matrix.json) freezes16 cells and five
+independently hashed fixtures. Batches use a fixed eight-record source and
+1/8/64/512 requests; follow uses a growing chain and explicit finite bounds.
+Exact request, work and retention boundaries are checked. Independent singular
+reads report sums of cumulative counts and maxima of invocation retention, never
+summed process memory. Diagnostic typed refusals are separate incomplete capacity
+observations, not successes. No timing, allocations, RSS or speedup is measured.
+
+Run `Scripts/ci.py --lane structural-acceptance` and
+`--lane structural-acceptance-safety`; full existing native/Linux/safety/docs gates
+remain. Actual M2 outcomes belong in source-bound private receipts under
+Artifacts/Release1.8M2. Compiler/fixture reconciliation failures are retained.
+M2 also executes the existing49-cap public qualification test, preserving its
+exact expected outcomes. The full43-function Integration target is checked in
+Debug/Release/ASan; its original names/assertions remain. Strict test-target
+compilation uses narrowly scoped acknowledgements for three preexisting
+UnsafeCurrentTask callback accesses; those handles never escape or cross tasks.
+No compiler flag or unsafe production annotation is added. The new cancellation
+case instead uses a safe Task handle and checked-continuation rendezvous.
+
+The M1 record below remains historical. Broad QA/performance/publication gates
+remain pending until their actual outcomes are recorded; no1.7 waiver is inherited.
+
# 1.8 bounded structural workflow qualification
Status: implemented local milestone, broader release qualification PARTIAL, on
diff --git a/Documentation/Qualification/StructuralWorkflows1.8Matrix.json b/Documentation/Qualification/StructuralWorkflows1.8Matrix.json
new file mode 100644
index 0000000..11d69b9
--- /dev/null
+++ b/Documentation/Qualification/StructuralWorkflows1.8Matrix.json
@@ -0,0 +1,88 @@
+{
+ "schema_version": 1,
+ "profile": "balanced",
+ "sizes": [
+ 1,
+ 8,
+ 64,
+ 512
+ ],
+ "batch_source_records": 8,
+ "batch_operations": [
+ "path-batch",
+ "snapshot-batch",
+ "store-values"
+ ],
+ "cells": [
+ "path-batch-1",
+ "snapshot-batch-1",
+ "store-values-1",
+ "follow-chain-1",
+ "path-batch-8",
+ "snapshot-batch-8",
+ "store-values-8",
+ "follow-chain-8",
+ "path-batch-64",
+ "snapshot-batch-64",
+ "store-values-64",
+ "follow-chain-64",
+ "path-batch-512",
+ "snapshot-batch-512",
+ "store-values-512",
+ "follow-chain-512"
+ ],
+ "required_complete": [
+ "path-batch-1",
+ "snapshot-batch-1",
+ "store-values-1",
+ "path-batch-8",
+ "snapshot-batch-8",
+ "store-values-8",
+ "path-batch-64",
+ "snapshot-batch-64",
+ "store-values-64",
+ "path-batch-512",
+ "snapshot-batch-512",
+ "store-values-512",
+ "follow-chain-1",
+ "follow-chain-8"
+ ],
+ "diagnostic_capacity": [
+ "follow-chain-64",
+ "follow-chain-512"
+ ],
+ "follow_bounds": "depth=n-1; pending frontier=1; visited/results=n; unchanged balanced native and store defaults",
+ "fixtures": [
+ {
+ "name": "fixed-eight.json",
+ "records": 8,
+ "bytes": 608,
+ "sha256": "f8372702217632eaa0903d11407534f44dd9f59e1c5916b3b2395258500912b7"
+ },
+ {
+ "name": "chain-1.json",
+ "records": 1,
+ "bytes": 62,
+ "sha256": "8fe8b3ce7bc73d5213c7152637bb3f5a0cc64e3bef04afde0b8f521cab0eaf8c"
+ },
+ {
+ "name": "chain-8.json",
+ "records": 8,
+ "bytes": 608,
+ "sha256": "f8372702217632eaa0903d11407534f44dd9f59e1c5916b3b2395258500912b7"
+ },
+ {
+ "name": "chain-64.json",
+ "records": 64,
+ "bytes": 5084,
+ "sha256": "b6c2c1f666a54393b191df5e903b4d3d33a926b7326b2a05717a5de95bcad7f2"
+ },
+ {
+ "name": "chain-512.json",
+ "records": 512,
+ "bytes": 41748,
+ "sha256": "37472330ac6dff6c5fa6142a0c69e867b548e54098d178a47731df150695d2c9"
+ }
+ ],
+ "policy": "A diagnostic typed resource refusal is an observed incomplete capacity cell, never an evaluated success or a default/joint-capacity claim. No timings, RSS or allocations are measured. Native limits/defaults and >10% performance review trigger are unchanged."
+}
diff --git a/Documentation/Qualification/TestDiscovery.json b/Documentation/Qualification/TestDiscovery.json
index 646ae2f..3251689 100644
--- a/Documentation/Qualification/TestDiscovery.json
+++ b/Documentation/Qualification/TestDiscovery.json
@@ -6,7 +6,7 @@
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 30,
"JSONPatchTests": 16,
- "JSONIntegrationTests": 34,
+ "JSONIntegrationTests": 43,
"JSONHTMLTests": 97,
"JSONBridgesTests": 10,
"JSONCLITests": 88,
@@ -31,7 +31,7 @@
"JSONSchemaTests": 3,
"JSONRegexRuntimeTests": 1,
"JSONStreamingTests": 16,
- "JSONIntegrationTests": 11,
+ "JSONIntegrationTests": 12,
"JSONHTMLTests": 5,
"JSONBridgesTests": 1,
"JSONCLITests": 1,
@@ -78,7 +78,7 @@
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 30,
"JSONPatchTests": 16,
- "JSONIntegrationTests": 34,
+ "JSONIntegrationTests": 43,
"JSONHTMLTests": 81,
"JSONBridgesTests": 10,
"JSONCLITests": 88,
@@ -94,7 +94,7 @@
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 30,
"JSONPatchTests": 16,
- "JSONIntegrationTests": 34,
+ "JSONIntegrationTests": 43,
"JSONHTMLTests": 81,
"JSONBridgesTests": 10,
"JSONNavigationTests": 57,
@@ -109,7 +109,7 @@
"JSONSchemaTests": 3,
"JSONRegexRuntimeTests": 1,
"JSONStreamingTests": 16,
- "JSONIntegrationTests": 11,
+ "JSONIntegrationTests": 12,
"JSONHTMLTests": 4,
"JSONBridgesTests": 1,
"JSONCLITests": 1,
@@ -174,5 +174,61 @@
"JSONInspectionTests.workflowS13FollowSelfCycleAndLongNarrowChain()",
"JSONInspectionTests.workflowB09NestedSnapshotRepeatsQuotasAndLateCancellation()",
"JSONInspectionTests.workflowS14MetadataAliasesAndScratchHaveExactRetentionDisposition()"
+ ],
+ "structural_acceptance_function_ids": [
+ "JSONIntegrationTests.acceptance18A01SchemaStoreTransformKeepsExactGoldenBytes()",
+ "JSONIntegrationTests.acceptance18A02InvalidSchemaVerdictIsDistinctFromOperationalRefusal()",
+ "JSONIntegrationTests.acceptance18A03PatchedRebuildHasNewOwnerAndNoInventedSourceMap()",
+ "JSONIntegrationTests.acceptance18A04DanglingPatchedReferenceCannotPublishStore()",
+ "JSONIntegrationTests.acceptance18A05ScalarCollectionNamesSurvivePatchAndCrossLinks()",
+ "JSONIntegrationTests.acceptance18A06TransformFailureStaysTerminalOverStoreInput()",
+ "JSONIntegrationTests.acceptance18A07RootRemovalIsNotNullOrAnEmptyStore()",
+ "JSONIntegrationTests.acceptance18A08CancelledWriterKeepsOnlyItsAcceptedPrefix()",
+ "JSONIntegrationTests.acceptance18SharedPlansKeepIndependentCrossProductHealth()"
+ ],
+ "integration_function_ids": [
+ "JSONIntegrationTests.acceptance18A01SchemaStoreTransformKeepsExactGoldenBytes()",
+ "JSONIntegrationTests.acceptance18A02InvalidSchemaVerdictIsDistinctFromOperationalRefusal()",
+ "JSONIntegrationTests.acceptance18A03PatchedRebuildHasNewOwnerAndNoInventedSourceMap()",
+ "JSONIntegrationTests.acceptance18A04DanglingPatchedReferenceCannotPublishStore()",
+ "JSONIntegrationTests.acceptance18A05ScalarCollectionNamesSurvivePatchAndCrossLinks()",
+ "JSONIntegrationTests.acceptance18A06TransformFailureStaysTerminalOverStoreInput()",
+ "JSONIntegrationTests.acceptance18A07RootRemovalIsNotNullOrAnEmptyStore()",
+ "JSONIntegrationTests.acceptance18A08CancelledWriterKeepsOnlyItsAcceptedPrefix()",
+ "JSONIntegrationTests.acceptance18SharedPlansKeepIndependentCrossProductHealth()",
+ "JSONIntegrationTests.caughtOperationalFailureCannotComplete()",
+ "JSONIntegrationTests.composedRetentionAndURIUseSameScope()",
+ "JSONIntegrationTests.composedWorkThresholdAndOneAdditionalUnit()",
+ "JSONIntegrationTests.compositionCallerErrorAndReportProvenance()",
+ "JSONIntegrationTests.compositionCancellationInsideOwnedScansIsSticky()",
+ "JSONIntegrationTests.compositionCheckedArithmeticAndStickyUnderflow()",
+ "JSONIntegrationTests.compositionExactSourceAndNativeErrors()",
+ "JSONIntegrationTests.compositionIndependentScopesCanBeSharedAcrossTasks()",
+ "JSONIntegrationTests.compositionInputAndCopyAdmissionBeforeAllocation()",
+ "JSONIntegrationTests.compositionMonotoneAliasesAndNoPlanPollution()",
+ "JSONIntegrationTests.compositionNestedScopesHaveExplicitIndependentBudgets()",
+ "JSONIntegrationTests.compositionRestrictionPreservesCapacityHistory()",
+ "JSONIntegrationTests.consolidationAuthoredInputCannotReopenFailedTransformLedger()",
+ "JSONIntegrationTests.consolidationAuthoredValuesValidateTransformAndTraverseWithoutScalarLoss()",
+ "JSONIntegrationTests.qaE2EAnnotationsFromFailedBranchesDoNotAuthorizeProperties()",
+ "JSONIntegrationTests.qaE2EAwaitedSinkCancellationKeepsOnlyAcceptedBytes()",
+ "JSONIntegrationTests.qaE2EAwaitedSinkFailureNeverCertifiesAcceptedPrefix()",
+ "JSONIntegrationTests.qaE2EBuiltinRegistryHasAllPinnedOfflineIdentities()",
+ "JSONIntegrationTests.qaE2ECumulativeWorkSucceedsAtNAndFailsAtNMinusOne()",
+ "JSONIntegrationTests.qaE2EExactScalarAndIntegerConstraintsRejectNearMatches()",
+ "JSONIntegrationTests.qaE2EExactWorkflowHasIndependentGoldenOutput()",
+ "JSONIntegrationTests.qaE2EMinimizedMalformedReplaysHavePreciseFailures()",
+ "JSONIntegrationTests.qaE2ENDJSONImportFailuresKeepAcceptedPrefixWithoutCommit()",
+ "JSONIntegrationTests.qaE2ENDJSONImportStrictFramingCommitsOnlyAtEOF()",
+ "JSONIntegrationTests.qaE2EOfflineReferenceRequiresExplicitRegistry()",
+ "JSONIntegrationTests.qaE2EPatchRollbackRootAbsenceAndMergeNullRemainDistinct()",
+ "JSONIntegrationTests.qaE2ERootImportCancellationAfterProvisionalAcceptanceIsTerminal()",
+ "JSONIntegrationTests.qaE2ERootImportCommitsOnlyAfterCertifiedCompletion()",
+ "JSONIntegrationTests.qaE2ERootImportMalformedTailDoesNotCommitAcceptedElements()",
+ "JSONIntegrationTests.qaE2ERootImportQuotaStopsBeforeCommitAndFurtherPulls()",
+ "JSONIntegrationTests.qaE2ESeededModelsMatchOrderedMemberAndIntegerOracles()",
+ "JSONIntegrationTests.qaE2ESharedPlanRecoversAfterIndependentFailures()",
+ "JSONIntegrationTests.qaE2EUnknownEnvelopeVersionIsCorruption()",
+ "JSONIntegrationTests.qaE2EVersionOneEnvelopePreservesExactInnerState()"
]
}
diff --git a/Examples/BoundedWorkflows/Package.swift b/Examples/BoundedWorkflows/Package.swift
index 62eaba2..6fb8b27 100644
--- a/Examples/BoundedWorkflows/Package.swift
+++ b/Examples/BoundedWorkflows/Package.swift
@@ -7,6 +7,9 @@ let package = Package(
dependencies: [.package(name: "Swift-JSON", path: "../..")],
targets: [.executableTarget(name: "BoundedWorkflows", dependencies: [
.product(name: "JSON", package: "Swift-JSON"),
+ .product(name: "JSONSchema", package: "Swift-JSON"),
+ .product(name: "JSONPatch", package: "Swift-JSON"),
+ .product(name: "JSONTransform", package: "Swift-JSON"),
.product(name: "JSONNavigation", package: "Swift-JSON"),
.product(name: "JSONInspection", package: "Swift-JSON")
])], swiftLanguageModes: [.v6]
diff --git a/Examples/BoundedWorkflows/README.md b/Examples/BoundedWorkflows/README.md
new file mode 100644
index 0000000..c052061
--- /dev/null
+++ b/Examples/BoundedWorkflows/README.md
@@ -0,0 +1,50 @@
+# Bounded structural workflows
+
+A public SwiftPM consumer for the unreleased 1.8 source. It imports JSON,
+JSONSchema, JSONNavigation, JSONInspection, JSONPatch and JSONTransform; no private
+meter/runtime module or external dependency is used. Swift6.4/language6 and the
+supported platform baselines still apply. Released1.7 does not supply the new APIs.
+
+From the repository root:
+
+```sh
+swift run --package-path Examples/BoundedWorkflows BoundedWorkflows
+swift run -c release --package-path Examples/BoundedWorkflows BoundedWorkflows
+```
+
+The twelve original witnesses preserve exact values, request order/repetition,
+missing versus null, source-local ownership, bounded follow and public reports.
+Six additional witnesses compose native schema validation, source interpretation,
+transformation and Patch. They check a golden exact-decimal byte sequence, a new
+owner/source wrapper after Patch, ordinary invalidity versus operational refusal,
+root removal versus null, and terminal transform completion after a caught error.
+The separate Swift Testing Integration cases also exercise declared-link corruption,
+scalar-distinct collection names, awaited-sink cancellation and concurrent plans.
+
+Each stage is its own native operation, not one implicit global pipeline ledger.
+Caller array assembly and callbacks are ordinary caller work. Validate a schema
+against its explicit offline registry/selected root before interpreting a source;
+the store does not infer schema requiredness. Rebuild after Patch and obtain new
+positions. Atomic publication into a sink is a caller staging decision.
+
+## Counter and capacity observations
+
+```sh
+swift run -c release --package-path Examples/BoundedWorkflows BoundedWorkflows --capacity /tmp/swift-json-capacity
+```
+
+Use a new output directory. This writes cells.json and five deterministic fixture
+files. The frozen matrix is in Documentation/Qualification/StructuralWorkflows1.8Matrix.json.
+The qualification script independently verifies fixture hashes, cell identities,
+expected results and exact/one-below request/work/retention boundaries.
+Batch cells use1/8/64/512 requests against eight records. Follow cells use growing
+chains with frontier1 and explicit depth/visited/results bounds; native balanced
+and store construction defaults stay unchanged. A typed diagnostic refusal is
+recorded as an incomplete capacity observation, never a successful closure.
+
+Independent singular invocations report summed work/admission counters and the
+maximum of their per-invocation retention. A batch reports its one original ledger.
+Those quantities do not measure elapsed time, heap allocations, process RSS,
+caller-held copies or jointly achievable default maxima. No performance improvement
+or policy change follows from the counter experiment. The controlled native
+performance gate remains a separate final-source requirement.
diff --git a/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Acceptance.swift b/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Acceptance.swift
new file mode 100644
index 0000000..caccc6d
--- /dev/null
+++ b/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Acceptance.swift
@@ -0,0 +1,57 @@
+// SPDX-License-Identifier: Apache-2.0
+import Foundation
+import JSON
+import JSONSchema
+import JSONPatch
+import JSONInspection
+import JSONTransform
+
+// Authored application recipe. These stages are separate native operations;
+// caller array assembly is not a library-wide instruction or memory quota.
+func structuralAcceptance() throws {
+ func witness(_ name: String, _ condition: Bool) {
+ precondition(condition, name); print("ACCEPTANCE18_CASE \(name) passed")
+ }
+ let source = try JSONParser().parseDocument(Data(#"{"u":[{"id":"é","amount":9007199254740993,"next":{"collection":"u","id":"e\u0301"}},{"id":"e\u0301","amount":0.10000000000000001,"next":null}]}"#.utf8))
+ let schema = try JSONSchemaCompiler().compile(JSONSchemaDocument(JSONParser().parse(
+ #"{"type":"object","required":["u"],"properties":{"u":{"type":"array","items":{"type":"object","required":["id","amount","next"],"properties":{"id":{"type":"string"},"amount":{"type":"number"},"next":{"type":["object","null"]}}}}}}"#)))
+ let validation = try JSONSchemaValidator().validate(source.root, against: schema)
+ precondition(validation.isValid)
+ let profile = JSONStoreProfile(version: .v1, collectionsAt: [], collections: [
+ .init(name: "u", recordsAt: [.key("u")], idKey: "id", referenceFields: [.init(key: "next", presence: .required, nullable: true)])
+ ], references: .strict(marker: .v1, validateTargets: true))
+ let store = try JSONStoreBuilder(document: source, profile: profile).build().result
+ guard let first = try store.lookup(collection: "u", id: "é").result else { preconditionFailure() }
+ let values = try store.values(at: store.follow(from: first).result)
+ let transform = try JSONTransformProgram.compile(".[] | .amount")
+ let output = try transform.withExecution(input: .array(values.result)) { execution in
+ var output: [JSON] = []
+ while let value = try execution.next() { output.append(value.value) }
+ return JSON.array(output)
+ }
+ witness("schema-store-transform", try JSONWriter(options: .init(numberStyle: .preserved)).write(output.result) == Data("[9007199254740993,0.10000000000000001]".utf8))
+ let patch = try JSONPatch(operations: [.replace(path: JSONPointer(parsing: "/u/1/amount"), value: .number(JSONNumber(parsing: "1.25")))])
+ guard case .document(let changed) = try patch.applying(to: source.root) else { preconditionFailure() }
+ let document = source.replacingRoot(changed), rebuilt = try JSONStoreBuilder(document: document, profile: profile).build().result
+ do { _ = try rebuilt.value(at: first); preconditionFailure() }
+ catch let error as JSONStoreFailure { witness("patched-owner", error.code == .invalidPosition) }
+ witness("source-map-reset", document.sourceMap == nil && document.sourceBytes == nil && store.document.sourceMap != nil)
+ let invalid = try JSONSchemaValidator().validate(.null, against: schema)
+ var native = JSONLimits.balanced; native[.evaluationWorkUnits] = 0
+ do { _ = try JSONSchemaValidator(options: .init(limits: native)).validate(source.root, against: schema); preconditionFailure() }
+ catch let error as JSONFailure { witness("schema-refusal", !invalid.isValid && error.resource?.metric == .evaluationWorkUnits) }
+ let rootPointer = try JSONPointer(parsing: "")
+ let removed = try JSONPatch(operations: [.remove(path: rootPointer)]).applying(to: source.root)
+ let null = try JSONPatch(operations: [.replace(path: rootPointer, value: .null)]).applying(to: source.root)
+ witness("root-removal", removed == .removed && null == .document(.null) && removed != null)
+ var limits = JSONTransformLimits(); limits[.results] = 1
+ do {
+ _ = try transform.withExecution(input: .array(values.result), limits: limits) { execution in
+ let yielded = try execution.next(); precondition(yielded != nil)
+ do { _ = try execution.next(); preconditionFailure() }
+ catch let error as JSONTransformFailure { precondition(error.category == .resource && error.metric == "results") }
+ }
+ preconditionFailure("A caught failure cannot reopen successful completion")
+ } catch let error as JSONTransformFailure { witness("terminal-transform", error.category == .resource && error.metric == "results") }
+ print("ACCEPTANCE18 passed cases=6")
+}
diff --git a/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Capacity.swift b/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Capacity.swift
new file mode 100644
index 0000000..de0a8bd
--- /dev/null
+++ b/Examples/BoundedWorkflows/Sources/BoundedWorkflows/Capacity.swift
@@ -0,0 +1,157 @@
+// SPDX-License-Identifier: Apache-2.0
+import Foundation
+import JSON
+import JSONInspection
+
+private struct CapacityCounters: Encodable {
+ let work: UInt64, nodes: UInt64, retained: UInt64, peak: UInt64
+ init(_ statistics: JSONOperationStatistics) {
+ work = statistics[.workUnits]; nodes = statistics[.nodes]
+ retained = statistics[.retainedBytes]; peak = statistics.peakLogicalRetainedBytes
+ }
+}
+private struct CapacitySingles: Encodable {
+ let workSum: UInt64, nodesSum: UInt64
+ let maximumInvocationRetained: UInt64, maximumInvocationPeak: UInt64
+}
+private struct CapacityCell: Encodable {
+ let id: String, operation: String, requests: Int, fixture: String, outcome: String
+ let report: CapacityCounters?, independentSingles: CapacitySingles?
+ let requestBoundary: Bool?, workBoundary: Bool?, retentionBoundary: Bool?
+ let errorMetric: String?, errorLimit: UInt64?, errorAttempted: UInt64?, errorStage: String?
+}
+private struct CapacityReceipt: Encodable {
+ let schemaVersion = 1
+ let profile = "balanced"
+ let scope = "Owned-work and logical retention observations; no elapsed-time, allocation, RSS or throughput measurement. Fixture construction, expected-value comparison and caller array copies are caller work."
+ let cells: [CapacityCell]
+}
+private enum CapacityInvariant: Error { case result, boundary, overflow }
+private func capacitySum(_ lhs: UInt64, _ rhs: UInt64) throws -> UInt64 {
+ let (result, overflow) = lhs.addingReportingOverflow(rhs)
+ guard !overflow else { throw CapacityInvariant.overflow }; return result
+}
+private func capacityProfile() -> JSONStoreProfile {
+ .init(version: .v1, collectionsAt: [], collections: [
+ .init(name: "records", recordsAt: [.key("records")], idKey: "id", referenceFields: [.init(key: "next", presence: .required, nullable: true)])
+ ], references: .strict(marker: .v1, validateTargets: true))
+}
+private func capacityFixture(_ count: Int) throws -> Data {
+ var records: [JSON] = []
+ for index in 0.. JSONOperationReport) throws -> CapacitySingles {
+ var work: UInt64 = 0, nodes: UInt64 = 0, peak: UInt64 = 0, retained: UInt64 = 0
+ for index in 0.. Void, metric: JSONResourceMetric) throws {
+ do { try body(); throw CapacityInvariant.boundary }
+ catch let error as JSONFailure { guard error.resource?.metric == metric else { throw error } }
+}
+private func capacityBoundaries(_ report: JSONOperationReport, _ body: (JSONLimits) throws -> JSONOperationReport) throws {
+ var limits = JSONLimits.balanced
+ limits[.workUnits] = report.statistics[.workUnits]
+ limits[.retainedBytes] = report.statistics.peakLogicalRetainedBytes
+ _ = try body(limits)
+ limits[.workUnits] -= 1
+ try capacityResource({ _ = try body(limits) }, metric: .workUnits)
+ limits = .balanced; limits[.retainedBytes] = report.statistics.peakLogicalRetainedBytes - 1
+ try capacityResource({ _ = try body(limits) }, metric: .retainedBytes)
+}
+
+func structuralCapacity(output: URL) throws {
+ try FileManager.default.createDirectory(at: output, withIntermediateDirectories: true)
+ let fixtureDirectory = output.appendingPathComponent("fixtures")
+ try FileManager.default.createDirectory(at: fixtureDirectory, withIntermediateDirectories: true)
+ let fixedBytes = try capacityFixture(8)
+ try fixedBytes.write(to: fixtureDirectory.appendingPathComponent("fixed-eight.json"))
+ let document = try JSONParser().parseDocument(fixedBytes)
+ let snapshot = try JSONStructuralSnapshotBuilder(document: document).build().result
+ let store = try JSONStoreBuilder(document: document, profile: capacityProfile()).build().result
+ var amountOrdinals: [Int] = []
+ for ordinal in 0.. JSONOperationReport<[JSON]> {
+ switch operation {
+ case "path-batch": return try JSONInspectionBatch(document: document, limits: limits).values(at: paths, maximumRequests: maximum)
+ case "snapshot-batch": return try snapshot.values(for: ordinals, maximumRequests: maximum, limits: limits)
+ default: return try store.values(at: positions, maximumRequests: maximum, limits: limits)
+ }
+ }
+ let report = try batch(.balanced, maximum: count)
+ guard report.result.count == count else { throw CapacityInvariant.result }
+ for (index, value) in report.result.enumerated() {
+ let observed = operation == "store-values" ? try value.requiredMember("amount") : value
+ guard observed == expected[index % 8] else { throw CapacityInvariant.result }
+ }
+ let singles = try sumSingles(count) { index in
+ switch operation {
+ case "path-batch": return try document.readWithReport { try $0["records"][index % 8]["amount"].requiredValue() }
+ case "snapshot-batch": return try snapshot.value(for: ordinals[index])
+ default: return try store.value(at: positions[index])
+ }
+ }
+ guard singles.nodesSum == report.statistics[.nodes] * UInt64(count) else { throw CapacityInvariant.result }
+ do { _ = try batch(.balanced, maximum: count - 1); throw CapacityInvariant.boundary }
+ catch let error as JSONInspectionBatchFailure {
+ guard error == .requestLimitExceeded(limit: count - 1, actual: count) else { throw error }
+ }
+ try capacityBoundaries(report) { try batch($0, maximum: count) }
+ cells.append(.init(id: operation + "-" + String(count), operation: operation, requests: count, fixture: "fixed-eight.json", outcome: "completed",
+ report: .init(report.statistics), independentSingles: singles, requestBoundary: true, workBoundary: true, retentionBoundary: true,
+ errorMetric: nil, errorLimit: nil, errorAttempted: nil, errorStage: nil))
+ }
+ let name = "chain-" + String(count) + ".json", fixture = try capacityFixture(count)
+ try fixture.write(to: fixtureDirectory.appendingPathComponent(name))
+ func refusal(_ error: JSONFailure, stage: String) throws -> CapacityCell {
+ guard error.category == .resource, let resource = error.resource else { throw error }
+ return .init(id: "follow-chain-" + String(count), operation: "follow-chain", requests: count, fixture: name, outcome: "operational_error",
+ report: nil, independentSingles: nil, requestBoundary: nil, workBoundary: nil, retentionBoundary: nil,
+ errorMetric: resource.metric.rawValue, errorLimit: resource.limit, errorAttempted: resource.attemptedUsage, errorStage: stage)
+ }
+ let chain: JSONConfiguredStore
+ do { chain = try JSONStoreBuilder(document: JSONParser().parseDocument(fixture), profile: capacityProfile()).build().result }
+ catch let error as JSONFailure { cells.append(try refusal(error, stage: "construction")); continue }
+ let start = try chain.record(collection: 0, row: 0).position
+ var bounds = JSONStoreFollowLimits(); bounds.maximumDepth = count - 1; bounds.maximumFrontier = 1
+ bounds.maximumVisited = count; bounds.maximumResults = count
+ let report: JSONOperationReport<[JSONStorePosition]>
+ do { report = try chain.follow(from: start, bounds: bounds) }
+ catch let error as JSONFailure { cells.append(try refusal(error, stage: "follow")); continue }
+ guard report.result.map(\.row) == Array(0..= report['retained'] and all(row.get(k) is True for k in ['requestBoundary','workBoundary','retentionBoundary'])
+ and all(row.get(k) is None for k in ['errorMetric','errorLimit','errorAttempted','errorStage'])):
+ return False
+ singles = row.get('independentSingles')
+ if operation == 'follow-chain':
+ if singles is not None: return False
+ elif not (isinstance(singles, dict) and all(finite(singles.get(k), positive=k in ('workSum','nodesSum','maximumInvocationPeak'))
+ for k in ['workSum','nodesSum','maximumInvocationRetained','maximumInvocationPeak'])
+ and singles['nodesSum'] == size * report['nodes'] and singles['maximumInvocationPeak'] >= singles['maximumInvocationRetained']):
+ return False
+ elif row.get('outcome') == 'operational_error' and row['id'] in MATRIX['diagnostic_capacity']:
+ if not (row.get('report') is None and row.get('independentSingles') is None
+ and all(row.get(k) is None for k in ['requestBoundary','workBoundary','retentionBoundary'])
+ and row.get('errorMetric') in METRICS
+ and finite(row.get('errorLimit')) and finite(row.get('errorAttempted'), positive=True)
+ and row['errorAttempted'] > row['errorLimit'] and row.get('errorStage') in ('construction','follow')):
+ return False
+ else:
+ return False
+ return all(next(r for r in rows if r['id'] == name)['outcome'] == 'completed' for name in MATRIX['required_complete'])
+
+def validate_fixtures(directory):
+ expected = {x['name']: x for x in MATRIX['fixtures']}
+ actual = sorted(directory.glob('*.json'))
+ if {p.name for p in actual} != set(expected) or any(p.is_symlink() for p in actual):
+ raise ValueError('Changed capacity fixture inventory')
+ records = []
+ for path in actual:
+ data = path.read_bytes(); record = dict(name=path.name, bytes=len(data), sha256=hashlib.sha256(data).hexdigest())
+ if any(record[k] != expected[path.name][k] for k in ('bytes','sha256')):
+ raise ValueError('Changed capacity fixture bytes: ' + path.name)
+ records.append(record)
+ return records
+
+def check_receipt(data):
+ commands = data.get('commands', [])
+ expected = {'consumer','debug-binpath','debug-guide','release-binpath','release-guide','capacity'}
+ fixtures = data.get('fixtures', [])
+ return (data.get('status') == 'passed' and data.get('source_inputs_unchanged') is True
+ and data.get('matrix_sha256') == hashlib.sha256(MATRIX_PATH.read_bytes()).hexdigest()
+ and len(commands) == len(expected) and {c.get('name') for c in commands} == expected
+ and all(c.get('exit') == 0 and c.get('compiler_crash') is False for c in commands)
+ and set(data.get('consumer_results', {})) == {'debug','release'}
+ and all(len(v) == len(CASES) and set(v) == CASES for v in data['consumer_results'].values())
+ and validate_capacity(data.get('capacity', {}))
+ and data.get('completed_cells') == sum(r['outcome'] == 'completed' for r in data['capacity']['cells'])
+ and data.get('operational_refusals') == sum(r['outcome'] == 'operational_error' for r in data['capacity']['cells'])
+ and data.get('performance_status', '').startswith('NOT MEASURED:')
+ and len(fixtures) == len(MATRIX['fixtures']) and sorted(fixtures, key=lambda x:x['name']) == sorted(
+ [{k:x[k] for k in ('name','bytes','sha256')} for x in MATRIX['fixtures']], key=lambda x:x['name']))
+
+def main():
+ parser = argparse.ArgumentParser(); parser.add_argument('--output', required=True)
+ args = parser.parse_args(); out = ROOT / args.output; out.mkdir(parents=True, exist_ok=True)
+ receipt = dict(status='running', source_inputs=[dict(path=p.relative_to(ROOT).as_posix(), sha256=hashlib.sha256(p.read_bytes()).hexdigest()) for p in files()],
+ matrix_sha256=hashlib.sha256(MATRIX_PATH.read_bytes()).hexdigest(), commands=[], consumer_results={})
+ def save(): (out/'receipt.json').write_text(json.dumps(receipt, indent=2)+'\n')
+ def run(name, command):
+ result = subprocess.run(command, cwd=ROOT, capture_output=True, text=True, timeout=1200)
+ log = out/(name+'.log'); log.write_text(result.stdout+result.stderr)
+ crash = (result.returncode < 0 and command[0] == 'swiftc') or any(x in log.read_text() for x in ['Stack dump:', 'Assertion failed', 'Found ownership error', 'fatal error encountered during compilation'])
+ receipt['commands'].append(dict(name=name, command=command, exit=result.returncode, compiler_crash=crash, log=str(log.relative_to(ROOT)), log_sha256=hashlib.sha256(log.read_bytes()).hexdigest()))
+ save(); print(name, result.returncode, flush=True)
+ if result.returncode != 0 or crash: raise ValueError(name)
+ return result.stdout
+ try:
+ child = out/'public-consumer'
+ run('consumer', ['python3','-B','Scripts/qualify_structural_workflows.py','--output',str(child.relative_to(ROOT))])
+ original = json.loads((child/'receipt.json').read_text())
+ if not check_original_consumer(original): raise ValueError('Original public consumer/compiler boundary failed')
+ for config in ['debug','release']:
+ log = child/(config+'-consumer.log')
+ cases = re.findall(r'^ACCEPTANCE18_CASE (\S+) passed$', log.read_text(), re.M)
+ if len(cases) != 6 or set(cases) != CASES or len(re.findall(r'^ACCEPTANCE18 passed cases=6$', log.read_text(), re.M)) != 1:
+ raise ValueError('Incomplete public cross-product witnesses')
+ receipt['consumer_results'][config] = cases
+ bp = pathlib.Path(run(config+'-binpath', ['swift','build','-c',config,'--package-path','Examples/BoundedWorkflows','--scratch-path','.build/workflow-consumer','--show-bin-path']).strip())
+ modules = bp/'Modules' if (bp/'Modules').exists() else bp
+ guide = ROOT/'Sources/JSONInspection/Documentation.docc/BoundedWorkflowRecipes.md'
+ blocks = re.findall(r'```swift\n(.*?)\n```', guide.read_text(), re.S)
+ if len(blocks) != 1: raise ValueError('Changed recipe snippet inventory')
+ source = out/'guide.swift'; source.write_text(blocks[0]+'\n')
+ run(config+'-guide', ['swiftc','-swift-version','6','-warnings-as-errors','-I',str(modules),'-Onone' if config == 'debug' else '-O',
+ '-parse-as-library','-c',str(source),'-o',str(out/(config+'-guide.o'))])
+ capacity = out/'capacity'
+ run('capacity', ['swift','run','-c','release','--jobs','2','--package-path','Examples/BoundedWorkflows','--scratch-path','.build/workflow-consumer','BoundedWorkflows','--capacity',str(capacity)])
+ data = json.loads((capacity/'cells.json').read_text())
+ if not validate_capacity(data): raise ValueError('Changed/incomplete capacity observations or boundary proof')
+ receipt['fixtures'] = validate_fixtures(capacity/'fixtures')
+ receipt['capacity'] = data
+ receipt['completed_cells'] = sum(r['outcome'] == 'completed' for r in data['cells'])
+ receipt['operational_refusals'] = sum(r['outcome'] == 'operational_error' for r in data['cells'])
+ receipt['capacity_status'] = 'observed_fixture_grid_complete' if not receipt['operational_refusals'] else 'partial_diagnostic_capacity'
+ receipt['performance_status'] = 'NOT MEASURED: owned counters are not timings, heap allocations, RSS or throughput'
+ receipt['status'] = 'passed'
+ except (OSError, ValueError, subprocess.TimeoutExpired) as error:
+ receipt.update(status='failed', error=str(error))
+ receipt['source_inputs_unchanged'] = receipt['source_inputs'] == [dict(path=p.relative_to(ROOT).as_posix(),sha256=hashlib.sha256(p.read_bytes()).hexdigest()) for p in files()]
+ if not receipt['source_inputs_unchanged']: receipt['status'] = 'failed'
+ if receipt['status'] == 'passed' and not check_receipt(receipt): receipt.update(status='failed', error='Incomplete final acceptance receipt')
+ save(); return 0 if receipt['status'] == 'passed' else 1
+
+if __name__ == '__main__': raise SystemExit(main())
diff --git a/Scripts/test_qualification_harness.py b/Scripts/test_qualification_harness.py
index d766277..254b3e7 100644
--- a/Scripts/test_qualification_harness.py
+++ b/Scripts/test_qualification_harness.py
@@ -28,6 +28,54 @@ def store_markers():
'STORE_RUNTIME R%02d passed' % n for n in range(1,11)])
class HarnessTests(unittest.TestCase):
+ def test_acceptance18_discovery_preserves_exact_public_pipeline_identities(self):
+ names = DISCOVERY['structural_acceptance_function_ids']
+ self.assertEqual(len(names), 9)
+ self.assertEqual(len(DISCOVERY['integration_function_ids']), 43)
+ events = dict(bundle_counts=[9], module_function_counts={'JSONIntegrationTests':9},
+ function_names=names, parameter_case_invocations=0)
+ command = ['swift','test','--test-product','JSONIntegrationTests','--filter','acceptance18']
+ self.assertTrue(check_result(command, 0, '', events)[0])
+ full = dict(events, bundle_counts=[43], module_function_counts={'JSONIntegrationTests':43}, function_names=DISCOVERY['integration_function_ids'])
+ full_command = ['swift','test','--test-product','JSONIntegrationTests','--filter','JSONIntegrationTests']
+ self.assertTrue(check_result(full_command, 0, '', full)[0])
+ self.assertFalse(check_result(full_command, 0, '', dict(full,function_names=full['function_names'][:-1]))[0])
+ for broken in [dict(events, function_names=names[:-1]), dict(events, function_names=names+names[:1]),
+ dict(events, module_function_counts={'JSONInspectionTests':9}), dict(events, parameter_case_invocations=1)]:
+ self.assertFalse(check_result(command, 0, '', broken)[0])
+
+ def test_capacity18_does_not_convert_refusals_or_incomplete_boundaries_to_passes(self):
+ from qualify_structural_acceptance import MATRIX, validate_capacity, validate_fixtures
+ cells = []
+ for identifier in MATRIX['cells']:
+ operation, count = identifier.rsplit('-',1); count=int(count)
+ cells.append(dict(id=identifier, operation=operation, requests=count,
+ fixture='chain-%d.json' % count if operation=='follow-chain' else 'fixed-eight.json',
+ outcome='completed', report=dict(work=1,nodes=1,retained=32,peak=32),
+ independentSingles=None if operation=='follow-chain' else dict(workSum=count,nodesSum=count,maximumInvocationRetained=32,maximumInvocationPeak=32),
+ requestBoundary=True,workBoundary=True,retentionBoundary=True))
+ data=dict(schemaVersion=1,profile='balanced',cells=cells)
+ self.assertTrue(validate_capacity(data))
+ refused=copy.deepcopy(data);refused['cells'][-1]=dict(refused['cells'][-1],outcome='operational_error',report=None,
+ requestBoundary=None,workBoundary=None,retentionBoundary=None,errorMetric='workUnits',errorLimit=1,errorAttempted=2,errorStage='construction')
+ self.assertTrue(validate_capacity(refused))
+ for mode in ['missing','duplicate','fake-boundary','bad-single-count','required-refusal','fake-metric','refusal-as-success','missing-report']:
+ bad=copy.deepcopy(refused)
+ if mode=='missing':bad['cells'].pop()
+ elif mode=='duplicate':bad['cells'][-1]=bad['cells'][0]
+ elif mode=='fake-boundary':bad['cells'][0]['workBoundary']=False
+ elif mode=='bad-single-count':bad['cells'][0]['independentSingles']['nodesSum']=2
+ elif mode=='required-refusal':bad['cells'][3]=dict(bad['cells'][-1],id='follow-chain-1',requests=1,fixture='chain-1.json')
+ elif mode=='fake-metric':bad['cells'][-1]['errorMetric']='unrelated'
+ elif mode=='refusal-as-success':bad['cells'][-1]['outcome']='completed'
+ else:bad['cells'][0]['report']=None
+ self.assertFalse(validate_capacity(bad))
+ with tempfile.TemporaryDirectory() as temporary:
+ directory=pathlib.Path(temporary)
+ with self.assertRaisesRegex(ValueError,'inventory'):validate_fixtures(directory)
+ for fixture in MATRIX['fixtures']:(directory/fixture['name']).write_bytes(b'corrupt')
+ with self.assertRaisesRegex(ValueError,'bytes'):validate_fixtures(directory)
+
def test_workflow_discovery_requires_all_exact_identities(self):
names = DISCOVERY['workflow_function_ids']
self.assertEqual(len(names), 24)
@@ -195,12 +243,12 @@ def test_composition_discovery_and_capacity_limits_are_not_blanket_successes(sel
self.assertEqual(policy[key]['JSONNavigationTests'], 57)
self.assertEqual(policy[key]['JSONSchemaExamplesTests'], 98)
self.assertEqual(policy[key]['JSONInspectionTests'], 214)
- self.assertEqual(policy[key]['JSONIntegrationTests'], 34)
+ self.assertEqual(policy[key]['JSONIntegrationTests'], 43)
for key in ('shared_state_functions', 'portable_shared_state_functions'):
self.assertEqual(policy[key]['JSONNavigationTests'], 2)
self.assertEqual(policy[key]['JSONSchemaExamplesTests'], 4)
self.assertEqual(policy[key]['JSONInspectionTests'], 11)
- self.assertEqual(policy[key]['JSONIntegrationTests'], 11)
+ self.assertEqual(policy[key]['JSONIntegrationTests'], 12)
def test_private_projection_identities_cannot_be_missing_renamed_or_duplicated(self):
names = sorted(PROJECTION_FUNCTIONS)
@@ -391,8 +439,8 @@ def test_exact_discovery_rejects_missing_duplicate_and_changed_parameters(self):
identity['function_names'][0] = 'JSONTests.unrelatedReplacement()'
self.assertFalse(check_restored(identity))
harness = ['python3', '-B', 'Scripts/test_qualification_harness.py']
- self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=28')[0])
- for stale in ['HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', '']:
+ self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=30')[0])
+ for stale in ['HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', 'HARNESS_TESTS passed cases=28', '']:
self.assertFalse(check_result(harness, 0, stale)[0])
lifetime = ['python3', '-B', 'Scripts/qualify_schema_example_lifetime.py']
marker = 'SCHEMA_EXAMPLES_CORPUS passed mode=owned scopes=128 values=6 exact=1'
diff --git a/Sources/JSONCLIEngine/CLIBuildIdentity.swift b/Sources/JSONCLIEngine/CLIBuildIdentity.swift
index 8af8093..2c10613 100644
--- a/Sources/JSONCLIEngine/CLIBuildIdentity.swift
+++ b/Sources/JSONCLIEngine/CLIBuildIdentity.swift
@@ -2,6 +2,6 @@
// Generated by Scripts/generate_cli_identity.py; excludes this file.
// SHA-256 of the ordered Package.swift + Sources content manifest, not a Git commit.
package enum CLIBuildIdentity {
- package static let revision = "sha256:43023670d6664373702741d960b010494d86149aaa181b4eeb9d6ff13701d60e"
+ package static let revision = "sha256:d29a80776f26331bf75b98c24d5df1cf5b69ecfbc4d9e39426d393e23c02c3ec"
package static let kind = "source-sha256"
}
diff --git a/Sources/JSONInspection/Documentation.docc/BoundedWorkflowRecipes.md b/Sources/JSONInspection/Documentation.docc/BoundedWorkflowRecipes.md
new file mode 100644
index 0000000..f559e8f
--- /dev/null
+++ b/Sources/JSONInspection/Documentation.docc/BoundedWorkflowRecipes.md
@@ -0,0 +1,67 @@
+# Combine checked structural workflows
+
+Compose existing authorities while preserving exact values and each operation's limits.
+
+## Validate, then interpret records
+
+JSONSchema owns schema semantics. A valid verdict does not construct a store;
+JSONInspection separately checks the explicit collection, ID and link profile.
+An ordinary false verdict is a completed evaluation. A thrown operational error
+must not be converted to false or an empty store.
+
+```swift
+import JSON
+import JSONSchema
+import JSONInspection
+
+func validatedStore(document: JSONDocument, schema: CompiledJSONSchema,
+ profile: JSONStoreProfile) throws -> JSONConfiguredStore? {
+ let validation = try JSONSchemaValidator().validateWithReport(document.root, against: schema)
+ guard validation.result.isValid else { return nil } // Application's explicit invalid-input result.
+ return try JSONStoreBuilder(document: document, profile: profile).build().result
+}
+```
+
+Compile the schema against an explicit immutable offline registry. If a schema
+contains only definitions, select the intended definition before validation;
+the store does not invent a schema root or infer required properties.
+
+## Read, transform and write
+
+The executable `Examples/BoundedWorkflows` validates an exact source, constructs
+a store, follows declared references, batch-reads the closure, runs the existing
+JSONTransform compiler/VM and writes an independent byte-level golden result.
+`9007199254740993` and `0.10000000000000001` never pass through Double.
+Caller collection assembly between stages is ordinary caller work. Each public
+stage has its own bounded report; this recipe does not create one pipeline-wide
+ledger or a caller-instruction/RSS bound. Within a batch/follow/transform invocation,
+all nested owned work still uses the original cumulative ledger.
+
+A caught transform resource failure remains terminal for that execution. It is
+not EOF, JSON null or schema invalidity. A later operation on the immutable source
+has independent health. With an awaited streaming sink, already accepted bytes
+remain accepted after cancellation; stage output in the caller if publication
+must be atomic.
+
+## Patch and rebuild deliberately
+
+Apply JSONPatch to the original canonical value, then use
+`JSONDocument.replacingRoot(_:)` to capture the result. The replacement has no
+invented original source bytes/spans. Rebuild its store and obtain new positions;
+old owner-bound tokens are refused even when most records are unchanged. A patch
+which creates a dangling declared link cannot publish a successful checked store.
+Root removal is `JSONPatchResult.removed`, distinct from a present JSON null.
+
+## Interpret capacity evidence
+
+The public consumer's `--capacity OUTPUT_DIRECTORY` mode uses fixed 1/8/64/512
+request cells and small growing chains, unchanged balanced native limits and
+explicit finite follow bounds. It checks exact work/retention and request boundaries,
+compares independently admitted reads and writes fixture bytes for independent
+hash verification. Logical counters are not heap allocations, RSS or elapsed time.
+A diagnostic resource refusal remains an incomplete capacity observation. These
+fixtures do not prove the default maxima jointly achievable or a speed improvement.
+
+The existing developer kit stays pinned to its published library version. This
+1.8 candidate recipe is compiled separately; it does not silently change that kit's
+compatibility or distribution claims.
diff --git a/Sources/JSONInspection/Documentation.docc/JSONInspection.md b/Sources/JSONInspection/Documentation.docc/JSONInspection.md
index ad9572a..405526a 100644
--- a/Sources/JSONInspection/Documentation.docc/JSONInspection.md
+++ b/Sources/JSONInspection/Documentation.docc/JSONInspection.md
@@ -61,6 +61,7 @@ or numeric matrix. Sampling does not exempt any source from admission.
### Bounded structural workflows (1.8 candidate)
-
+-
- ``JSONInspectionBatch``
- ``JSONInspectionBatchFailure``
- ``JSONStoreBuilder``
diff --git a/Tests/JSONIntegrationTests/CompositionContractTests.swift b/Tests/JSONIntegrationTests/CompositionContractTests.swift
index c650339..caa665b 100644
--- a/Tests/JSONIntegrationTests/CompositionContractTests.swift
+++ b/Tests/JSONIntegrationTests/CompositionContractTests.swift
@@ -249,7 +249,9 @@ private func composedImport(limits: JSONLimits = .balanced, extra: UInt64 = 0) t
_ = try JSON.withComposition(limits: limits) { scope in
try scope.withMeter { meter in
meter.checkpointObserver = { work in
- if work >= (stage == 5 ? 4200 : 32) { withUnsafeCurrentTask { $0?.cancel() } }
+ // Test-only current-task handle stays inside its standard-library callback.
+ // Only this invocation is cancelled; no unsafe handle escapes or is shared.
+ if work >= (stage == 5 ? 4200 : 32) { unsafe withUnsafeCurrentTask { unsafe $0?.cancel() } }
}
}
do {
diff --git a/Tests/JSONIntegrationTests/PublicWorkflowTests.swift b/Tests/JSONIntegrationTests/PublicWorkflowTests.swift
index b6aa799..0fef152 100644
--- a/Tests/JSONIntegrationTests/PublicWorkflowTests.swift
+++ b/Tests/JSONIntegrationTests/PublicWorkflowTests.swift
@@ -28,7 +28,9 @@ private actor Output {
calls += 1
if calls == rejectAt { throw AppError.sink }
bytes.append(contentsOf: chunk.bytes)
- if calls == cancelAt { withUnsafeCurrentTask { $0?.cancel() } }
+ // Test-only current-task handle stays inside its standard-library callback.
+ // Only this invocation is cancelled; no unsafe handle escapes or is shared.
+ if calls == cancelAt { unsafe withUnsafeCurrentTask { unsafe $0?.cancel() } }
}
func snapshot() -> ([UInt8], Int) { (bytes, calls) }
}
diff --git a/Tests/JSONIntegrationTests/StagedImportTests.swift b/Tests/JSONIntegrationTests/StagedImportTests.swift
index ed05876..739cce2 100644
--- a/Tests/JSONIntegrationTests/StagedImportTests.swift
+++ b/Tests/JSONIntegrationTests/StagedImportTests.swift
@@ -13,7 +13,9 @@ private actor ImportSource {
func next() throws -> JSONByteChunk? {
pulls += 1
if pulls == failAt { throw ImportError.source }
- if pulls == cancelAt { withUnsafeCurrentTask { $0?.cancel() } }
+ // Test-only current-task handle stays inside its standard-library callback.
+ // Only this invocation is cancelled; no unsafe handle escapes or is shared.
+ if pulls == cancelAt { unsafe withUnsafeCurrentTask { unsafe $0?.cancel() } }
return pulls <= texts.count ? .init(Array(texts[pulls - 1].utf8)) : nil
}
func count() -> Int { pulls }
diff --git a/Tests/JSONIntegrationTests/StructuralAcceptanceTests.swift b/Tests/JSONIntegrationTests/StructuralAcceptanceTests.swift
new file mode 100644
index 0000000..6720187
--- /dev/null
+++ b/Tests/JSONIntegrationTests/StructuralAcceptanceTests.swift
@@ -0,0 +1,209 @@
+// SPDX-License-Identifier: Apache-2.0
+import Foundation
+import Testing
+import JSON
+import JSONSchema
+import JSONPatch
+import JSONStreaming
+import JSONTransform
+import JSONInspection
+
+private let acceptance18Input = #"{"u":[{"id":"é","amount":9007199254740993,"next":{"collection":"u","id":"e\u0301"}},{"id":"e\u0301","amount":0.10000000000000001,"next":null}]}"#
+private func acceptance18Source() throws -> JSONDocument {
+ try JSONParser().parseDocument(Data(acceptance18Input.utf8))
+}
+private func acceptance18Profile() -> JSONStoreProfile {
+ .init(version: .v1, collectionsAt: [], collections: [
+ .init(name: "u", recordsAt: [.key("u")], idKey: "id", referenceFields: [
+ .init(key: "next", presence: .required, nullable: true)])
+ ], references: .strict(marker: .v1, validateTargets: true))
+}
+private func acceptance18Plan() throws -> CompiledJSONSchema {
+ try JSONSchemaCompiler().compile(JSONSchemaDocument(JSONParser().parse(
+ #"{"type":"object","required":["u"],"properties":{"u":{"type":"array","minItems":2,"items":{"type":"object","required":["id","amount","next"],"properties":{"id":{"type":"string"},"amount":{"type":"number"},"next":{"type":["object","null"]}}}}}}"#)))
+}
+private func acceptance18Store(_ source: JSONDocument) throws -> JSONConfiguredStore {
+ try JSONStoreBuilder(document: source, profile: acceptance18Profile()).build().result
+}
+private enum Acceptance18Error: Error { case expectedFailure, removed }
+
+@Test func acceptance18A01SchemaStoreTransformKeepsExactGoldenBytes() throws {
+ let source = try acceptance18Source(), schema = try acceptance18Plan()
+ let verdict = try JSONSchemaValidator().validateWithReport(source.root, against: schema)
+ #expect(verdict.result.isValid && verdict.statistics[.evaluationWorkUnits] > 0)
+ let store = try acceptance18Store(source)
+ let start = try #require(store.lookup(collection: "u", id: "é").result)
+ let closure = try store.follow(from: start), records = try store.values(at: closure.result)
+ let output = try JSONTransformProgram.compile(".[] | .amount").withExecution(input: .array(records.result)) { execution in
+ // This array is authored caller storage, not owned transform output accounting.
+ var values: [JSON] = []
+ while let item = try execution.next() { values.append(item.value) }
+ return JSON.array(values)
+ }
+ #expect(try JSONWriter(options: .init(numberStyle: .preserved)).write(output.result) == Data("[9007199254740993,0.10000000000000001]".utf8))
+ #expect(records.statistics[.nodes] == (try JSONInspectionBatch(document: source).values(at: []).statistics[.nodes]))
+ #expect(closure.result.count == 2 && output.statistics[.workUnits] > 0)
+}
+
+@Test func acceptance18A02InvalidSchemaVerdictIsDistinctFromOperationalRefusal() throws {
+ let plan = try acceptance18Plan()
+ let invalid = try JSONSchemaValidator().validate(42, against: plan)
+ #expect(!invalid.isValid && !invalid.issues.isEmpty)
+ var limits = JSONLimits.balanced; limits[.evaluationWorkUnits] = 0
+ do {
+ _ = try JSONSchemaValidator(options: .init(limits: limits)).validate(acceptance18Source().root, against: plan)
+ Issue.record("An exhausted evaluator returned a verdict")
+ } catch let error as JSONFailure {
+ #expect(error.category == .resource && error.resource?.metric == .evaluationWorkUnits)
+ }
+ #expect(try JSONSchemaValidator().validate(acceptance18Source().root, against: plan).isValid)
+}
+
+@Test func acceptance18A03PatchedRebuildHasNewOwnerAndNoInventedSourceMap() throws {
+ let source = try acceptance18Source(), original = try acceptance18Store(source)
+ let position = try #require(original.lookup(collection: "u", id: "e\u{301}").result)
+ let amount = try JSONNumber(parsing: "1.25")
+ let patch = try JSONPatch(operations: [.replace(path: JSONPointer(parsing: "/u/1/amount"), value: .number(amount))])
+ guard case .document(let root) = try patch.applying(to: source.root) else { throw Acceptance18Error.removed }
+ let changed = source.replacingRoot(root), rebuilt = try acceptance18Store(changed)
+ #expect(changed.sourceMap == nil && changed.sourceBytes == nil && original.document.sourceMap != nil)
+ do { _ = try rebuilt.values(at: [position]); Issue.record("Rebound an original position") }
+ catch let error as JSONStoreFailure { #expect(error.code == .invalidPosition) }
+ let replacement = try #require(rebuilt.lookup(collection: "u", id: "e\u{301}").result)
+ #expect(try rebuilt.value(at: replacement).result.requiredMember("amount") == .number(amount))
+ #expect(try original.value(at: position).result.requiredMember("amount") == .number(JSONNumber(parsing: "0.10000000000000001")))
+}
+
+@Test func acceptance18A04DanglingPatchedReferenceCannotPublishStore() throws {
+ let source = try acceptance18Source(), original = try acceptance18Store(source)
+ let patch = try JSONPatch(operations: [.replace(path: JSONPointer(parsing: "/u/1/id"), value: "renamed")])
+ guard case .document(let root) = try patch.applying(to: source.root) else { throw Acceptance18Error.removed }
+ do { _ = try acceptance18Store(source.replacingRoot(root)); Issue.record("Published a dangling reference") }
+ catch let error as JSONStoreFailure {
+ #expect(error.code == .referenceDanglingID && error.phase == 7)
+ #expect(error.path == [.key("u"),.index(0),.key("next"),.key("id")])
+ #expect(error.sourceSpan == nil)
+ }
+ let first = try #require(original.lookup(collection: "u", id: "é").result)
+ #expect(try original.follow(from: first).result.count == 2 && source.root == original.document.root)
+}
+
+@Test func acceptance18A05ScalarCollectionNamesSurvivePatchAndCrossLinks() throws {
+ let source = try JSONParser().parseDocument(Data(#"{"é":[{"id":"a","v":1,"go":{"collection":"e\u0301","id":"a"}}],"e\u0301":[{"id":"a","v":2,"go":{"collection":"é","id":"a"}}]}"#.utf8))
+ let profile = JSONStoreProfile(version: .v1, collectionsAt: [], collections: ["e\u{301}", "é"].map { name in
+ .init(name: JSONKey(name), recordsAt: [.key(JSONKey(name))], idKey: "id", referenceFields: [.init(key: "go", presence: .required, nullable: false)])
+ }, references: .strict(marker: .v1, validateTargets: true))
+ let store = try JSONStoreBuilder(document: source, profile: profile).build().result
+ let a = try #require(store.lookup(collection: "é", id: "a").result)
+ let b = try #require(store.lookup(collection: JSONKey("e\u{301}"), id: "a").result)
+ #expect(a != b && a.collection == 0 && b.collection == 1)
+ #expect(try store.follow(from: a).result == [a,b])
+ let patch = try JSONPatch(operations: [.replace(path: JSONPointer(parsing: "/é/0/v"), value: 3)])
+ guard case .document(let root) = try patch.applying(to: source.root) else { throw Acceptance18Error.removed }
+ let selected = try JSONInspectionBatch(document: source.replacingRoot(root)).values(at: [
+ [.key("é"),.index(0),.key("v")], [.key(JSONKey("e\u{301}")),.index(0),.key("v")]])
+ #expect(selected.result == [3,2])
+}
+
+@Test func acceptance18A06TransformFailureStaysTerminalOverStoreInput() throws {
+ let store = try acceptance18Store(acceptance18Source())
+ let positions = [try store.record(collection: 0, row: 0).position, try store.record(collection: 0, row: 1).position]
+ let values = try store.values(at: positions), plan = try JSONTransformProgram.compile(".[] | .amount")
+ var limits = JSONTransformLimits(); limits[.results] = 1
+ var reachedFailure = false
+ do {
+ _ = try plan.withExecution(input: .array(values.result), limits: limits) { execution in
+ #expect(try execution.next()?.value == .number(JSONNumber(parsing: "9007199254740993")))
+ let firstWork = execution.statistics[.workUnits], remaining = execution.remainingWorkUnits
+ for _ in 0..<2 {
+ do { _ = try execution.next(); Issue.record("A failed execution resumed") }
+ catch let error as JSONTransformFailure {
+ #expect(error.category == .resource && error.metric == "results" && error.limit == 1)
+ reachedFailure = true
+ }
+ }
+ #expect(execution.statistics[.workUnits] >= firstWork && execution.remainingWorkUnits <= remaining)
+ }
+ Issue.record("Caught failure allowed successful completion")
+ } catch let error as JSONTransformFailure { #expect(error.category == .resource && error.metric == "results") }
+ #expect(reachedFailure)
+ #expect(try store.values(at: positions).result == values.result)
+}
+
+@Test func acceptance18A07RootRemovalIsNotNullOrAnEmptyStore() throws {
+ let source = try acceptance18Source(), pointer = try JSONPointer(parsing: "")
+ let removed = try JSONPatch(operations: [.remove(path: pointer)]).applying(to: source.root)
+ let null = try JSONPatch(operations: [.replace(path: pointer, value: .null)]).applying(to: source.root)
+ #expect(removed == .removed && null == .document(.null) && removed != null)
+ guard case .document(let root) = null else { throw Acceptance18Error.removed }
+ do { _ = try acceptance18Store(source.replacingRoot(root)); Issue.record("Null became an empty store") }
+ catch let error as JSONStoreFailure { #expect(error.code == .collectionsNotObject) }
+}
+
+private actor Acceptance18Sink {
+ var bytes: [UInt8] = [], calls = 0
+ private var accepted: CheckedContinuation?
+ private var handoff: CheckedContinuation?
+ func accept(_ chunk: JSONByteChunk) async {
+ calls += 1; bytes.append(contentsOf: chunk.bytes)
+ await withCheckedContinuation { continuation in
+ handoff = continuation
+ accepted?.resume(); accepted = nil
+ }
+ }
+ func waitForAcceptance() async {
+ if handoff != nil { return }
+ await withCheckedContinuation { accepted = $0 }
+ }
+ func completeAcceptance() {
+ handoff?.resume(); handoff = nil
+ }
+ func snapshot() -> ([UInt8], Int) { (bytes, calls) }
+}
+@Test func acceptance18A08CancelledWriterKeepsOnlyItsAcceptedPrefix() async throws {
+ let store = try acceptance18Store(acceptance18Source()), sink = Acceptance18Sink()
+ let position = try store.record(collection: 0, row: 0).position
+ let values = try store.values(at: [position]), value = JSON.array(values.result)
+ let golden = try JSONWriter().write(value)
+ let task = Task { try await JSONAsyncWriter(chunkSize: 4).write(value) { chunk in await sink.accept(chunk) } }
+ await sink.waitForAcceptance()
+ task.cancel()
+ await sink.completeAcceptance()
+ do { try await task.value; Issue.record("Expected deterministic cancellation") } catch is CancellationError {}
+ let accepted = await sink.snapshot()
+ #expect(accepted.1 == 1 && accepted.0 == Array(golden.prefix(4)))
+ #expect(try store.values(at: [position]).result == values.result)
+}
+
+@Test func acceptance18SharedPlansKeepIndependentCrossProductHealth() async throws {
+ let source = try acceptance18Source(), schema = try acceptance18Plan(), store = try acceptance18Store(source)
+ let transform = try JSONTransformProgram.compile(".[] | .amount"), position = try store.record(collection: 0, row: 0).position
+ let transformInput = JSON.array(try store.values(at: [position]).result)
+ try await withThrowingTaskGroup(of: Bool.self) { group in
+ for index in 0..<12 { group.addTask {
+ switch index % 4 {
+ case 0:
+ guard try JSONSchemaValidator().validate(source.root, against: schema).isValid else { return false }
+ let values = try store.values(at: [position])
+ return try transform.withExecution(input: .array(values.result)) { execution in
+ let value = try execution.next()?.value
+ return try value == .number(JSONNumber(parsing: "9007199254740993")) && execution.next() == nil
+ }.result
+ case 1:
+ var limits = JSONLimits.balanced; limits[.evaluationWorkUnits] = 0
+ do { _ = try JSONSchemaValidator(options: .init(limits: limits)).validate(source.root, against: schema); return false }
+ catch let error as JSONFailure { return error.resource?.metric == .evaluationWorkUnits }
+ case 2:
+ var limits = JSONLimits.balanced; limits[.workUnits] = 0
+ do { _ = try store.values(at: [position], limits: limits); return false }
+ catch let error as JSONFailure { return error.resource?.metric == .workUnits }
+ default:
+ var limits = JSONTransformLimits(); limits[.results] = 0
+ do { _ = try transform.withExecution(input: transformInput, limits: limits) { try $0.next() }; return false }
+ catch let error as JSONTransformFailure { return error.metric == "results" }
+ }
+ } }
+ for try await success in group { #expect(success) }
+ }
+ #expect(try JSONSchemaValidator().validate(source.root, against: schema).isValid)
+}
From 70df9ba8d47322abf0d4aec1439256d7a5e97e94 Mon Sep 17 00:00:00 2001
From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com>
Date: Thu, 29 Oct 2026 10:51:56 +0100
Subject: [PATCH 3/6] Add bounded Transform source definitions and compile
reports
---
AGENTS.md | 6 +
.../Release1.8/TransformBindings.md | 96 ++++++
.../Qualification/TestDiscovery.json | 53 +++-
.../Qualification/TransformBindings1.8.md | 89 ++++++
Package.swift | 2 +-
Scripts/ci.py | 40 ++-
Scripts/qualify_transform_bindings.py | 55 ++++
Scripts/test_qualification_harness.py | 17 +-
Sources/JSONCLIEngine/CLIBuildIdentity.swift | 2 +-
.../Documentation.docc/JSONTransform.md | 13 +-
.../Documentation.docc/SourceDefinitions.md | 41 +++
.../JSONTransform/JSONTransformProgram.swift | 25 +-
.../JSONTransformProgramDefinition.swift | 266 +++++++++++++++++
.../JSONTransformTests/DefinitionTests.swift | 205 +++++++++++++
.../IndexedSelectionSpike.swift | 276 ++++++++++++++++++
.../Package.swift | 8 +
Tools/TransformBindingQualification/README.md | 8 +
.../TransformBindingConsumer/Main.swift | 42 +++
18 files changed, 1226 insertions(+), 18 deletions(-)
create mode 100644 Documentation/Architecture/Release1.8/TransformBindings.md
create mode 100644 Documentation/Qualification/TransformBindings1.8.md
create mode 100644 Scripts/qualify_transform_bindings.py
create mode 100644 Sources/JSONTransform/Documentation.docc/SourceDefinitions.md
create mode 100644 Sources/JSONTransform/JSONTransformProgramDefinition.swift
create mode 100644 Tests/JSONTransformTests/DefinitionTests.swift
create mode 100644 Tests/JSONTransformTests/IndexedSelectionSpike.swift
create mode 100644 Tools/TransformBindingQualification/Package.swift
create mode 100644 Tools/TransformBindingQualification/README.md
create mode 100644 Tools/TransformBindingQualification/Sources/TransformBindingConsumer/Main.swift
diff --git a/AGENTS.md b/AGENTS.md
index 88f7f62..444b376 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -63,3 +63,9 @@ Preserve its accepted 42 authored declarations and exact semantics. The original
1.7.0 P4 waiver and later partial measurements are historical scoped evidence;
no allocation, speed, RSS or controlled-performance pass follows from consolidation.
See Documentation/Qualification/Consolidation2026.md for combined gates and holds.
+
+The commissioned 1.8 Transform binding train adds a source-definition codec and
+compile reports while preserving those original declarations. Indexed selection
+is a test-only private-kernel ownership experiment in Sprint 1, not public API.
+See Documentation/Architecture/Release1.8/TransformBindings.md. No Content code,
+lock, message, merge or publication is authorized by implementing this sprint.
diff --git a/Documentation/Architecture/Release1.8/TransformBindings.md b/Documentation/Architecture/Release1.8/TransformBindings.md
new file mode 100644
index 0000000..6b01420
--- /dev/null
+++ b/Documentation/Architecture/Release1.8/TransformBindings.md
@@ -0,0 +1,96 @@
+# Transform bindings — commissioned 1.8 candidate, Sprint 1
+
+The maintainer commissioned the source-definition codec, indexed selection and
+compile reporting on 29 October 2026. Base is b8ddeb26042005a06360f50fcf20b1f840e66d1c.
+This additive train preserves the existing 42 authored Transform declarations,
+ordinary execution/CLI semantics, thirteen products and empty root dependencies.
+Implementation and a successful spike are not release or downstream admission.
+
+## Definition V1
+
+JSONTransformProgramDefinition stores exact source and unique ASCII variable names
+in ascending UTF-8 order. It does not add source storage to compiled plans. There
+is no synthesized String-based equality/hash or serialized VM/host state.
+
+Canonical encoding is compact UTF-8 without a BOM or newline, with fields in this
+order: definition, version, dialect, source, variables. Their values are
+"swift-json-transform-source", mathematical integer 1, "swift-json-query-v1", the
+exact source string, and the ordered name array. Native JSONWriter supplies all
+escaping: direct non-ASCII UTF-8, escaped quote/backslash, lowercase control-byte
+escapes. This format is distinct from an implementation digest and execution policy.
+
+Decoding accepts one strict native JSON document with exactly those five decoded
+keys and correct field types. Equivalent JSON escaping, field order, whitespace
+and numeric spelling of version 1 are accepted; re-encoding is canonical. Unknown
+fields, duplicate decoded keys/names and unknown versions/dialects fail. This is
+strict semantic decoding, not a canonical-input assertion. A storage gateway that
+requires canonical input must compare against bounded canonical output under its
+own explicitly reviewed policy. Query syntax is checked by the existing compiler,
+not the codec. Names use [A-Za-z_][A-Za-z0-9_]* without dollar prefixes and are
+validated before any Set conversion. No credentials, files, closures, Content
+objects, SCCV/SCCD data, artifact hashes or online resolution are encoded.
+
+The finite codec defaults are source 1 MiB, 128 names, 128 bytes/name, 8192 total name
+bytes and 2 MiB wire bytes, alongside the existing balanced JSON limits. Native work
+meters sorting, parsing, scanning, construction and writing; retention is reserved
+before owned growth. Limits are checked at invocation entry; helpers share that
+one meter. Existing 49 core policies and their probes are unchanged. Logical bytes
+are reservations, not RSS. Supplied input storage and later caller copies are caller
+work. Prepare, decode, encode and compile are explicitly separate operations; no
+decode-and-compile convenience secretly restarts a single composed invocation.
+
+Successful preparation/codec operations return native reports. Compilation uses
+the existing compiler/context and returns actual native work. The additive
+compileWithReport path preserves old compile behavior; the definition adapter
+also charges its owned variable Set construction. No public meter is exposed.
+
+## Indexed selection contract for Sprint 2
+
+The proposed convention is an invocation-owned ordered array [0,...,N-1] as input
+and a reserved explicit values binding containing the complete population. The
+selection convention needs its own revision, separate from the language/wire
+revisions. Reserved-name collisions and provider policy must be frozen before
+public selection API admission. Content-specific mapping/identity stays outside.
+
+The existing query engine alone filters/sorts; it must explicitly emit numeric
+ordinals. Native mathematical integer conversion admits 42.0 and negative zero as
+their exact integer identities; fractions, negative values, other kinds, overflow,
+native-width failure and out-of-population values fail. Output order/repeats are
+preserved. Deterministic tie keys belong in the program; no hidden deduplication.
+
+One invocation must own population admission, index construction, VM evaluation,
+numeric checks and typed output slots. Position handles need an unforgeable owner
+and a joint population lease even when the numeric output was newly computed.
+An explicit EOF state is required to certify completion. A caught boundary error
+or abandoned prefix cannot produce a completion witness. Generic JSONOperationReport
+has a public pairing initializer and is not that witness; public completion will
+need a private initializer and owner-produced statistics. Query catches cannot
+swallow boundary failures. First cause, cancellation and terminal lease refunds
+must remain observable without reopening execution. Handles stay guarded and
+non-Sendable; immutable plans/definitions remain checked Sendable.
+
+Sprint 1 keeps this experiment solely in IndexedSelectionSpike.swift in the test
+target. It reuses FilterVM/TransformRuntimeContext and native values; it is not a
+second interpreter or a shipped selection facade. Its 1024 population bound and
+test provider are experimental, not approved public defaults/provider semantics.
+Full reentry/public compiler-negative/lifetime/profile qualification belongs to
+the next increment. General provenance propagation, persistence of executions,
+unbounded input/sort guarantees and Content implementation are outside scope.
+
+## Qualification and admission
+
+Run Scripts/ci.py lanes transform-bindings and transform-bindings-safety for this
+sprint's actual cases and public consumer. Preserve the original 23 Transform cases,
+all old suites/fixtures and CLI checks. Source/CLI identity, public declarations,
+native strict-memory checks and DocC must bind the final checkpoint. Full Native,
+Linux ARM64/x86_64, WASI and relevant Apple checks are required for the eventual
+release; unavailable/unrun lanes stay open. Earlier 1.8 receipts and 1.7 performance
+waivers do not qualify these source changes. Existing performance/allocation holds
+remain. No merge, tag, publication or cross-session message is commissioned here.
+
+The downstream S06/S13 profile mapping and deliberate Content lock selection remain
+separate decisions after a qualified public binding. M0 stays independent; no S04
+history, Content repository, source pin or admission gate is changed by this train.
+
+The [Sprint 1 qualification record](../../Qualification/TransformBindings1.8.md)
+separates completed local checks from the remaining public-selection/release gates.
diff --git a/Documentation/Qualification/TestDiscovery.json b/Documentation/Qualification/TestDiscovery.json
index 3251689..60b4b1e 100644
--- a/Documentation/Qualification/TestDiscovery.json
+++ b/Documentation/Qualification/TestDiscovery.json
@@ -13,7 +13,7 @@
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
"JSONInspectionTests": 214,
- "JSONTransformTests": 23,
+ "JSONTransformTests": 41,
"JSONDSLTests": 47
},
"parameterized_invocations": 390,
@@ -38,7 +38,7 @@
"JSONNavigationTests": 2,
"JSONSchemaExamplesTests": 4,
"JSONInspectionTests": 11,
- "JSONTransformTests": 1,
+ "JSONTransformTests": 3,
"JSONDSLTests": 1
},
"coverage_unmapped": {
@@ -85,7 +85,7 @@
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
"JSONInspectionTests": 214,
- "JSONTransformTests": 23,
+ "JSONTransformTests": 41,
"JSONDSLTests": 47
},
"wasi_root_functions": {
@@ -100,7 +100,7 @@
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
"JSONInspectionTests": 214,
- "JSONTransformTests": 23,
+ "JSONTransformTests": 41,
"JSONDSLTests": 47
},
"wasi_parameterized_invocations": 353,
@@ -116,7 +116,7 @@
"JSONNavigationTests": 2,
"JSONSchemaExamplesTests": 4,
"JSONInspectionTests": 11,
- "JSONTransformTests": 1,
+ "JSONTransformTests": 3,
"JSONDSLTests": 1
},
"export_functions": {
@@ -230,5 +230,48 @@
"JSONIntegrationTests.qaE2ESharedPlanRecoversAfterIndependentFailures()",
"JSONIntegrationTests.qaE2EUnknownEnvelopeVersionIsCorruption()",
"JSONIntegrationTests.qaE2EVersionOneEnvelopePreservesExactInnerState()"
+ ],
+ "transform_function_ids": [
+ "JSONTransformTests.bindingD01CanonicalGoldenBytesAndReload()",
+ "JSONTransformTests.bindingD02CanonicalEscapingAndExactScalarSource()",
+ "JSONTransformTests.bindingD03StrictSemanticDecodeCanonicalizesPresentation()",
+ "JSONTransformTests.bindingD04ClosedFieldsVersionsAndKindsArePrecise()",
+ "JSONTransformTests.bindingD05NativeGrammarAndDuplicateFieldsStayAuthoritative()",
+ "JSONTransformTests.bindingD06NamesRejectBeforeSetConversion()",
+ "JSONTransformTests.bindingD07SpecificCapsAreInclusiveAndFinite()",
+ "JSONTransformTests.bindingD08WireAndOwnedWorkRetentionHaveExactBoundaries()",
+ "JSONTransformTests.bindingD09CompileReportsPreserveExistingPlanBehaviorAndBudget()",
+ "JSONTransformTests.bindingD10CodecDoesNotInventQueryValidationOrDecimalAuthority()",
+ "JSONTransformTests.bindingD11CancellationOccursInsideOwnedCodecWork()",
+ "JSONTransformTests.bindingP01ExplicitIndicesPreserveDuplicatesOrderAndTies()",
+ "JSONTransformTests.bindingP02InvalidOutputAndAbandonedPrefixNeverCertify()",
+ "JSONTransformTests.bindingP03PopulationLeaseSurvivesCloseAndForeignOwnerRefuses()",
+ "JSONTransformTests.bindingP04ResourceAndProviderFailuresStayTerminal()",
+ "JSONTransformTests.bindingP05ProjectionEvaluationAndCancellationShareOneLedger()",
+ "JSONTransformTests.bindingSharedDefinitionAndCompilationHaveIndependentHealth()",
+ "JSONTransformTests.bindingSharedSelectionSpikeReusesOnePlanIndependently()",
+ "JSONTransformTests.jtxBindingNamesUseTheCompilerIdentifierGrammar()",
+ "JSONTransformTests.jtxBindingsAndProviderAreExplicitAndLazy()",
+ "JSONTransformTests.jtxCancellationLatchesAndStopsProviderCalls()",
+ "JSONTransformTests.jtxCompiledLiteralsCannotBypassTighterExecutionValueLimits()",
+ "JSONTransformTests.jtxConfiguredCompilerGrowthHasInclusiveBoundaries()",
+ "JSONTransformTests.jtxCopiedHandlesShareOneCumulativeLedger()",
+ "JSONTransformTests.jtxCumulativeProviderAndResultCapsCannotResetPerPull()",
+ "JSONTransformTests.jtxEscapedHandleInvalidatesAndReleasesProvider()",
+ "JSONTransformTests.jtxExactNumbersAndScalarKeysKeepCoreAuthority()",
+ "JSONTransformTests.jtxOutputAdmissionWorkAndTemporaryFramesStayOnTheOneLedger()",
+ "JSONTransformTests.jtxPlanIsSendableAndExecutionsHaveIndependentHealth()",
+ "JSONTransformTests.jtxProviderCanObserveSnapshotAndCancelAfterOwnedProgress()",
+ "JSONTransformTests.jtxProviderFailureIsOperationalAndNeverCaughtByLanguageTry()",
+ "JSONTransformTests.jtxPublicEngineMatchesNativeExactDialectMatrix()",
+ "JSONTransformTests.jtxReentrantProviderFailureLatchesWithoutAccessConflict()",
+ "JSONTransformTests.jtxReportsUseCoreDomainAndCallerErrorRetainsIdentity()",
+ "JSONTransformTests.jtxResourceErrorBypassesCatchAndLatchesAfterHostCatch()",
+ "JSONTransformTests.jtxRuntimeCapsRejectBeforePublishingAndRemainTyped()",
+ "JSONTransformTests.jtxSelectedAliasKeepsSourceReservationUntilLastHandleDies()",
+ "JSONTransformTests.jtxTaskCancellationIsObservedAfterProviderOwnedProgress()",
+ "JSONTransformTests.jtxTerminalAliasesRefundBeforeThrowingTheSameFailure()",
+ "JSONTransformTests.jtxTypedCompileLocationAndLanguageFailureStayDistinct()",
+ "JSONTransformTests.jtxZeroNullAndManyAreDistinctOrderedPullResults()"
]
}
diff --git a/Documentation/Qualification/TransformBindings1.8.md b/Documentation/Qualification/TransformBindings1.8.md
new file mode 100644
index 0000000..37bd195
--- /dev/null
+++ b/Documentation/Qualification/TransformBindings1.8.md
@@ -0,0 +1,89 @@
+# Transform bindings 1.8 — first sprint
+
+Status: implemented and scoped native qualification complete; full release
+qualification remains PARTIAL. This isolated candidate extends
+`b8ddeb26042005a06360f50fcf20b1f840e66d1c` on
+`codex/1.8-transform-bindings-s1`. The original batches/store candidate, its
+PR and all earlier source-bound receipts remain independent and unchanged.
+No merge, tag, publication, Content pin or downstream admission is implied.
+
+The [contract](../Architecture/Release1.8/TransformBindings.md) defines the
+candidate wire format and next selection boundary. Public additions are the
+immutable source definition, finite definition policy, typed definition failures,
+codec operations and compile-with-report route. Existing JSON/decimal/Unicode
+and query semantics remain authoritative. The indexed-selection feasibility code
+exists only in the test target; no public position, completion or selection API
+ships in this sprint.
+
+## Observed qualification — 29 October 2026
+
+Supported host: macOS 27.0 arm64, Apple Swift 6.4.2
+(`swiftlang-6.4.2.1.9`, driver 1.168.6), Xcode 27.2/27B48, SDK 27.2,
+Swift language mode 6. Raw commands, exits, events, source manifests and failed
+attempts are retained privately under `Artifacts/TransformBindingsS1`.
+
+| Check | Observed outcome |
+| --- | --- |
+| `Scripts/ci.py --lane transform-bindings` | Passed: 41 functions in Debug and Release; original 23 preserved; 31 Python harness cases; public privacy/identity checks |
+| `Scripts/ci.py --lane transform-bindings-safety` | Passed: all 41 Release ASan functions; three shared-plan TSan functions; strict-memory target build with warnings as errors |
+| New public-only consumer outside Git | Passed: six witnesses in each configuration, one exact compiled DocC fence, four ordinary full-codegen constructor/immutability refusals |
+| Existing public Transform consumer | Passed: original twelve witnesses in each configuration |
+| Existing Transform ownership compiler cases | Passed: seven cases; intended compiler refusals distinguished from import failures/crashes |
+| `Scripts/ci.py --lane cli` | Passed: 13 commands, 88 functions/37 parameter invocations and all existing command/vector/provider checks |
+| Existing manifest/dependency/cap checks | Passed through unchanged CI comparators: thirteen products, empty external graph, one test reaching all 49 core caps with exact outcomes |
+| Scoped JSONTransform native symbol graph and DocC | Passed: 67 authored declarations and parameter comments; original 42 identities/signatures preserved; warnings as errors |
+
+The public graph adds 25 authored declarations. Three historical graph fragment
+differences are retained and explicitly reviewed: two nested types acquire their
+parent qualification while retaining the same precise type IDs and byte-identical
+owning sources; generic Result metadata loses a local preciseIdentifier with
+unchanged spelling. The original withExecution declaration/body is byte-identical.
+This is a scoped Transform catalog result, not a fresh thirteen-catalog pass.
+
+Initial compiler errors were in new test assertions with throwing expressions;
+they remain in the private logs. The first native harness run rejected a stale
+30-case self-test expectation after a meaningful 31st case was added. The
+successor checks both the new exact population and rejection of stale populations.
+The first strict graph comparison retained the three rendering differences above;
+its source-backed successor records their precise disposition. No failed attempt
+is relabeled as a passed check or omitted from the return.
+
+## Semantic and ownership coverage
+
+D01–D03 pin literal canonical bytes, native escaping, exact decoded source scalars,
+UTF-8 name ordering and canonical reload. D04–D06 reject closed-field/type/version/
+dialect/name errors, exact duplicates and malformed native JSON. D07–D11 pin zero,
+exact, one-over and overflow-policy boundaries, work/retention accounting, compile
+reports, compiler errors and deterministic inner-loop cancellation. Shared cases
+exercise independent concurrent reuse. The reports describe actual native work;
+caller storage/instructions and process RSS remain outside those claims.
+
+The P01–P05 test-only selection prototype exercises explicit indices over duplicate
+values, order/repeats/ties, empty completion, invalid late output bypassing query
+catch, incomplete-prefix refusal, foreign owner refusal, post-close leases/refunds,
+terminal quota/provider failures and cumulative projection/evaluation/cancellation.
+It uses the existing VM and private context, not a second evaluator. Its profile,
+provider and completion types are experiments, not public contracts or conformance.
+
+## Next bounded increment and held gates
+
+Sprint 2 must freeze reserved binding/provider policy and a versioned selection
+convention, implement a guarded public selection scope with private position and
+completion constructors, and qualify reentry, public constructor refusals, aliases,
+first-cause terminal accounting, exact inclusive capacities and independent plans.
+Retain the admitted population through every output's joint lease. Full EOF is
+required before an owner-produced completion witness. Ordinary execution stays
+unchanged. The downstream S06/S13 mapping remains a separate deliberate decision.
+
+Fresh full Native/Linux ARM64/x86_64/safety/docs, WASI, relevant Apple SDK/runtime,
+archive reproduction and complete API checks remain final-release gates. The
+portable harness is prepared but this sprint makes no new Linux/WASI/device pass.
+The existing 49 policies/probes are unchanged; the codec's five finite capacities
+are additive policy, not replacements. No new benchmark/allocation campaign ran.
+Existing 1.8 loaded-host comparisons, unresolved regressions, unmeasured allocations
+and older scoped holds remain visible; the 1.7 waiver does not apply to 1.8.
+
+The final checkpoint binding separately records this documentary closeout and
+architecture text polish after the passing receipts. Product, manifest, tests,
+DocC, fixtures, generator, CLI identity and executable harness bytes remain equal
+to those checked. No independent human review or release approval is claimed.
diff --git a/Package.swift b/Package.swift
index 164b251..6aa75ee 100644
--- a/Package.swift
+++ b/Package.swift
@@ -43,7 +43,7 @@ let package = Package(
.target(name: "JSONHTML", dependencies: ["JSON"], exclude: ["Assets"]),
.target(name: "JSONFilterRuntime", dependencies: ["JSON"]),
.target(name: "JSONTransform", dependencies: ["JSON", "JSONFilterRuntime"]),
- .testTarget(name: "JSONTransformTests", dependencies: ["JSON", "JSONTransform"]),
+ .testTarget(name: "JSONTransformTests", dependencies: ["JSON", "JSONTransform", "JSONFilterRuntime"]),
.target(name: "JSONCLIEngine", dependencies: ["JSON", "JSONSchema", "JSONStreaming", "JSONPatch", "JSONHTML", "JSONFilterRuntime", "JSONTransform"]),
.executableTarget(name: "JSONCLI", dependencies: ["JSONCLIEngine"]),
.testTarget(name: "JSONCLITests", dependencies: ["JSONCLIEngine", "JSONFilterRuntime", "JSON", "JSONPatch", "JSONSchema", "JSONHTML"]),
diff --git a/Scripts/ci.py b/Scripts/ci.py
index 4cc12c0..09aa283 100644
--- a/Scripts/ci.py
+++ b/Scripts/ci.py
@@ -28,6 +28,7 @@
SHARED_TESTS += '|graphG30ConcurrentViewsHaveIndependentHealth|graphGP14ConcurrentQueries'
SHARED_TESTS += '|dslSharedFinalizationAndPlansAreInvocationOwned'
SHARED_TESTS += '|storeR06TerminalQuotasNeverPublishAndImmutableConcurrentHealth'
+SHARED_TESTS += '|bindingSharedDefinitionAndCompilationHaveIndependentHealth|bindingSharedSelectionSpikeReusesOnePlanIndependently'
SHARED_TESTS += '|workflowSharedImmutablePlansHaveIndependentBatchAndStoreHealth'
SHARED_TESTS += '|acceptance18SharedPlansKeepIndependentCrossProductHealth'
R0_TESTS = 'readerProbe|indexProbes|duplicateRecognitionPrecedesValueAndClosingSyntax'
@@ -38,6 +39,23 @@ def snapshot():
def commands(lane, output):
# The qualification executable resolves output paths from its repository root.
output = output.relative_to(ROOT)
+ if lane in ('transform-bindings', 'transform-bindings-safety'):
+ def portable(values):
+ return [c + ['--disable-xctest'] if sys.platform == 'linux' and c[:2] == ['swift', 'test'] else c for c in values]
+ if lane.endswith('safety'):
+ return portable([
+ ['swift', 'test', '--test-product', 'JSONTransformTests', '-c', 'release', '--jobs', '2', '--sanitize', 'address', '--scratch-path', '.build/bindings-address'],
+ ['swift', 'test', '--test-product', 'JSONTransformTests', '-c', 'release', '--jobs', '2', '--sanitize', 'thread', '--scratch-path', '.build/bindings-thread', '--filter', 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth|bindingShared'],
+ ['swift', 'build', '--target', 'JSONTransform', '--jobs', '2', '--scratch-path', '.build/bindings-strict', '-Xswiftc', '-strict-memory-safety', '-Xswiftc', '-warnings-as-errors'],
+ ])
+ return portable([
+ ['python3', '-B', 'Scripts/public_check.py'],
+ ['python3', '-B', 'Scripts/generate_cli_identity.py', '--check'],
+ ['python3', '-B', 'Scripts/test_qualification_harness.py'],
+ ['swift', 'test', '--test-product', 'JSONTransformTests', '--jobs', '2'],
+ ['swift', 'test', '--test-product', 'JSONTransformTests', '-c', 'release', '--jobs', '2'],
+ ['python3', '-B', 'Scripts/qualify_transform_bindings.py', '--output', str(output / 'public-consumer')],
+ ])
if lane in ('structural-acceptance', 'structural-acceptance-safety'):
if lane.endswith('safety'):
result = [
@@ -245,6 +263,7 @@ def commands(lane, output):
['python3', '-B', 'Scripts/test_transform_ownership.py'],
['python3', '-B', 'Scripts/qualify_transform_ownership.py', '--output', str(output / 'transform-ownership')],
['python3', '-B', 'Scripts/qualify_transform.py', '--output', str(output / 'transform-consumer')],
+ ['python3', '-B', 'Scripts/qualify_transform_bindings.py', '--output', str(output / 'transform-binding-consumer')],
['python3', '-B', 'Scripts/qualify_release_1_1.py', '--mode', 'ownership', '--output', str(output / 'ownership.json')],
['python3', '-B', 'Scripts/qualify_release_1_1.py', '--mode', 'reader', '--output', str(output / 'reader.json')],
['swift', 'run', '-c', 'release', '--package-path', 'Examples/Consumer', 'Consumer'],
@@ -335,6 +354,14 @@ def check_result(command, code, log, events=None):
if 'JSONSchemaTests' in command and events is not None:
parameters = {name: ROOT_PARAMETER_CASES[name] for name in ['successfulAlternativesCannotSwallowRegexExhaustion(keyword:)', 'URIIdentityUsesSyntaxNormalizationWithoutUnicodeFolding(input:expected:)']}
passed = passed and events['module_function_counts'] == {'JSONSchemaTests': 79} and events['parameterized_functions'] == parameters and events['parameter_case_invocations'] == 12
+ if 'JSONTransformTests' in command:
+ ids = set(DISCOVERY['transform_function_ids'])
+ if 'thread' in command:
+ ids = {n for n in ids if 'bindingShared' in n or 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth' in n}
+ elif selected_filter == 'binding':
+ ids = {n for n in ids if '.binding' in n}
+ expected = [len(ids)]
+ passed = passed and events is not None and events['module_function_counts'] == {'JSONTransformTests': len(ids)} and set(events['function_names']) == ids and len(events['function_names']) == len(ids) and events['parameter_case_invocations'] == 0
if 'navigation' in command:
expected = [DISCOVERY['root_functions']['JSONNavigationTests']]
if events is not None:
@@ -375,7 +402,7 @@ def check_result(command, code, log, events=None):
passed = passed and events['module_function_counts'] == {'CompositionScaleTests': DISCOVERY['composition_external_functions']} and events['parameter_case_invocations'] == 0
if 'qa' in command and events is not None:
passed = passed and events['module_function_counts'] == DISCOVERY['qa_functions'] and check_restored(events)
- if 'thread' in command and 'JSONSchemaExamplesTests' not in command and not prepared_test and not store_test and not workflow_test and not acceptance_test and events is not None:
+ if 'thread' in command and 'JSONSchemaExamplesTests' not in command and 'JSONTransformTests' not in command and not prepared_test and not store_test and not workflow_test and not acceptance_test and events is not None:
passed = passed and events['module_function_counts'] == SHARED_COUNTS and check_projection_cases(events, shared=True) and check_p2_cases(events, shared=True) and check_exact_cases(events, shared=True) and check_graph_cases(events, shared=True) and events['parameter_case_invocations'] == 0 and check_store_tests(events, log, shared=True)
if '--filter' not in command and '--test-product' not in command and not any(x.startswith('Tools/') for x in command):
if events is not None:
@@ -413,7 +440,7 @@ def check_result(command, code, log, events=None):
passed = passed and set(targets) == {'JSON', 'JSONNavigation', 'JSONNavigationTests', 'JSONInspection', 'JSONInspectionTests', 'JSONIntegrationTests', 'JSONBridges', 'JSONBridgesTests', 'JSONSchema', 'JSONSchemaExamples', 'JSONSchemaExamplesTests', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONRegexRuntime', 'JSONTests', 'JSONSchemaTests', 'JSONStreamingTests', 'JSONPatchTests', 'JSONHTMLTests', 'JSONRegexRuntimeTests', 'JSONFilterRuntime', 'JSONCLIEngine', 'JSONCLI', 'JSONCLITests', 'JSONTransform', 'JSONTransformTests', 'JSONDSLCore', 'JSONValueDSL', 'JSONSchemaDSL', 'JSONDSLTests'} and targets['JSONHTML']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONBridges']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONFilterRuntime']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONCLI']['type'] == 'executable'
passed = passed and targets['JSONNavigation']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONNavigationTests']['dependencies'] == [{'byName': ['JSONNavigation', None]}, {'byName': ['JSON', None]}]
passed = passed and targets['JSONSchemaExamples']['dependencies'] == [{'byName': ['JSON', None]}, {'byName': ['JSONSchema', None]}] and targets['JSON']['dependencies'] == [] and targets['JSONSchema']['dependencies'] == [{'byName': ['JSON', None]}, {'byName': ['JSONRegexRuntime', None]}]
- passed = passed and targets['JSONTransform']['dependencies'] == [{'byName': ['JSON', None]}, {'byName': ['JSONFilterRuntime', None]}] and targets['JSONTransformTests']['dependencies'] == [{'byName': ['JSON', None]}, {'byName': ['JSONTransform', None]}]
+ passed = passed and targets['JSONTransform']['dependencies'] == [{'byName': ['JSON', None]}, {'byName': ['JSONFilterRuntime', None]}] and targets['JSONTransformTests']['dependencies'] == [{'byName': ['JSON', None]}, {'byName': ['JSONTransform', None]}, {'byName': ['JSONFilterRuntime', None]}]
passed = passed and targets['JSONInspection']['dependencies'] == [{'byName': ['JSON', None]}, {'byName': ['JSONNavigation', None]}] and targets['JSONInspectionTests']['dependencies'] == [{'byName': ['JSONInspection', None]}, {'byName': ['JSONNavigation', None]}, {'byName': ['JSON', None]}]
for name, dependencies in [('JSONDSLCore', ['JSON']), ('JSONValueDSL', ['JSON', 'JSONDSLCore']), ('JSONSchemaDSL', ['JSON', 'JSONSchema', 'JSONDSLCore']), ('JSONDSLTests', ['JSON', 'JSONSchema', 'JSONDSLCore', 'JSONValueDSL', 'JSONSchemaDSL'])]:
passed = passed and targets[name]['dependencies'] == [{'byName': [item, None]} for item in dependencies]
@@ -501,12 +528,15 @@ def check_result(command, code, log, events=None):
if command[:3] == ['python3', '-B', 'Scripts/qualify_html_browser.py']:
passed = passed and 'JSONHTML_BROWSER passed engines=3 cases=76 skips=0' in log.splitlines()
if 'Scripts/test_qualification_harness.py' in command:
- passed = passed and 'HARNESS_TESTS passed cases=30' in log.splitlines()
+ passed = passed and 'HARNESS_TESTS passed cases=31' in log.splitlines()
+ if command[:3] == ['python3', '-B', 'Scripts/qualify_transform_bindings.py']:
+ data = json.loads(log.strip().splitlines()[-1])
+ passed = passed and data['status'] == 'passed' and len(data['commands']) == 8 and all(x['passed'] for x in data['commands']) and data['witnesses_per_configuration'] == 6 and data['negative_fixture_invocations'] == 4 and data['compiled_guide_fences'] == 1
return passed, counts
def main():
parser = argparse.ArgumentParser(description=__doc__)
- parser.add_argument('--lane', choices=['structural-acceptance', 'structural-acceptance-safety', 'structural-workflows', 'structural-workflows-safety', 'store', 'store-preparation', 'composition', 'navigation', 'inspection', 'native', 'safety', 'docs', 'r0', 'qa', 'linux', 'linux-safety', 'wasm', 'browser', 'export', 'cli', 'schema-examples', 'schema-examples-safety', 'schema-examples-determinism', 'schema-examples-construction', 'schema-examples-advanced', 'schema-examples-qualification', 'coverage', 'linux-coverage', 'linux-qa', 'wasm-qa', 'archive'], required=True)
+ parser.add_argument('--lane', choices=['transform-bindings', 'transform-bindings-safety', 'structural-acceptance', 'structural-acceptance-safety', 'structural-workflows', 'structural-workflows-safety', 'store', 'store-preparation', 'composition', 'navigation', 'inspection', 'native', 'safety', 'docs', 'r0', 'qa', 'linux', 'linux-safety', 'wasm', 'browser', 'export', 'cli', 'schema-examples', 'schema-examples-safety', 'schema-examples-determinism', 'schema-examples-construction', 'schema-examples-advanced', 'schema-examples-qualification', 'coverage', 'linux-coverage', 'linux-qa', 'wasm-qa', 'archive'], required=True)
parser.add_argument('--output')
parser.add_argument('--skip-host-measurements', action='store_true', help='For WASI runtime qualification only: record optional host timings as NOT RUN.')
args = parser.parse_args()
@@ -518,7 +548,7 @@ def main():
repository = subprocess.run(['git', 'rev-parse', '--show-toplevel'], cwd=ROOT, capture_output=True, text=True)
is_checkout = repository.returncode == 0 and pathlib.Path(repository.stdout.strip()).resolve() == ROOT.resolve()
commit = subprocess.run(['git', 'rev-parse', 'HEAD'], cwd=ROOT, capture_output=True, text=True) if is_checkout else None
- environment = collect_linux() if args.lane.startswith('linux') or (args.lane in ('composition', 'inspection', 'structural-workflows', 'structural-workflows-safety', 'structural-acceptance', 'structural-acceptance-safety') and sys.platform == 'linux') else collect()
+ environment = collect_linux() if args.lane.startswith('linux') or (args.lane in ('transform-bindings', 'transform-bindings-safety', 'composition', 'inspection', 'structural-workflows', 'structural-workflows-safety', 'structural-acceptance', 'structural-acceptance-safety') and sys.platform == 'linux') else collect()
receipt = {'schema_version': 1, 'lane': args.lane, 'status': 'blocked', 'environment': environment, 'started_utc': datetime.datetime.now(datetime.timezone.utc).isoformat(), 'git_commit': commit.stdout.strip() if commit is not None and commit.returncode == 0 else None, 'github_run_id': os.environ.get('GITHUB_RUN_ID'), 'source_inputs': inputs, 'source_manifest_sha256': hashlib.sha256(json.dumps(inputs, sort_keys=True).encode()).hexdigest(), 'commands': []}
def save():
(output / 'receipt.json').write_text(json.dumps(receipt, indent=2) + '\n')
diff --git a/Scripts/qualify_transform_bindings.py b/Scripts/qualify_transform_bindings.py
new file mode 100644
index 0000000..53098c8
--- /dev/null
+++ b/Scripts/qualify_transform_bindings.py
@@ -0,0 +1,55 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Public-only definition consumer and ordinary full-codegen constructor refusals."""
+import argparse, hashlib, json, pathlib, platform, re, shutil, subprocess, sys, tempfile, textwrap
+from qualify_transform_ownership import ordinary_rejection
+ROOT = pathlib.Path(__file__).resolve().parents[1]
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__); parser.add_argument('--output', required=True)
+ args = parser.parse_args(); output = (ROOT/args.output).resolve(); output.mkdir(parents=True, exist_ok=True)
+ guide=(ROOT/'Sources/JSONTransform/Documentation.docc/SourceDefinitions.md').read_text()
+ fences=re.findall(r'```swift\n(.*?)\n```',guide,re.S)
+ consumer_text=(ROOT/'Tools/TransformBindingQualification/Sources/TransformBindingConsumer/Main.swift').read_text()
+ body=re.search(r'func guideExample\(\) throws \{\n(.*?)\n\}\n@main',consumer_text,re.S).group(1)
+ assert len(fences)==1 and textwrap.dedent(body).strip()=='\n'.join(line for line in fences[0].splitlines() if not line.startswith('import ')).strip()
+ commands = []
+ def run(command, name):
+ result = subprocess.run(command,cwd=ROOT,capture_output=True,text=True)
+ data=result.stdout+result.stderr; path=output/(name+'.log');path.write_text(data)
+ row=dict(command=command,exit_code=result.returncode,log=path.name,sha256=hashlib.sha256(path.read_bytes()).hexdigest())
+ commands.append(row); return result,row
+ with tempfile.TemporaryDirectory(prefix='swift-json-binding-consumer-') as temporary:
+ consumer=pathlib.Path(temporary)
+ for p in (ROOT/'Tools/TransformBindingQualification').rglob('*'):
+ if p.is_file() and not any(x in p.parts for x in ('.build','.swiftpm')):
+ target=consumer/p.relative_to(ROOT/'Tools/TransformBindingQualification');target.parent.mkdir(parents=True,exist_ok=True);target.write_bytes(p.read_bytes())
+ manifest=consumer/'Package.swift';manifest.write_text(manifest.read_text().replace('path: "../.."','path: '+json.dumps(str(ROOT))))
+ inputs=[dict(path=p.relative_to(consumer).as_posix(),sha256=hashlib.sha256(p.read_bytes()).hexdigest()) for p in sorted(consumer.rglob('*')) if p.is_file()]
+ for config in ['debug','release']:
+ result,row=run(['swift','run','--package-path',str(consumer),'-c',config,'--jobs','2','TransformBindingConsumer'],config+'-consumer')
+ row['passed']=result.returncode==0 and result.stdout.splitlines()==['TRANSFORM_BINDINGS passed witnesses=6']
+ if not row['passed']:break
+ result,row=run(['swift','build','--package-path',str(consumer),'-c',config,'--show-bin-path'],config+'-binpath')
+ row['passed']=result.returncode==0
+ if not row['passed']:break
+ modules=pathlib.Path(result.stdout.strip()); binary=modules/'TransformBindingConsumer'
+ target=output/(config+'-TransformBindingConsumer');shutil.copyfile(binary,target)
+ row['binary_sha256']=hashlib.sha256(target.read_bytes()).hexdigest()
+ base=['swiftc','-swift-version','6','-I',str(modules),'-module-cache-path',str(output/'ModuleCache'),'-c']
+ if sys.platform=='darwin':
+ sdk=subprocess.check_output(['xcrun','--sdk','macosx','--show-sdk-path'],text=True).strip()
+ base+=['-sdk',sdk,'-target',platform.machine()+'-apple-macos27.0','-F',str(modules/'PackageFrameworks')]
+ for name,source,diagnostic in [
+ ('immutable','import JSONTransform\nfunc check(_ d: JSONTransformProgramDefinition) { d.source = "changed" }\n','cannot assign to property'),
+ ('constructor','import JSONTransform\nlet d = JSONTransformProgramDefinition(validatedSource: ".", canonicalNames: [])\n',"inaccessible due to 'private'")]:
+ path=output/(config+'-'+name+'.swift');path.write_text(source)
+ result,row=run(base+[str(path),'-o',str(output/(config+'-'+name+'.o'))],config+'-'+name)
+ row['expected_diagnostic']=diagnostic;row['passed']=ordinary_rejection(result.returncode,result.stderr,diagnostic)
+ passed=len(commands)==8 and all(x['passed'] for x in commands)
+ receipt=dict(status='passed' if passed else 'failed',commands=commands,source_inputs=inputs,compiled_guide_fences=1,
+ configurations=2,witnesses_per_configuration=6,negative_fixture_invocations=4,
+ scope='Outside-Git public-only path consumer; no private imports or released-version resolution.')
+ (output/'receipt.json').write_text(json.dumps(receipt,indent=2)+'\n')
+ print(json.dumps(receipt));return 0 if passed else 1
+if __name__=='__main__':raise SystemExit(main())
diff --git a/Scripts/test_qualification_harness.py b/Scripts/test_qualification_harness.py
index 254b3e7..ed4e9f0 100644
--- a/Scripts/test_qualification_harness.py
+++ b/Scripts/test_qualification_harness.py
@@ -28,6 +28,19 @@ def store_markers():
'STORE_RUNTIME R%02d passed' % n for n in range(1,11)])
class HarnessTests(unittest.TestCase):
+ def test_transform_binding_discovery_rejects_incomplete_duplicate_and_foreign_events(self):
+ ids = DISCOVERY['transform_function_ids']
+ command = ['swift', 'test', '--test-product', 'JSONTransformTests']
+ events = dict(bundle_counts=[41], module_function_counts={'JSONTransformTests':41},
+ function_names=ids, parameter_case_invocations=0)
+ self.assertTrue(check_result(command, 0, '', events)[0])
+ for broken in [dict(events,function_names=ids[:-1]),dict(events,function_names=ids+ids[:1]),
+ dict(events,module_function_counts={'JSONTests':41}),dict(events,parameter_case_invocations=1)]:
+ self.assertFalse(check_result(command, 0, '', broken)[0])
+ shared = [n for n in ids if 'bindingShared' in n or 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth' in n]
+ thread = command + ['--sanitize', 'thread']
+ self.assertTrue(check_result(thread, 0, '', dict(events,bundle_counts=[3],module_function_counts={'JSONTransformTests':3},function_names=shared))[0])
+
def test_acceptance18_discovery_preserves_exact_public_pipeline_identities(self):
names = DISCOVERY['structural_acceptance_function_ids']
self.assertEqual(len(names), 9)
@@ -439,8 +452,8 @@ def test_exact_discovery_rejects_missing_duplicate_and_changed_parameters(self):
identity['function_names'][0] = 'JSONTests.unrelatedReplacement()'
self.assertFalse(check_restored(identity))
harness = ['python3', '-B', 'Scripts/test_qualification_harness.py']
- self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=30')[0])
- for stale in ['HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', 'HARNESS_TESTS passed cases=28', '']:
+ self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=31')[0])
+ for stale in ['HARNESS_TESTS passed cases=30', 'HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', 'HARNESS_TESTS passed cases=28', '']:
self.assertFalse(check_result(harness, 0, stale)[0])
lifetime = ['python3', '-B', 'Scripts/qualify_schema_example_lifetime.py']
marker = 'SCHEMA_EXAMPLES_CORPUS passed mode=owned scopes=128 values=6 exact=1'
diff --git a/Sources/JSONCLIEngine/CLIBuildIdentity.swift b/Sources/JSONCLIEngine/CLIBuildIdentity.swift
index 2c10613..a2f826c 100644
--- a/Sources/JSONCLIEngine/CLIBuildIdentity.swift
+++ b/Sources/JSONCLIEngine/CLIBuildIdentity.swift
@@ -2,6 +2,6 @@
// Generated by Scripts/generate_cli_identity.py; excludes this file.
// SHA-256 of the ordered Package.swift + Sources content manifest, not a Git commit.
package enum CLIBuildIdentity {
- package static let revision = "sha256:d29a80776f26331bf75b98c24d5df1cf5b69ecfbc4d9e39426d393e23c02c3ec"
+ package static let revision = "sha256:2e6e326d160776afbff186a2c68d01968dc866159ede10038593be415098d91d"
package static let kind = "source-sha256"
}
diff --git a/Sources/JSONTransform/Documentation.docc/JSONTransform.md b/Sources/JSONTransform/Documentation.docc/JSONTransform.md
index ee828d3..0dfbfc2 100644
--- a/Sources/JSONTransform/Documentation.docc/JSONTransform.md
+++ b/Sources/JSONTransform/Documentation.docc/JSONTransform.md
@@ -2,9 +2,9 @@
Compile the existing exact query dialect and pull bounded native JSON results.
-This is the optional transformation library in the unreleased 1.7 source line.
-Development main and a separately versioned candidate retain their own source
-identities and qualification. Publication requires a separate maintainer decision. It shares one private compiler
+The library was introduced in 1.7.0. This 1.8 candidate adds exact source definitions
+and successful compile reports; indexed selection remains a test-only experiment.
+Source identities and qualification stay separate. Publication requires a separate maintainer decision. It shares one private compiler
and VM with `swift-json query`; it does not introduce a new interpreter or numeric
authority. Plans are immutable and Sendable. Each synchronous invocation has one
finite cumulative meter and a guarded, non-Sendable execution handle.
@@ -18,6 +18,13 @@ finite cumulative meter and a guarded, non-Sendable execution handle.
- ``JSONTransformExecution``
- ``JSONTransformOutput``
+### Source definitions
+
+-
+- ``JSONTransformProgramDefinition``
+- ``JSONTransformDefinitionLimits``
+- ``JSONTransformDefinitionFailure``
+
### Limits and failures
- ``JSONTransformLimits``
diff --git a/Sources/JSONTransform/Documentation.docc/SourceDefinitions.md b/Sources/JSONTransform/Documentation.docc/SourceDefinitions.md
new file mode 100644
index 0000000..fda7777
--- /dev/null
+++ b/Sources/JSONTransform/Documentation.docc/SourceDefinitions.md
@@ -0,0 +1,41 @@
+# Persist exact source definitions
+
+Store a versioned source definition without serializing compiled VM state.
+Definitions are immutable and checked Sendable; source keeps its decoded scalar
+spelling and declared ASCII names are unique in ascending UTF-8 order.
+
+```swift
+import JSON
+import JSONTransform
+
+let definition = try JSONTransformProgramDefinition.prepare(
+ source: "$amount", variables: ["amount"]).result
+let bytes = try definition.encode().result
+let restored = try JSONTransformProgramDefinition.decode(bytes).result
+let compiled = try restored.compile()
+let amount = JSON.number(try JSONNumber(parsing: "9007199254740993"))
+let output = try compiled.result.withExecution(input: .null, bindings: ["amount": amount]) {
+ try $0.next()?.value
+}
+precondition(output.result == amount)
+precondition(compiled.statistics[.workUnits] > 0)
+```
+
+Prepare, decode, encode and compile each have their own explicit finite operation.
+Within an operation all helpers share one ledger. Codec limits bound source/names,
+wire bytes, native traversal, sorting and logical retention. Execution limits are
+supplied separately. Logical reservations are not RSS or caller instruction quotas.
+
+The closed V1 object contains definition, version, dialect, source and variables
+in that canonical order. Strict decoding accepts equivalent native JSON spelling
+and field order, then encoding emits canonical compact bytes. It rejects unknown
+fields, duplicate keys/names and unsupported revisions. Canonical-input checking
+is a separate gateway policy; codec decoding alone does not assert byte identity.
+Malformed JSON remains a native JSON error; invalid query syntax is a compiler
+error when compilation is explicitly requested. Neither schema validation nor a
+new numeric or query engine is introduced.
+
+Use ``JSONTransformProgram/compileWithReport(_:variables:limits:)`` when compiling
+ordinary source with successful native work statistics. These reports are data;
+they are not unforgeable selection-completion witnesses. The indexed selection
+experiment remains test-only in this sprint, with no public selection API.
diff --git a/Sources/JSONTransform/JSONTransformProgram.swift b/Sources/JSONTransform/JSONTransformProgram.swift
index a0c5cfb..bd06c3c 100644
--- a/Sources/JSONTransform/JSONTransformProgram.swift
+++ b/Sources/JSONTransform/JSONTransformProgram.swift
@@ -20,6 +20,30 @@ public struct JSONTransformProgram: Sendable {
limits: JSONTransformLimits = .init()) throws -> Self {
do {
var context = try TransformRuntimeContext(limits: limits.runtime)
+ return try compileValidated(source, variables: variables, context: &context)
+ } catch { throw JSONTransformFailure.classify(error) }
+ }
+
+ /// Compiles with the same compiler and returns actual successful native work.
+ /// Compilation is independent of later execution. Statistics are data, not a
+ /// selection-completion witness or a charge for caller-owned work.
+ /// - Parameters:
+ /// - source: Exact query source in the existing dialect.
+ /// - variables: Declared ASCII names without dollar prefixes.
+ /// - limits: Finite compilation policy shared by validation and compilation.
+ /// - Returns: The immutable program and this compilation's native statistics.
+ /// - Throws: The same typed compilation, configuration and operational failures as compile.
+ public static func compileWithReport(_ source: String, variables: Set = [],
+ limits: JSONTransformLimits = .init()) throws -> JSONOperationReport {
+ do {
+ var context = try TransformRuntimeContext(limits: limits.runtime)
+ let result = try compileValidated(source, variables: variables, context: &context)
+ return .init(result: result, statistics: context.meter.statistics())
+ } catch { throw JSONTransformFailure.classify(error) }
+ }
+
+ static func compileValidated(_ source: String, variables: Set,
+ context: inout TransformRuntimeContext) throws -> Self {
for name in variables {
try context.charge(.bindings)
try JSONAdmission.text(name, key: true, meter: &context.meter, owned: true)
@@ -36,7 +60,6 @@ public struct JSONTransformProgram: Sendable {
}
let program = try FilterCompiler.compile(source, variables: variables, context: &context)
return .init(program: program, planReservation: context.meter.used(.retainedBytes), variables: variables)
- } catch { throw JSONTransformFailure.classify(error) }
}
/// Runs a synchronous, invocation-owned guarded pull scope.
diff --git a/Sources/JSONTransform/JSONTransformProgramDefinition.swift b/Sources/JSONTransform/JSONTransformProgramDefinition.swift
new file mode 100644
index 0000000..bdcd32f
--- /dev/null
+++ b/Sources/JSONTransform/JSONTransformProgramDefinition.swift
@@ -0,0 +1,266 @@
+// SPDX-License-Identifier: Apache-2.0
+import JSON
+import JSONFilterRuntime
+
+/// Finite codec policy, independent of compilation and execution policies.
+/// Native work, traversal, text, cancellation and live logical storage limits also apply.
+public struct JSONTransformDefinitionLimits: Sendable {
+ /// Native cumulative quotas for one prepare, decode or encode operation.
+ public var json: JSONLimits = .balanced
+ /// Maximum UTF-8 bytes in query source; default one MiB.
+ public var maximumSourceBytes: UInt64 = 1_048_576
+ /// Maximum declared names, including every supplied occurrence before duplicate detection.
+ public var maximumVariableNames: UInt64 = 128
+ /// Maximum UTF-8 bytes in one ASCII variable name.
+ public var maximumNameBytes: UInt64 = 128
+ /// Maximum total UTF-8 bytes in all supplied variable names.
+ public var maximumTotalNameBytes: UInt64 = 8_192
+ /// Inclusive input/output wire-byte ceiling, additionally bounded by native quotas.
+ public var maximumEncodedBytes: UInt64 = 2_097_152
+ var checkpointObserver: (@Sendable (UInt64) -> Void)?
+ /// Creates the finite balanced codec policy; zero capacities remain valid.
+ public init() {}
+
+ func effective() throws -> JSONLimits {
+ for cap in [maximumSourceBytes, maximumVariableNames, maximumNameBytes,
+ maximumTotalNameBytes, maximumEncodedBytes] {
+ guard cap < UInt64.max else { throw JSONFailure(.configuration, code: .invalidLimits) }
+ }
+ var result = json
+ result[.inputBytes] = min(result[.inputBytes], maximumEncodedBytes)
+ result[.outputBytes] = min(result[.outputBytes], maximumEncodedBytes)
+ try result.validate()
+ return result
+ }
+ func meter() throws -> JSONWorkMeter {
+ var result = try JSONWorkMeter(limits: effective(), domain: .core)
+ result.checkpointObserver = checkpointObserver
+ return result
+ }
+}
+
+/// Closed-definition refusals; native JSON syntax/resource errors and cancellation stay distinct.
+public enum JSONTransformDefinitionFailure: Error, Sendable, Equatable {
+ /// Wrong root, fields, field types or format identifier.
+ case invalidDefinition
+ /// A definition revision other than mathematical integer one.
+ case unsupportedDefinitionRevision
+ /// A query dialect other than swift-json-query-v1.
+ case unsupportedDialectRevision
+ /// A name outside the compiler's ASCII identifier grammar.
+ case invalidVariableName
+ /// Two identical names were supplied; no Set conversion silently removed one.
+ case duplicateVariableName
+ /// An inclusive codec-specific capacity was exceeded.
+ /// - Parameters:
+ /// - metric: Fixed codec policy identifier, without rejected payloads.
+ /// - limit: Configured finite maximum.
+ /// - attempted: Observed usage at refusal.
+ case limitExceeded(metric: String, limit: UInt64, attempted: UInt64)
+}
+
+/// Immutable source definition, separate from compiled VM plans and effective policies.
+/// Source scalar spelling is preserved. Names are unique ASCII identifiers in UTF-8
+/// order. No synthesized String equality, source rewriting or bytecode serialization is used.
+public struct JSONTransformProgramDefinition: Sendable {
+ /// Exact decoded query source, including whitespace and scalar spelling.
+ public let source: String
+ /// Unique declared variable names in exact ascending UTF-8 order.
+ public let variableNames: [String]
+ /// Known wire definition revision; independent of implementation identity.
+ public var definitionRevision: UInt32 { 1 }
+ /// Known query-language revision; independent of limits and the CLI source digest.
+ public var dialectRevision: String { "swift-json-query-v1" }
+
+ private init(validatedSource: String, canonicalNames: [String]) {
+ source = validatedSource; variableNames = canonicalNames
+ }
+
+ /// Validates authored source metadata using one finite ledger; query compilation is explicit.
+ /// - Parameters:
+ /// - source: Exact query source; syntax is checked only by compile.
+ /// - variables: Names before sorting; exact duplicates are rejected.
+ /// - limits: Finite text, sort, traversal and retention policy.
+ /// - Returns: The immutable definition and actual successful validation accounting.
+ /// - Throws: Definition/name/capacity, native resource/configuration errors or cancellation.
+ public static func prepare(source: String, variables: [String] = [],
+ limits: JSONTransformDefinitionLimits = .init()) throws -> JSONOperationReport {
+ var meter = try limits.meter()
+ let result = try make(source: source, variables: variables, limits: limits, meter: &meter)
+ return .init(result: result, statistics: meter.statistics())
+ }
+
+ /// Decodes one strict native JSON document with an exact closed V1 field set.
+ /// Field order, whitespace and equivalent JSON escapes may differ; encoding
+ /// produces canonical bytes. Duplicate fields/names and unknown fields fail.
+ /// - Parameters:
+ /// - bytes: Caller-owned UTF-8 input; no ambient I/O or byte-copy prepass.
+ /// - limits: One cumulative policy for parsing, field checks and canonical names.
+ /// - Returns: The decoded definition and actual successful cumulative accounting.
+ /// - Throws: Native JSON syntax/resource errors, definition failures or cancellation.
+ public static func decode(_ bytes: [UInt8], limits: JSONTransformDefinitionLimits = .init()) throws -> JSONOperationReport {
+ var meter = try limits.meter()
+ var root = try JSONParser().parse(bytes, meter: &meter)
+ guard case .object(let object) = root, object.count == 5 else { throw JSONTransformDefinitionFailure.invalidDefinition }
+ try meter.retain(80)
+ var fields: [JSON?] = Array(repeating: nil, count: 5)
+ for member in object {
+ var found = false
+ for (index, key) in ["definition", "version", "dialect", "source", "variables"].enumerated() {
+ if try JSONComparison.textEquals(member.key.string, key, meter: &meter) {
+ fields[index] = member.value; found = true; break
+ }
+ }
+ guard found else { throw JSONTransformDefinitionFailure.invalidDefinition }
+ }
+ guard case .string(let tag) = fields[0],
+ try JSONComparison.textEquals(tag, "swift-json-transform-source", meter: &meter),
+ case .number(let revision) = fields[1], case .string(let dialect) = fields[2],
+ case .string(let source) = fields[3], case .array(let names) = fields[4] else {
+ throw JSONTransformDefinitionFailure.invalidDefinition
+ }
+ guard try revision.compare(to: JSONNumber(integer: 1), meter: &meter) == .orderedSame else {
+ throw JSONTransformDefinitionFailure.unsupportedDefinitionRevision
+ }
+ guard try JSONComparison.textEquals(dialect, "swift-json-query-v1", meter: &meter) else {
+ throw JSONTransformDefinitionFailure.unsupportedDialectRevision
+ }
+ try require(UInt64(names.count), limits.maximumVariableNames, "variableNames", meter: &meter)
+ var strings: [String] = []
+ for name in names {
+ try meter.work()
+ guard case .string(let text) = name else { throw JSONTransformDefinitionFailure.invalidDefinition }
+ try meter.retain(16); strings.append(text)
+ }
+ let baseline = meter.used(.retainedBytes)
+ let result = try make(source: source, variables: strings, limits: limits, meter: &meter)
+ let resultBytes = meter.used(.retainedBytes) - baseline
+ root = .null; fields = []; strings = []
+ // Parsed tree and extraction aliases are temporary; result strings were
+ // independently reserved by make. Work is never refunded or restarted.
+ try meter.checkpoint(); try meter.release(meter.used(.retainedBytes) - resultBytes)
+ return .init(result: result, statistics: meter.statistics())
+ }
+
+ /// Encodes canonical compact UTF-8 with fixed metadata order and native escaping.
+ /// The ordered fields are definition, version, dialect, source, variables.
+ /// - Parameter limits: One finite ledger for admission, metadata, sorting and output.
+ /// - Returns: Canonical bytes without a trailing newline and successful accounting.
+ /// - Throws: Codec/native capacity or configuration errors, or cancellation.
+ public func encode(limits: JSONTransformDefinitionLimits = .init()) throws -> JSONOperationReport<[UInt8]> {
+ var meter = try limits.meter()
+ let admitted = try Self.make(source: source, variables: variableNames, limits: limits, meter: &meter)
+ var names: [JSON] = []
+ for name in admitted.variableNames { try meter.work(); try meter.retain(16); names.append(.string(name)) }
+ try meter.retain(5 * JSONLogicalWeights.member)
+ let members: [JSONMember] = [
+ .init("definition", value: .string("swift-json-transform-source")),
+ .init("version", value: .number(JSONNumber(integer: 1))),
+ .init("dialect", value: .string(dialectRevision)),
+ .init("source", value: .string(admitted.source)),
+ .init("variables", value: .array(names))]
+ let object = try JSONObject.checked(members: members, meter: &meter)
+ let bytes = try JSONWriter().write(.object(object), meter: &meter)
+ try meter.checkpoint(); try meter.release(meter.used(.retainedBytes) - UInt64(bytes.count))
+ return .init(result: bytes, statistics: meter.statistics())
+ }
+
+ /// Compiles this definition using the existing compiler and one compilation context.
+ /// The source definition and codec policy are not captured by the compiled plan.
+ /// - Parameter limits: Explicit finite compilation policy, independent of codec policy.
+ /// - Returns: The native compiled program and actual successful compilation work.
+ /// - Throws: The existing typed compiler/configuration/resource/cancellation failures.
+ public func compile(limits: JSONTransformLimits = .init()) throws -> JSONOperationReport {
+ do {
+ var context = try TransformRuntimeContext(limits: limits.runtime)
+ var names: Set = []
+ for name in variableNames {
+ // Admit hashing/scan work before constructing an owned Set entry.
+ for _ in name.utf8 { try context.work() }
+ try context.meter.retain(32)
+ names.insert(name)
+ try context.checkpoint()
+ }
+ let result = try JSONTransformProgram.compileValidated(source, variables: names, context: &context)
+ return .init(result: result, statistics: context.meter.statistics())
+ } catch { throw JSONTransformFailure.classify(error) }
+ }
+
+ private static func require(_ attempted: UInt64, _ limit: UInt64, _ metric: String,
+ meter: inout JSONWorkMeter) throws {
+ try meter.checkpoint()
+ guard attempted <= limit else {
+ throw JSONTransformDefinitionFailure.limitExceeded(metric: metric, limit: limit, attempted: attempted)
+ }
+ }
+ private static func scanned(_ text: String, maximum: UInt64, metric: String,
+ meter: inout JSONWorkMeter) throws -> UInt64 {
+ var count: UInt64 = 0
+ for _ in text.utf8 {
+ try meter.work(); count = try meter.adding(.aggregateTextBytes, count, 1)
+ try require(count, maximum, metric, meter: &meter)
+ }
+ return count
+ }
+ private static func make(source: String, variables: [String], limits: JSONTransformDefinitionLimits,
+ meter: inout JSONWorkMeter) throws -> Self {
+ try meter.retain(64)
+ _ = try scanned(source, maximum: limits.maximumSourceBytes, metric: "sourceBytes", meter: &meter)
+ try JSONAdmission.text(source, meter: &meter, owned: true)
+ try require(UInt64(variables.count), limits.maximumVariableNames, "variableNames", meter: &meter)
+ var names: [String] = [], total: UInt64 = 0
+ try meter.retain(32)
+ for name in variables {
+ let size = try scanned(name, maximum: limits.maximumNameBytes, metric: "nameBytes", meter: &meter)
+ total = try meter.adding(.aggregateTextBytes, total, size)
+ try require(total, limits.maximumTotalNameBytes, "totalNameBytes", meter: &meter)
+ var first = true
+ for byte in name.utf8 {
+ try meter.work()
+ let letter = byte == 95 || (65...90).contains(byte) || (97...122).contains(byte)
+ guard letter || (!first && (48...57).contains(byte)) else { throw JSONTransformDefinitionFailure.invalidVariableName }
+ first = false
+ }
+ guard !first else { throw JSONTransformDefinitionFailure.invalidVariableName }
+ try JSONAdmission.text(name, key: true, meter: &meter, owned: true)
+ try meter.retain(16); names.append(name)
+ }
+ var scratch: [String] = []
+ for name in names { try meter.work(); try meter.retain(16); scratch.append(name) }
+ var width = 1
+ while width < names.count {
+ var start = 0
+ while start < names.count {
+ let middle = start + min(width, names.count - start)
+ let end = middle + min(width, names.count - middle)
+ var a = start, b = middle
+ for target in start.. names.count / 2 { break }; width *= 2
+ }
+ if names.count > 1 {
+ for index in 1.. Bool {
+ var a = lhs.utf8.makeIterator(), b = rhs.utf8.makeIterator()
+ while true {
+ try meter.work(); let x = a.next(), y = b.next()
+ if x != y { if let x, let y { return x < y }; return x == nil }
+ if x == nil { return false }
+ }
+ }
+}
diff --git a/Tests/JSONTransformTests/DefinitionTests.swift b/Tests/JSONTransformTests/DefinitionTests.swift
new file mode 100644
index 0000000..8d19dc6
--- /dev/null
+++ b/Tests/JSONTransformTests/DefinitionTests.swift
@@ -0,0 +1,205 @@
+// SPDX-License-Identifier: Apache-2.0
+import Testing
+import Foundation
+import Synchronization
+import JSON
+@testable import JSONTransform
+
+private enum DefinitionTestError: Error { case expectedFailure }
+private func definitionFailure(_ body: () throws -> Void) throws -> JSONTransformDefinitionFailure {
+ do { try body(); Issue.record("Expected definition failure"); throw DefinitionTestError.expectedFailure }
+ catch let error as JSONTransformDefinitionFailure { return error }
+}
+private func definitionResource(_ body: () throws -> Void) throws -> JSONFailure {
+ do { try body(); Issue.record("Expected native failure"); throw DefinitionTestError.expectedFailure }
+ catch let error as JSONFailure { #expect(error.category == .resource); return error }
+}
+private let dotDefinition = #"{"definition":"swift-json-transform-source","version":1,"dialect":"swift-json-query-v1","source":".","variables":[]}"#
+
+@Test func bindingD01CanonicalGoldenBytesAndReload() throws {
+ let definition = try JSONTransformProgramDefinition.prepare(source: "$z + $a", variables: ["z", "a"])
+ let expected = #"{"definition":"swift-json-transform-source","version":1,"dialect":"swift-json-query-v1","source":"$z + $a","variables":["a","z"]}"#
+ let bytes = try definition.result.encode()
+ #expect(bytes.result == Array(expected.utf8))
+ let decoded = try JSONTransformProgramDefinition.decode(bytes.result)
+ #expect(decoded.result.variableNames == ["a", "z"])
+ #expect(try decoded.result.encode().result == bytes.result)
+ #expect(decoded.result.definitionRevision == 1 && decoded.result.dialectRevision == "swift-json-query-v1")
+ #expect(decoded.statistics[.workUnits] > definition.statistics[.workUnits])
+}
+
+@Test func bindingD02CanonicalEscapingAndExactScalarSource() throws {
+ let source = "é e\u{301}\n\t\"\\/"
+ let expected = #"{"definition":"swift-json-transform-source","version":1,"dialect":"swift-json-query-v1","source":"é é\u000a\u0009\"\\/","variables":[]}"#
+ let a = try JSONTransformProgramDefinition.prepare(source: source).result
+ #expect(try a.encode().result == Array(expected.utf8))
+ let reload = try JSONTransformProgramDefinition.decode(Array(expected.utf8)).result
+ #expect(Array(reload.source.utf8) == Array(source.utf8))
+ let composed = try JSONTransformProgramDefinition.prepare(source: "é").result
+ let decomposed = try JSONTransformProgramDefinition.prepare(source: "e\u{301}").result
+ #expect(try composed.encode().result != decomposed.encode().result)
+}
+
+@Test func bindingD03StrictSemanticDecodeCanonicalizesPresentation() throws {
+ let input = #" { "variables":[], "source":"\u002e", "dialect":"swift-json-query-v1", "version":1.0, "definition":"swift-json-transform-source" } "#
+ let value = try JSONTransformProgramDefinition.decode(Array(input.utf8)).result
+ #expect(try value.encode().result == Array(dotDefinition.utf8))
+ let reordered = #"{"definition":"swift-json-transform-source","version":1,"dialect":"swift-json-query-v1","source":".","variables":["z","_a","B"]}"#
+ #expect(try JSONTransformProgramDefinition.decode(Array(reordered.utf8)).result.variableNames == ["B", "_a", "z"])
+}
+
+@Test func bindingD04ClosedFieldsVersionsAndKindsArePrecise() throws {
+ for input in ["null", "[]", "{}", dotDefinition.replacingOccurrences(of: "\"version\":1", with: "\"version\":true"),
+ dotDefinition.replacingOccurrences(of: "\"source\":\".\"", with: "\"unknown\":\".\""),
+ dotDefinition.replacingOccurrences(of: "[]}", with: "[],\"extra\":null}"),
+ dotDefinition.replacingOccurrences(of: "[]}", with: "[42]}"),
+ dotDefinition.replacingOccurrences(of: "swift-json-transform-source", with: "other")] {
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.decode(Array(input.utf8)) } == .invalidDefinition)
+ }
+ let version = dotDefinition.replacingOccurrences(of: "\"version\":1", with: "\"version\":2")
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.decode(Array(version.utf8)) } == .unsupportedDefinitionRevision)
+ let dialect = dotDefinition.replacingOccurrences(of: "query-v1", with: "query-v2")
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.decode(Array(dialect.utf8)) } == .unsupportedDialectRevision)
+}
+
+@Test func bindingD05NativeGrammarAndDuplicateFieldsStayAuthoritative() throws {
+ let duplicate = dotDefinition.replacingOccurrences(of: "\"version\":1", with: "\"version\":1,\"ver\\u0073ion\":1")
+ do { _ = try JSONTransformProgramDefinition.decode(Array(duplicate.utf8)); Issue.record("Accepted duplicate key") }
+ catch let error as JSONFailure { #expect(error.category == .syntax && error.code == .duplicateKey) }
+ for bytes in [Array((dotDefinition + "false").utf8), [0xef, 0xbb, 0xbf] + Array(dotDefinition.utf8), [0xff]] {
+ do { _ = try JSONTransformProgramDefinition.decode(bytes); Issue.record("Accepted malformed JSON") }
+ catch let error as JSONFailure { #expect(error.category == .syntax) }
+ }
+}
+
+@Test func bindingD06NamesRejectBeforeSetConversion() throws {
+ for name in ["", "$x", "é", "K", "a-b", "1a", "a\u{0}"] {
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: [name]) } == .invalidVariableName)
+ }
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["b","a","b"]) } == .duplicateVariableName)
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["K","K"]) } == .invalidVariableName)
+ let bytes = Array(dotDefinition.replacingOccurrences(of: "[]}", with: "[\"a\",\"a\"]}").utf8)
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.decode(bytes) } == .duplicateVariableName)
+}
+
+@Test func bindingD07SpecificCapsAreInclusiveAndFinite() throws {
+ var limits = JSONTransformDefinitionLimits(); limits.maximumSourceBytes = 1
+ _ = try JSONTransformProgramDefinition.prepare(source: ".", limits: limits)
+ limits.maximumSourceBytes = 0
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.prepare(source: ".", limits: limits) } == .limitExceeded(metric: "sourceBytes", limit: 0, attempted: 1))
+ _ = try JSONTransformProgramDefinition.prepare(source: "", limits: limits)
+ limits = .init(); limits.maximumVariableNames = 2
+ _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["a","b"], limits: limits)
+ limits.maximumVariableNames = 1
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["a","b"], limits: limits) } == .limitExceeded(metric: "variableNames", limit: 1, attempted: 2))
+ limits = .init(); limits.maximumNameBytes = 1
+ _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["a"], limits: limits)
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["ab"], limits: limits) } == .limitExceeded(metric: "nameBytes", limit: 1, attempted: 2))
+ limits = .init(); limits.maximumTotalNameBytes = 2
+ _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["a","b"], limits: limits)
+ limits.maximumTotalNameBytes = 1
+ #expect(try definitionFailure { _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["a","b"], limits: limits) } == .limitExceeded(metric: "totalNameBytes", limit: 1, attempted: 2))
+ let keys: [WritableKeyPath] = [\.maximumSourceBytes, \.maximumVariableNames, \.maximumNameBytes, \.maximumTotalNameBytes, \.maximumEncodedBytes]
+ for key in keys {
+ limits = .init(); limits[keyPath: key] = .max
+ do { _ = try JSONTransformProgramDefinition.prepare(source: ".", limits: limits); Issue.record("Accepted infinite cap") }
+ catch let error as JSONFailure { #expect(error.category == .configuration && error.code == .invalidLimits) }
+ }
+}
+
+@Test func bindingD08WireAndOwnedWorkRetentionHaveExactBoundaries() throws {
+ let definition = try JSONTransformProgramDefinition.prepare(source: ".").result
+ let encoded = try definition.encode(); var limits = JSONTransformDefinitionLimits()
+ limits.maximumEncodedBytes = UInt64(encoded.result.count)
+ #expect(try definition.encode(limits: limits).result == encoded.result)
+ #expect(try JSONTransformProgramDefinition.decode(encoded.result, limits: limits).result.source == ".")
+ limits.maximumEncodedBytes -= 1
+ #expect(try definitionResource { _ = try definition.encode(limits: limits) }.resource?.metric == .outputBytes)
+ #expect(try definitionResource { _ = try JSONTransformProgramDefinition.decode(encoded.result, limits: limits) }.resource?.metric == .inputBytes)
+ for operation in 0..<3 {
+ func run(_ policy: JSONTransformDefinitionLimits) throws -> JSONOperationStatistics {
+ if operation == 0 { return try JSONTransformProgramDefinition.prepare(source: "$b", variables: ["b","a"], limits: policy).statistics }
+ if operation == 1 { return try JSONTransformProgramDefinition.decode(encoded.result, limits: policy).statistics }
+ return try definition.encode(limits: policy).statistics
+ }
+ let measured = try run(.init())
+ for metric in [JSONResourceMetric.workUnits, .retainedBytes] {
+ var p = JSONTransformDefinitionLimits()
+ let boundary = metric == .workUnits ? measured[metric] : measured.peakLogicalRetainedBytes
+ #expect(boundary > 0); p.json[metric] = boundary; _ = try run(p)
+ p.json[metric] = boundary - 1
+ #expect(try definitionResource { _ = try run(p) }.resource?.metric == metric)
+ }
+ }
+ #expect(encoded.statistics[.retainedBytes] == UInt64(encoded.result.count))
+}
+
+@Test func bindingD09CompileReportsPreserveExistingPlanBehaviorAndBudget() throws {
+ let report = try JSONTransformProgram.compileWithReport("[$n, .]", variables: ["n"])
+ #expect(report.statistics.domain == .core && report.statistics[.workUnits] > 0)
+ var limits = JSONTransformLimits(); limits[.workUnits] = report.statistics[.workUnits]
+ let old = try JSONTransformProgram.compile("[$n, .]", variables: ["n"], limits: limits)
+ for plan in [report.result, old] {
+ let result = try plan.withExecution(input: 1, bindings: ["n": 42]) { try $0.next()?.value }
+ #expect(result.result == .array([42,1]))
+ }
+ limits[.workUnits] -= 1
+ do { _ = try JSONTransformProgram.compile("[$n, .]", variables: ["n"], limits: limits); Issue.record("Compile reset work") }
+ catch let error as JSONTransformFailure { #expect(error.category == .resource && error.metric == "workUnits") }
+ let definition = try JSONTransformProgramDefinition.prepare(source: "[$n, .]", variables: ["n"]).result
+ #expect(try definition.compile().statistics[.workUnits] > report.statistics[.workUnits])
+}
+
+@Test func bindingD10CodecDoesNotInventQueryValidationOrDecimalAuthority() throws {
+ let bad = try JSONTransformProgramDefinition.prepare(source: "(").result
+ _ = try JSONTransformProgramDefinition.decode(bad.encode().result)
+ do { _ = try bad.compile(); Issue.record("Accepted invalid query") }
+ catch let error as JSONTransformFailure { #expect(error.category == .compilation) }
+ let source = "[9007199254740993,0.1,\"é\"]"
+ let definition = try JSONTransformProgramDefinition.prepare(source: source).result
+ let reload = try JSONTransformProgramDefinition.decode(definition.encode().result).result
+ let plan = try reload.compile().result
+ let result = try plan.withExecution(input: .null) { try $0.next()?.value }
+ let expected: JSON = .array([.number(try JSONNumber(parsing: "9007199254740993")),
+ .number(try JSONNumber(parsing: "0.1")),.string("e\u{301}")])
+ #expect(result.result == expected)
+}
+
+@Test func bindingD11CancellationOccursInsideOwnedCodecWork() async throws {
+ let handle = Mutex?>(nil)
+ let ready = Mutex?>(nil)
+ let observed = Mutex(0)
+ let task = Task {
+ await withCheckedContinuation { continuation in ready.withLock { $0 = continuation } }
+ var limits = JSONTransformDefinitionLimits()
+ limits.checkpointObserver = { work in
+ if work >= 32 { observed.withLock { $0 = work }; handle.withLock { $0?.cancel() } }
+ }
+ _ = try JSONTransformProgramDefinition.prepare(source: String(repeating: "x", count: 4096), limits: limits)
+ }
+ handle.withLock { $0 = task }
+ var continuation: CheckedContinuation?
+ while continuation == nil { continuation = ready.withLock { $0 }; if continuation == nil { await Task.yield() } }
+ continuation?.resume()
+ do { try await task.value; Issue.record("Owned codec ignored cancellation") } catch is CancellationError {}
+ #expect(observed.withLock { $0 } >= 32)
+}
+
+@Test func bindingSharedDefinitionAndCompilationHaveIndependentHealth() async throws {
+ let definition = try JSONTransformProgramDefinition.prepare(source: ".").result
+ let results = try await withThrowingTaskGroup(of: Bool.self) { group in
+ for index in 0..<6 { group.addTask {
+ if index == 0 {
+ var limits = JSONTransformLimits(); limits[.workUnits] = 0
+ do { _ = try definition.compile(limits: limits); return false }
+ catch let error as JSONTransformFailure { return error.category == .resource }
+ }
+ let bytes = try definition.encode().result
+ let reload = try JSONTransformProgramDefinition.decode(bytes).result
+ let plan = try reload.compile().result
+ return try plan.withExecution(input: 42) { try $0.next()?.value }.result == 42
+ } }
+ var values: [Bool] = []; for try await value in group { values.append(value) }; return values
+ }
+ #expect(results.count == 6 && results.allSatisfy { $0 })
+}
diff --git a/Tests/JSONTransformTests/IndexedSelectionSpike.swift b/Tests/JSONTransformTests/IndexedSelectionSpike.swift
new file mode 100644
index 0000000..6aee3fe
--- /dev/null
+++ b/Tests/JSONTransformTests/IndexedSelectionSpike.swift
@@ -0,0 +1,276 @@
+// SPDX-License-Identifier: Apache-2.0
+// Test-only ownership experiment. This is not a shipped selection API.
+import Testing
+import Synchronization
+import JSON
+import JSONFilterRuntime
+@testable import JSONTransform
+
+private enum SelectionSpikeFailure: Error, Equatable {
+ case invalidPosition, incomplete, closed, provider, invalidBindings
+}
+private final class PopulationOwner {
+ let population: FilterYield
+ let metadata: TransformValueLease
+ init(_ population: FilterYield, metadata: TransformValueLease) {
+ self.population = population; self.metadata = metadata
+ }
+}
+private struct SpikePosition {
+ let ordinal: UInt64
+ let owner: PopulationOwner
+ let reservation: TransformValueLease
+}
+private struct SpikeInput: FilterInput {
+ var fail = false
+ var calls = 0
+ mutating func next(context: inout TransformRuntimeContext) throws -> FilterYield? {
+ calls += 1
+ if fail { throw SelectionSpikeFailure.provider }
+ return nil
+ }
+}
+private final class SelectionSpike {
+ private var context: TransformRuntimeContext
+ private var vm: FilterVM?
+ private var owner: PopulationOwner?
+ private var input = SpikeInput()
+ private var failure: (any Error)?
+ private var ended = false, closed = false
+ private var planBytes: UInt64 = 0
+ private init(_ context: consuming TransformRuntimeContext) { self.context = consume context }
+
+ static func make(_ plan: JSONTransformProgram, population: [JSON],
+ limits: JSONTransformLimits = .init(), providerFailure: Bool = false,
+ observer: (@Sendable (UInt64) -> Void)? = nil,
+ projectionObserver: (@Sendable (Int) -> Void)? = nil,
+ cleanupObserver: (@Sendable (UInt64) -> Void)? = nil) throws -> SelectionSpike {
+ var context = try TransformRuntimeContext(limits: limits.runtime)
+ try context.meter.retain(128)
+ context.meter.checkpointObserver = observer
+ let result = SelectionSpike(consume context)
+ do {
+ guard plan.variables == ["values"] else { throw SelectionSpikeFailure.invalidBindings }
+ try result.prepare(plan, population: population, providerFailure: providerFailure, projectionObserver: projectionObserver)
+ return result
+ } catch {
+ let cause = error
+ do { try result.close() } catch { Issue.record("Prototype preparation cleanup failed: \(error)") }
+ if let cleanupObserver {
+ do { cleanupObserver(try result.settlement()[.retainedBytes]) }
+ catch { Issue.record("Prototype terminal observation failed: \(error)") }
+ }
+ throw cause
+ }
+ }
+ private func prepare(_ plan: JSONTransformProgram, population: [JSON], providerFailure: Bool,
+ projectionObserver: (@Sendable (Int) -> Void)?) throws {
+ try context.meter.retain(plan.planReservation); planBytes = plan.planReservation
+ // A prototype population cap, independent of lexical token counts.
+ guard population.count <= 1024 else { throw SelectionSpikeFailure.invalidBindings }
+ try context.charge(.bindings)
+ let value = try FilterValues.produced(.array(population), context: &context)
+ try context.meter.retain(48)
+ let metadata = try TransformValueLease(ownedRetained: 48, context: &context)
+ owner = PopulationOwner(value, metadata: metadata)
+ var indices: [JSON] = [], scratch: UInt64 = 0
+ defer {
+ do { try context.meter.release(scratch) }
+ catch { Issue.record("Prototype projection scratch cleanup failed: \(error)") }
+ }
+ for ordinal in population.indices {
+ try context.work(); try context.meter.retain(64); scratch += 64
+ indices.append(.number(JSONNumber(integer: ordinal)))
+ projectionObserver?(indices.count)
+ }
+ let root = try FilterValues.array(indices, context: &context)
+ indices = []; try context.meter.release(scratch); scratch = 0
+ try context.charge(.totalRecords)
+ vm = try FilterVM(program: plan.program, input: root, bindings: ["values": value], context: &context)
+ input.fail = providerFailure
+ }
+ func next() throws -> SpikePosition? {
+ do {
+ try context.reclaimValueLeases()
+ if let failure { throw failure }
+ try context.checkpoint()
+ guard !closed else { throw SelectionSpikeFailure.closed }
+ guard var active = vm else { throw SelectionSpikeFailure.closed }
+ vm = nil; defer { vm = active }
+ guard let raw = try active.next(input: &input, context: &context) else { ended = true; return nil }
+ let before = context.meter.used(.retainedBytes)
+ do {
+ try context.meter.resetValueBounds()
+ try JSONAdmission.value(raw.value, meter: &context.meter)
+ } catch {
+ try context.meter.release(context.meter.used(.retainedBytes)-before); throw error
+ }
+ try context.charge(.totalResults)
+ guard case .number(let number) = raw.value else { throw SelectionSpikeFailure.invalidPosition }
+ let ordinal: UInt64
+ do { ordinal = try number.integer(as: UInt64.self, meter: &context.meter) }
+ catch let error as JSONFailure where error.category == .conversion { throw SelectionSpikeFailure.invalidPosition }
+ guard let native = Int(exactly: ordinal), let owner,
+ case .array(let values) = owner.population.value, native < values.count else {
+ throw SelectionSpikeFailure.invalidPosition
+ }
+ try context.meter.retain(32)
+ let slot = try TransformValueLease(ownedRetained: 32, context: &context)
+ return .init(ordinal: ordinal, owner: owner, reservation: slot)
+ } catch {
+ if failure == nil { failure = error; context.latch(error) }
+ throw failure ?? error
+ }
+ }
+ func accepts(_ position: SpikePosition) -> Bool { owner === position.owner }
+ func finish() throws -> JSONOperationStatistics {
+ if failure == nil && !ended { failure = SelectionSpikeFailure.incomplete; context.latch(SelectionSpikeFailure.incomplete) }
+ let cause = failure
+ try close()
+ if let cause { throw cause }
+ return try settlement()
+ }
+ func settlement() throws -> JSONOperationStatistics {
+ try context.reclaimValueLeases()
+ return context.meter.statistics()
+ }
+ var providerCalls: Int { input.calls }
+ private func close() throws {
+ guard !closed else { return }; closed = true
+ if var active = vm { vm = nil; try active.close(context: &context) }
+ owner = nil
+ try context.reclaimValueLeases()
+ try context.meter.release(planBytes + 128); planBytes = 0
+ try context.release(.bindings, context.used(.bindings))
+ }
+}
+private func spikePlan(_ source: String) throws -> JSONTransformProgram {
+ try JSONTransformProgram.compile(source, variables: ["values"])
+}
+
+@Test func bindingP01ExplicitIndicesPreserveDuplicatesOrderAndTies() throws {
+ let row = try JSON.parse(#"{"score":1}"#)
+ for (source, expected) in [(".[] | select(. == 1)", [UInt64(1)]),
+ ("1,0,1", [1,0,1]),
+ ("sort_by([$values[.].score, .])[]", [0,1])] {
+ let probe = try SelectionSpike.make(spikePlan(source), population: [row,row])
+ var ordinals: [UInt64] = []
+ while let position = try probe.next() { ordinals.append(position.ordinal) }
+ let report = try probe.finish()
+ #expect(ordinals == expected && report[.workUnits] > 0)
+ }
+ let empty = try SelectionSpike.make(spikePlan("empty"), population: [])
+ #expect(try empty.next() == nil); #expect(try empty.finish()[.workUnits] > 0)
+}
+
+@Test func bindingP02InvalidOutputAndAbandonedPrefixNeverCertify() throws {
+ for bad in ["null", "true", "\"0\"", "-1", "0.1", "18446744073709551616", "9223372036854775808", "1"] {
+ let probe = try SelectionSpike.make(spikePlan("try (0, " + bad + ") catch 0"), population: [42])
+ #expect(try probe.next()?.ordinal == 0)
+ for _ in 0..<2 {
+ do { _ = try probe.next(); Issue.record("Invalid position escaped") }
+ catch let error as SelectionSpikeFailure { #expect(error == .invalidPosition) }
+ }
+ do { _ = try probe.finish(); Issue.record("Caught refusal certified selection") }
+ catch let error as SelectionSpikeFailure { #expect(error == .invalidPosition) }
+ }
+ let prefix = try SelectionSpike.make(spikePlan("0,0"), population: [42])
+ #expect(try prefix.next()?.ordinal == 0)
+ do { _ = try prefix.finish(); Issue.record("Abandoned prefix certified") }
+ catch let error as SelectionSpikeFailure { #expect(error == .incomplete) }
+}
+
+@Test func bindingP03PopulationLeaseSurvivesCloseAndForeignOwnerRefuses() throws {
+ let plan = try spikePlan("0")
+ let source: [JSON] = [.string(String(repeating: "x", count: 8192))]
+ let a = try SelectionSpike.make(plan, population: source), b = try SelectionSpike.make(plan, population: source)
+ var position = try a.next(), alias = position
+ #expect(position?.ordinal == 0)
+ #expect(position.map { !b.accepts($0) } == true)
+ #expect(try a.next() == nil)
+ _ = try a.finish()
+ let retainedBefore = try withExtendedLifetime(alias) { try a.settlement()[.retainedBytes] }
+ #expect(retainedBefore > 8192)
+ position = nil
+ let retainedAfter = try withExtendedLifetime(alias) { try a.settlement()[.retainedBytes] }
+ #expect(retainedAfter > 8192)
+ alias = nil
+ #expect(try a.settlement()[.retainedBytes] == 0)
+ do { _ = try a.next(); Issue.record("Closed handle resumed") }
+ catch let error as SelectionSpikeFailure { #expect(error == .closed) }
+ #expect(try b.next()?.ordinal == 0); #expect(try b.next() == nil); _ = try b.finish()
+ for source in ["0.0", "-0"] {
+ let probe = try SelectionSpike.make(spikePlan(source), population: [42])
+ #expect(try probe.next()?.ordinal == 0); #expect(try probe.next() == nil); _ = try probe.finish()
+ }
+}
+
+@Test func bindingP04ResourceAndProviderFailuresStayTerminal() throws {
+ var limits = JSONTransformLimits(); limits[.results] = 1
+ let probe = try SelectionSpike.make(spikePlan("0,0"), population: [42], limits: limits)
+ #expect(try probe.next()?.ordinal == 0)
+ for _ in 0..<2 {
+ do { _ = try probe.next(); Issue.record("Results cap reset") }
+ catch { let e = JSONTransformFailure.classify(error); #expect(e.category == .resource && e.metric == "results") }
+ }
+ do { _ = try probe.finish(); Issue.record("Failed quota certified") }
+ catch { #expect(JSONTransformFailure.classify(error).category == .resource) }
+ let provider = try SelectionSpike.make(spikePlan("try input catch 0"), population: [42], providerFailure: true)
+ for _ in 0..<2 {
+ do { _ = try provider.next(); Issue.record("Provider error was swallowed") }
+ catch let error as SelectionSpikeFailure { #expect(error == .provider) }
+ }
+ #expect(provider.providerCalls == 1)
+ do { _ = try provider.finish(); Issue.record("Provider error certified") }
+ catch let error as SelectionSpikeFailure { #expect(error == .provider) }
+}
+
+@Test func bindingSharedSelectionSpikeReusesOnePlanIndependently() async throws {
+ let plan = try spikePlan(".[]")
+ let values = try await withThrowingTaskGroup(of: Bool.self) { group in
+ for _ in 0..<6 { group.addTask {
+ let probe = try SelectionSpike.make(plan, population: [42,42])
+ let a = try probe.next()?.ordinal, b = try probe.next()?.ordinal
+ guard try probe.next() == nil else { return false }
+ return try probe.finish()[.workUnits] > 0 && a == 0 && b == 1
+ } }
+ var result: [Bool] = []; for try await x in group { result.append(x) }; return result
+ }
+ #expect(values.count == 6 && values.allSatisfy { $0 })
+}
+
+@Test func bindingP05ProjectionEvaluationAndCancellationShareOneLedger() async throws {
+ let plan = try spikePlan("0"), population: [JSON] = Array(repeating: .string("row"), count: 64)
+ func run(_ limits: JSONTransformLimits) throws -> JSONOperationStatistics {
+ let probe = try SelectionSpike.make(plan, population: population, limits: limits)
+ do {
+ while let position = try probe.next() { #expect(position.ordinal == 0) }
+ return try probe.finish()
+ } catch {
+ let first = error
+ do { _ = try probe.finish() } catch { #expect(JSONTransformFailure.classify(error) == JSONTransformFailure.classify(first)) }
+ throw first
+ }
+ }
+ let measured = try run(.init())
+ var limits = JSONTransformLimits(); limits[.workUnits] = measured[.workUnits]
+ #expect(try run(limits)[.workUnits] == measured[.workUnits])
+ limits[.workUnits] -= 1
+ do { _ = try run(limits); Issue.record("Projection/evaluation restarted work") }
+ catch { #expect(JSONTransformFailure.classify(error).category == .resource) }
+ let handle = Mutex?>(nil), ready = Mutex?>(nil)
+ let observed = Mutex(0), finalRetained = Mutex(nil)
+ let task = Task {
+ await withCheckedContinuation { continuation in ready.withLock { $0 = continuation } }
+ _ = try SelectionSpike.make(plan, population: population, projectionObserver: { count in
+ if count == 4 { observed.withLock { $0 = count }; handle.withLock { $0?.cancel() } }
+ }, cleanupObserver: { value in finalRetained.withLock { $0 = value } })
+ }
+ handle.withLock { $0 = task }
+ var continuation: CheckedContinuation?
+ while continuation == nil { continuation = ready.withLock { $0 }; if continuation == nil { await Task.yield() } }
+ continuation?.resume()
+ do { try await task.value; Issue.record("Projection ignored cancellation") } catch is CancellationError {}
+ #expect(observed.withLock { $0 } == 4)
+ #expect(finalRetained.withLock { $0 } == 0)
+}
diff --git a/Tools/TransformBindingQualification/Package.swift b/Tools/TransformBindingQualification/Package.swift
new file mode 100644
index 0000000..91d5f5b
--- /dev/null
+++ b/Tools/TransformBindingQualification/Package.swift
@@ -0,0 +1,8 @@
+// swift-tools-version: 6.4
+// SPDX-License-Identifier: Apache-2.0
+import PackageDescription
+let package = Package(name: "TransformBindingQualification", platforms: [.macOS("27.0"), .iOS("27.0")],
+ dependencies: [.package(name: "Swift-JSON", path: "../..")],
+ targets: [.executableTarget(name: "TransformBindingConsumer", dependencies: [
+ .product(name: "JSON", package: "Swift-JSON"), .product(name: "JSONTransform", package: "Swift-JSON")
+ ])], swiftLanguageModes: [.v6])
diff --git a/Tools/TransformBindingQualification/README.md b/Tools/TransformBindingQualification/README.md
new file mode 100644
index 0000000..aa8fb40
--- /dev/null
+++ b/Tools/TransformBindingQualification/README.md
@@ -0,0 +1,8 @@
+# Source definition public consumer
+
+Run `swift run --package-path Tools/TransformBindingQualification TransformBindingConsumer`.
+Only JSON and JSONTransform products are imported. Six named witnesses cover the
+compiled DocC example, independent canonical bytes, canonical names, duplicate-name
+rejection, operational compilation failure and null/value/EOF execution reports.
+This is path-dependent preparation, not version-tag resolution. No public indexed
+selection API is claimed; its Sprint1 experiment stays in the package test target.
diff --git a/Tools/TransformBindingQualification/Sources/TransformBindingConsumer/Main.swift b/Tools/TransformBindingQualification/Sources/TransformBindingConsumer/Main.swift
new file mode 100644
index 0000000..f0ff9f4
--- /dev/null
+++ b/Tools/TransformBindingQualification/Sources/TransformBindingConsumer/Main.swift
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: Apache-2.0
+import JSON
+import JSONTransform
+
+func guideExample() throws {
+ let definition = try JSONTransformProgramDefinition.prepare(
+ source: "$amount", variables: ["amount"]).result
+ let bytes = try definition.encode().result
+ let restored = try JSONTransformProgramDefinition.decode(bytes).result
+ let compiled = try restored.compile()
+ let amount = JSON.number(try JSONNumber(parsing: "9007199254740993"))
+ let output = try compiled.result.withExecution(input: .null, bindings: ["amount": amount]) {
+ try $0.next()?.value
+ }
+ precondition(output.result == amount)
+ precondition(compiled.statistics[.workUnits] > 0)
+}
+@main enum Consumer {
+ static func main() throws {
+ try guideExample()
+ let golden = #"{"definition":"swift-json-transform-source","version":1,"dialect":"swift-json-query-v1","source":".","variables":[]}"#
+ let definition = try JSONTransformProgramDefinition.prepare(source: ".").result
+ let canonical = try definition.encode().result
+ precondition(canonical == Array(golden.utf8))
+ let names = try JSONTransformProgramDefinition.prepare(source: "$z", variables: ["z","a"]).result
+ precondition(names.variableNames == ["a","z"])
+ do {
+ _ = try JSONTransformProgramDefinition.prepare(source: ".", variables: ["a","a"])
+ preconditionFailure("Duplicate name accepted")
+ } catch let error as JSONTransformDefinitionFailure { precondition(error == .duplicateVariableName) }
+ var limits = JSONTransformLimits(); limits[.workUnits] = 0
+ do { _ = try definition.compile(limits: limits); preconditionFailure("Compile budget reset") }
+ catch let error as JSONTransformFailure { precondition(error.category == .resource) }
+ let report = try JSONTransformProgram.compileWithReport("null,42")
+ let output = try report.result.withExecution(input: .null) { execution in
+ let first = try execution.next(), second = try execution.next(), end = try execution.next()
+ precondition(first?.value == .null && second?.value == 42 && end == nil)
+ }
+ precondition(report.statistics[.workUnits] > 0 && output.statistics[.workUnits] > 0)
+ print("TRANSFORM_BINDINGS passed witnesses=6")
+ }
+}
From dfa0f9626be6dc2aba8456ddd7d11ee6f99a45c2 Mon Sep 17 00:00:00 2001
From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com>
Date: Thu, 29 Oct 2026 13:09:28 +0100
Subject: [PATCH 4/6] Add bounded public Transform selection scopes and EOF
reports
---
AGENTS.md | 6 +-
CHANGELOG.md | 8 +
Documentation/Architecture/API.md | 6 +
.../Release1.8/TransformBindings.md | 52 ++-
.../Qualification/TestDiscovery.json | 28 +-
.../Qualification/TransformSelection1.8.md | 104 +++++
README.md | 2 +
Scripts/ci.py | 14 +-
Scripts/qualify_transform_selection.py | 63 +++
Scripts/test_qualification_harness.py | 25 +-
Sources/JSONCLIEngine/CLIBuildIdentity.swift | 2 +-
.../Documentation.docc/IndexedSelection.md | 83 ++++
.../Documentation.docc/JSONTransform.md | 14 +-
.../Documentation.docc/SourceDefinitions.md | 4 +-
.../JSONTransformSelection.swift | 382 ++++++++++++++++++
Tests/JSONTransformTests/SelectionTests.swift | 338 ++++++++++++++++
.../Package.swift | 8 +
.../TransformSelectionQualification/README.md | 10 +
.../TransformSelectionConsumer/Main.swift | 103 +++++
19 files changed, 1229 insertions(+), 23 deletions(-)
create mode 100644 Documentation/Qualification/TransformSelection1.8.md
create mode 100644 Scripts/qualify_transform_selection.py
create mode 100644 Sources/JSONTransform/Documentation.docc/IndexedSelection.md
create mode 100644 Sources/JSONTransform/JSONTransformSelection.swift
create mode 100644 Tests/JSONTransformTests/SelectionTests.swift
create mode 100644 Tools/TransformSelectionQualification/Package.swift
create mode 100644 Tools/TransformSelectionQualification/README.md
create mode 100644 Tools/TransformSelectionQualification/Sources/TransformSelectionConsumer/Main.swift
diff --git a/AGENTS.md b/AGENTS.md
index 444b376..b414076 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -65,7 +65,9 @@ no allocation, speed, RSS or controlled-performance pass follows from consolidat
See Documentation/Qualification/Consolidation2026.md for combined gates and holds.
The commissioned 1.8 Transform binding train adds a source-definition codec and
-compile reports while preserving those original declarations. Indexed selection
-is a test-only private-kernel ownership experiment in Sprint 1, not public API.
+compile reports while preserving those original declarations. Sprint 1
+retains its historical test-only selection experiment. Sprint 2 adds public
+guarded selection scopes, owner-bound positions and EOF completion reports;
+preserve ordinary Transform semantics and require one cumulative invocation ledger.
See Documentation/Architecture/Release1.8/TransformBindings.md. No Content code,
lock, message, merge or publication is authorized by implementing this sprint.
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 630926e..988c5b1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,14 @@
## Unreleased — 1.8.0-dev
+- Add exact versioned Transform source definitions, a bounded native codec and
+ successful compilation reports through the existing compiler.
+- Add finite indexed selection scopes over explicit populations, owner-bound
+ positions and private-constructor EOF/accounting witnesses. Admission, projection,
+ VM evaluation, provider/output handling and completion share one ledger.
+ Preserve order/repeats, first failures and terminal lease accounting; ordinary
+ Transform/CLI semantics and all thirteen products remain unchanged.
+
- Extend public acceptance across schema validation, store/transform reads and
checked Patch rebuilds, with exact numeric/scalar identity and lifecycle cases.
- Add a frozen capacity/owned-counter matrix and compiled cross-product recipes;
diff --git a/Documentation/Architecture/API.md b/Documentation/Architecture/API.md
index 55b2bcd..885c757 100644
--- a/Documentation/Architecture/API.md
+++ b/Documentation/Architecture/API.md
@@ -7,6 +7,12 @@ No product or dependency is added. JSONInspection now includes
lookup, position value batches and finite complete BFS follow.
See [the exact behavior/ownership boundary](StructuralIntelligence/BoundedWorkflows1.8.md)
and [qualification](../Qualification/Release1.8.md).
+The optional JSONTransform candidate also adds immutable versioned source definitions,
+a bounded native codec and successful compilation reports. Finite public selection
+scopes produce owner-bound positions and private-constructor EOF completion reports
+through the same compiler/VM. See the [binding contract](Release1.8/TransformBindings.md)
+and [selection qualification](../Qualification/TransformSelection1.8.md). No released
+1.7 signature or ordinary execution/CLI semantics change.
Navigation public signatures stay unchanged; its new selection capability is
package-only and uses the original admitted scope. Earlier inventories below
remain historical; compiler graphs must establish the final additive surface.
diff --git a/Documentation/Architecture/Release1.8/TransformBindings.md b/Documentation/Architecture/Release1.8/TransformBindings.md
index 6b01420..d93a155 100644
--- a/Documentation/Architecture/Release1.8/TransformBindings.md
+++ b/Documentation/Architecture/Release1.8/TransformBindings.md
@@ -1,4 +1,4 @@
-# Transform bindings — commissioned 1.8 candidate, Sprint 1
+# Transform bindings — commissioned 1.8 candidate
The maintainer commissioned the source-definition codec, indexed selection and
compile reporting on 29 October 2026. Base is b8ddeb26042005a06360f50fcf20b1f840e66d1c.
@@ -6,6 +6,54 @@ This additive train preserves the existing 42 authored Transform declarations,
ordinary execution/CLI semantics, thirteen products and empty root dependencies.
Implementation and a successful spike are not release or downstream admission.
+
+## Sprint 2 current selection contract
+
+The maintainer commissioned the public selection increment on 29 October 2026.
+Its exact base is Sprint 1 commit 70df9ba8d47322abf0d4aec1439256d7a5e97e94.
+Historical Sprint 1 scope/evidence below remain source-bound. This increment
+ships the selection facade in JSONTransform without a new product/dependency or
+changes to the private compiler/VM, ordinary execution or CLI adapters.
+
+Selection convention V1 is explicit: input [0,...,N-1], reserved declared `$values`
+containing the admitted population, and exactly the other declared bindings from
+the caller. Overriding values or missing/foreign/invalid binding names fails.
+The finite population default is 65,536; zero is valid. Native and Transform
+limits remain additional ceilings, including live collection slots. Projection
+slots/text/metadata and numeric construction are admitted before owned growth.
+Population admission occurs once, including unused tails, on the same context as
+projection, VM work, native index validation, output leases and completion.
+
+Optional input/inputs callbacks use existing query semantics and explicit caller
+code. They cannot expand the declared population. Nil is provider EOF, null is a
+value. Provider admission/calls triggered by evaluation consume the same finite
+input/work/retention policy; arbitrary callback instructions and prior caller
+allocations remain caller work. A separate health cell supplies prepaid entry
+snapshots and prevents context borrowing during reentry. Provider/reentry/cancel
+failures latch before accepting another value or another callback.
+
+Public positions retain one shared population owner and a per-output slot lease.
+The population owner includes unforgeable invocation identity. Completion exists
+only after explicit evaluator EOF and successful cleanup; its identity/report
+lease does not retain the population. Completion.ordinal(of:) rejects foreign
+positions through constant metadata inspection, without reopening a completed
+operation. Raw ordinals/caller mappings are not stable IDs or read grants.
+
+Boundary invalidity bypasses query catches. Uncaught language, resource, provider,
+cancellation and lifecycle failures latch. Caught errors, abandoned prefixes and
+late failures cannot certify. A first operational failure wins over a later caller
+error; otherwise caller errors retain identity. Escaped scopes/closures cannot
+resume after close. report() drains deferred refunds after success/error/cancel;
+work never refunds. Completion.statistics is a close-time snapshot; current
+terminal accounting is observed through the guarded scope. Scope/position/
+completion handles are deliberately non-Sendable; plans remain checked Sendable.
+
+Completion, report, scope and position constructors are inaccessible to public
+callers. The generic JSONOperationReport initializer cannot construct a selection
+witness. No provenance propagation, second interpreter, VM serialization, online
+resolution, Content code/lock or unbounded streaming/sort promise is introduced.
+See the [public guide](../../../Sources/JSONTransform/Documentation.docc/IndexedSelection.md) for usage.
+
## Definition V1
JSONTransformProgramDefinition stores exact source and unique ASCII variable names
@@ -44,7 +92,7 @@ the existing compiler/context and returns actual native work. The additive
compileWithReport path preserves old compile behavior; the definition adapter
also charges its owned variable Set construction. No public meter is exposed.
-## Indexed selection contract for Sprint 2
+## Historical Sprint 1 selection plan
The proposed convention is an invocation-owned ordered array [0,...,N-1] as input
and a reserved explicit values binding containing the complete population. The
diff --git a/Documentation/Qualification/TestDiscovery.json b/Documentation/Qualification/TestDiscovery.json
index 60b4b1e..fddc9ab 100644
--- a/Documentation/Qualification/TestDiscovery.json
+++ b/Documentation/Qualification/TestDiscovery.json
@@ -13,7 +13,7 @@
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
"JSONInspectionTests": 214,
- "JSONTransformTests": 41,
+ "JSONTransformTests": 57,
"JSONDSLTests": 47
},
"parameterized_invocations": 390,
@@ -38,7 +38,7 @@
"JSONNavigationTests": 2,
"JSONSchemaExamplesTests": 4,
"JSONInspectionTests": 11,
- "JSONTransformTests": 3,
+ "JSONTransformTests": 4,
"JSONDSLTests": 1
},
"coverage_unmapped": {
@@ -85,7 +85,7 @@
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
"JSONInspectionTests": 214,
- "JSONTransformTests": 41,
+ "JSONTransformTests": 57,
"JSONDSLTests": 47
},
"wasi_root_functions": {
@@ -100,7 +100,7 @@
"JSONNavigationTests": 57,
"JSONSchemaExamplesTests": 98,
"JSONInspectionTests": 214,
- "JSONTransformTests": 41,
+ "JSONTransformTests": 57,
"JSONDSLTests": 47
},
"wasi_parameterized_invocations": 353,
@@ -116,7 +116,7 @@
"JSONNavigationTests": 2,
"JSONSchemaExamplesTests": 4,
"JSONInspectionTests": 11,
- "JSONTransformTests": 3,
+ "JSONTransformTests": 4,
"JSONDSLTests": 1
},
"export_functions": {
@@ -272,6 +272,22 @@
"JSONTransformTests.jtxTaskCancellationIsObservedAfterProviderOwnedProgress()",
"JSONTransformTests.jtxTerminalAliasesRefundBeforeThrowingTheSameFailure()",
"JSONTransformTests.jtxTypedCompileLocationAndLanguageFailureStayDistinct()",
- "JSONTransformTests.jtxZeroNullAndManyAreDistinctOrderedPullResults()"
+ "JSONTransformTests.jtxZeroNullAndManyAreDistinctOrderedPullResults()",
+ "JSONTransformTests.selectionS01ExplicitOrdinalsPreserveDuplicateValuesOrderRepeatsAndTies()",
+ "JSONTransformTests.selectionS02EmptyEOFNullAndIncompletePrefixAreDistinct()",
+ "JSONTransformTests.selectionS03EveryInvalidOutputIsExactAndTerminalOutsideQueryCatch()",
+ "JSONTransformTests.selectionS04CompletionRejectsForeignInvocationsAndPositionsRetainPopulation()",
+ "JSONTransformTests.selectionS05FinalAliasAndCompletionReleaseSettleToZeroWithoutDataPulls()",
+ "JSONTransformTests.selectionS06BindingsAreExplicitReservedAndScalarChecked()",
+ "JSONTransformTests.selectionS07PopulationPolicyIsInclusiveFiniteAndCannotSkipCompleteAdmission()",
+ "JSONTransformTests.selectionS08ProjectionEvaluationAndCompletionShareExactCumulativeWorkAndRetention()",
+ "JSONTransformTests.selectionS09QuotaAndCancellationRefundTerminalAliasesAndPreserveFirstCause()",
+ "JSONTransformTests.selectionS10ProviderEOFNullErrorAndNoFurtherPullsRemainDistinct()",
+ "JSONTransformTests.selectionS11ReentryAndProviderCancellationUseSafeEntrySnapshots()",
+ "JSONTransformTests.selectionS12CallerErrorAndEscapedClosuresReleaseProviderWithoutCompletion()",
+ "JSONTransformTests.selectionS13InnerProjectionTaskCancellationReleasesUnpublishedPreparation()",
+ "JSONTransformTests.selectionS14FailedPreparationAndCompletionAllocationsRefundExactBoundaries()",
+ "JSONTransformTests.selectionS15LanguageCatchesRemainOrdinaryAndUncaughtFailuresStayTerminal()",
+ "JSONTransformTests.selectionSharedImmutablePlanHasIndependentPopulationAndFailureHealth()"
]
}
diff --git a/Documentation/Qualification/TransformSelection1.8.md b/Documentation/Qualification/TransformSelection1.8.md
new file mode 100644
index 0000000..0913cc8
--- /dev/null
+++ b/Documentation/Qualification/TransformSelection1.8.md
@@ -0,0 +1,104 @@
+# Transform indexed selection — Sprint 2
+
+Status: implemented with scoped native qualification complete; full 1.8 release
+qualification remains PARTIAL. This local candidate extends sealed Sprint 1
+`70df9ba8d47322abf0d4aec1439256d7a5e97e94` on
+`codex/1.8-transform-selection-s2`. The previous branch/checkpoint, batches/store
+PR and all source-bound receipts remain intact. No merge, tag, publication or
+Content source-lock change is implied.
+
+The [contract](../Architecture/Release1.8/TransformBindings.md) and
+[compiled public guide](../../Sources/JSONTransform/Documentation.docc/IndexedSelection.md)
+define selection convention V1. The input is an owned index array and `$values`
+is the completely admitted population. Bindings supply exactly the other declared
+names. The finite population cap defaults to 65,536, alongside existing native
+and Transform policies. One private invocation context owns population admission,
+projection/index slots, evaluation, provider/output admission, exact conversion,
+position leases and completion. No engine language or numeric authority changes.
+
+## Ownership and failure decision
+
+Positions have private construction, exact ordinals, invocation identity and a
+joint population/output-slot lease. Completion/report construction is inaccessible
+to public callers. Explicit evaluator EOF and successful cleanup are mandatory;
+an abandoned prefix never certifies. A completion retains only identity/report
+storage. Its constant identity check rejects foreign positions even for identical
+populations. It does not reopen or retroactively fail a completed invocation.
+
+Scopes, positions and completions are deliberately non-Sendable. Escaped handles
+are guarded after close; plans remain checked Sendable for independent concurrent
+scopes. Providers are optional and explicit. Their values may influence selection
+but cannot enlarge the population. Callback work and prior caller allocations
+remain caller work. During callbacks, observation uses an admitted entry snapshot
+and reentry/cancellation touches health before the loaned context.
+
+Boundary invalidity, resources, uncaught language/provider failures, cancellation
+and reentry latch. Query catches retain their ordinary semantics but cannot catch
+boundary/operational failures. A first operational cause wins over a later caller
+error; otherwise caller errors retain identity. report() settles actual deferred
+refunds after failure/close without resetting work. Completion statistics are a
+close-time snapshot, not a later ARC settlement. Logical weights are not RSS,
+allocation-event measurements or arbitrary caller instruction quotas.
+
+## Actual native qualification — 29 October 2026
+
+Host: macOS 27.0 arm64, Apple Swift 6.4.2 (`swiftlang-6.4.2.1.9`, driver 1.168.6),
+Xcode 27.2/27B48, SDK 27.2, language mode 6. Private receipts/logs/events/source
+manifests are retained under `Artifacts/TransformBindingsS2`.
+
+| Check | Observed outcome |
+| --- | --- |
+| `Scripts/ci.py --lane transform-bindings` | Passed: 57 functions in Debug and Release, original 41 preserved, 32 Python harness cases, privacy/source identity and both public consumers |
+| `Scripts/ci.py --lane transform-bindings-safety` | Passed: 57 Release ASan functions, four shared-plan TSan functions, strict-memory target build with warnings as errors |
+| New outside-Git JSON/JSONTransform consumer | Passed: ten witnesses/configuration, exact compiled DocC guide, fourteen ordinary constructor/non-Sendable compiler refusals and two guarded-alias positives across Debug/Release |
+| Original source-definition public consumer | Passed: six witnesses/configuration and its four ordinary compiler refusals |
+| Original public Transform consumer and ownership compiler cases | Passed: twelve witnesses/configuration and all seven compiler cases |
+| `Scripts/ci.py --lane cli` | Passed: thirteen commands, 88 functions/37 parameter invocations and all existing command/vector/provider outcomes |
+| Existing manifest/dependency/cap commands | Passed through unchanged CI comparators: thirteen products, empty root external graph and all 49 exact cap outcomes |
+| Native Transform graph/DocC | Passed: 93 authored declarations/parameters documented, all 67 Sprint 1 identities/declaration fragments unchanged, 26 additions, warnings as errors |
+
+Sixteen new Swift Testing functions cover explicit duplicates/order/repeats/ties,
+exact scalar/number behavior, empty/EOF/null/incomplete distinctions, malformed/
+negative/fractional/overflow/native-width/out-of-population outputs, late invalidity
+outside query catch, foreign owners, final aliases and report-only settlement,
+reserved/invalid/missing bindings, complete unused-tail admission and zero/exact/
+one-over/overflow policies. Tests also pin cumulative work/retention/collection
+slots, completion-allocation failure, terminal quota/cancel refunds, provider error/
+EOF/null/reentry snapshots, caller/provider release, deterministic inner-projection
+task cancellation, uncaught language failure and independent concurrent reuse.
+
+The first test compilation retained ordinary errors in throwing assertion inference
+and one incorrect native metric name. Its corrected 13-case run passed before the
+final 16-case increment. No production semantic failure or compiler crash occurred.
+A supplemental private runner initially lacked its output directory and executed
+no command; the raw setup failure remains, and the diagnosed successor runs the
+same three commands. No failed attempt is relabeled as a pass or hidden.
+
+## Final binding and remaining gates
+
+The checkpoint binding separately maps this qualification closeout and the API
+index paragraph after passing receipts. Product, test, fixture, manifest, DocC,
+generator, identity and executable harness bytes match those checked. Native
+receipts are scoped to this actual source. No independent human review or broad
+platform acceptance is inferred. The scripted graph comparison is exact for all
+67 predecessor declaration fragments; full thirteen-catalog graphs remain separate.
+
+The private return retains deterministic source/checkpoint/diff and reproduction
+results. Full final-source Native/Linux ARM64+x86_64/safety/docs/WASI/relevant Apple
+SDK/runtime/API/archive gates and branch reconciliation remain release work.
+No new hosted CI, Linux, WASI or device pass follows from this sprint. Preserve
+main's independent documentation automation when deliberately reconciling later.
+No push/merge/tag/publication is performed by local qualification.
+
+The current expected discovery is 57 Transform roots/four sharing functions; all
+other populations and 390 native/portable parameter invocations (353 WASI) remain.
+Existing 49 policy probes preserve their exact distinctions, including zero-scratch
+success and invalid cancellation configuration. The population cap is additive
+policy, not a replacement metric or a calibrated RSS bound.
+
+No benchmark/allocation campaign ran. Earlier 1.8 loaded-host observations and
+unresolved regressions remain bound to 5e27899. Allocations remain unmeasured/
+unwaived for 1.8; the 1.7 waiver and historical holds are not promoted. Content's
+S06/S13 profile mapping and deliberate public source re-resolution remain separate
+acceptance work. This implementation supplies no Content IDs, locator/read grants,
+persistence, provenance propagation, online resolution or unbounded streaming.
diff --git a/README.md b/README.md
index 3b204ee..35c9ae3 100644
--- a/README.md
+++ b/README.md
@@ -4,6 +4,8 @@ This is **Swift-JSON 1.8.0-dev**, based on the published
[1.7.0 release](https://github.com/ThorFuchs/Swift-JSON/releases/tag/1.7.0)
(`8557badef562cf2f059e964c113d4a7f60ecbe6c`). The next additive wave brings
ordered batch reads and an immutable configured read-only store to JSONInspection.
+The optional JSONTransform candidate also adds exact source-definition codecs,
+compile reports and [finite indexed selection](Sources/JSONTransform/Documentation.docc/IndexedSelection.md).
It is under qualification; it is not an admitted 1.8 release.
Twelve libraries plus CLI, thirteen DocC catalogs and empty root dependencies remain.
See the [1.8 implementation contract](Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md),
diff --git a/Scripts/ci.py b/Scripts/ci.py
index 09aa283..bea99b3 100644
--- a/Scripts/ci.py
+++ b/Scripts/ci.py
@@ -28,6 +28,7 @@
SHARED_TESTS += '|graphG30ConcurrentViewsHaveIndependentHealth|graphGP14ConcurrentQueries'
SHARED_TESTS += '|dslSharedFinalizationAndPlansAreInvocationOwned'
SHARED_TESTS += '|storeR06TerminalQuotasNeverPublishAndImmutableConcurrentHealth'
+SHARED_TESTS += '|selectionSharedImmutablePlanHasIndependentPopulationAndFailureHealth'
SHARED_TESTS += '|bindingSharedDefinitionAndCompilationHaveIndependentHealth|bindingSharedSelectionSpikeReusesOnePlanIndependently'
SHARED_TESTS += '|workflowSharedImmutablePlansHaveIndependentBatchAndStoreHealth'
SHARED_TESTS += '|acceptance18SharedPlansKeepIndependentCrossProductHealth'
@@ -45,7 +46,7 @@ def portable(values):
if lane.endswith('safety'):
return portable([
['swift', 'test', '--test-product', 'JSONTransformTests', '-c', 'release', '--jobs', '2', '--sanitize', 'address', '--scratch-path', '.build/bindings-address'],
- ['swift', 'test', '--test-product', 'JSONTransformTests', '-c', 'release', '--jobs', '2', '--sanitize', 'thread', '--scratch-path', '.build/bindings-thread', '--filter', 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth|bindingShared'],
+ ['swift', 'test', '--test-product', 'JSONTransformTests', '-c', 'release', '--jobs', '2', '--sanitize', 'thread', '--scratch-path', '.build/bindings-thread', '--filter', 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth|bindingShared|selectionShared'],
['swift', 'build', '--target', 'JSONTransform', '--jobs', '2', '--scratch-path', '.build/bindings-strict', '-Xswiftc', '-strict-memory-safety', '-Xswiftc', '-warnings-as-errors'],
])
return portable([
@@ -55,6 +56,7 @@ def portable(values):
['swift', 'test', '--test-product', 'JSONTransformTests', '--jobs', '2'],
['swift', 'test', '--test-product', 'JSONTransformTests', '-c', 'release', '--jobs', '2'],
['python3', '-B', 'Scripts/qualify_transform_bindings.py', '--output', str(output / 'public-consumer')],
+ ['python3', '-B', 'Scripts/qualify_transform_selection.py', '--output', str(output / 'selection-consumer')],
])
if lane in ('structural-acceptance', 'structural-acceptance-safety'):
if lane.endswith('safety'):
@@ -264,6 +266,7 @@ def portable(values):
['python3', '-B', 'Scripts/qualify_transform_ownership.py', '--output', str(output / 'transform-ownership')],
['python3', '-B', 'Scripts/qualify_transform.py', '--output', str(output / 'transform-consumer')],
['python3', '-B', 'Scripts/qualify_transform_bindings.py', '--output', str(output / 'transform-binding-consumer')],
+ ['python3', '-B', 'Scripts/qualify_transform_selection.py', '--output', str(output / 'transform-selection-consumer')],
['python3', '-B', 'Scripts/qualify_release_1_1.py', '--mode', 'ownership', '--output', str(output / 'ownership.json')],
['python3', '-B', 'Scripts/qualify_release_1_1.py', '--mode', 'reader', '--output', str(output / 'reader.json')],
['swift', 'run', '-c', 'release', '--package-path', 'Examples/Consumer', 'Consumer'],
@@ -357,7 +360,9 @@ def check_result(command, code, log, events=None):
if 'JSONTransformTests' in command:
ids = set(DISCOVERY['transform_function_ids'])
if 'thread' in command:
- ids = {n for n in ids if 'bindingShared' in n or 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth' in n}
+ ids = {n for n in ids if 'bindingShared' in n or 'selectionShared' in n or 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth' in n}
+ elif selected_filter == 'selection':
+ ids = {n for n in ids if '.selection' in n}
elif selected_filter == 'binding':
ids = {n for n in ids if '.binding' in n}
expected = [len(ids)]
@@ -528,7 +533,10 @@ def check_result(command, code, log, events=None):
if command[:3] == ['python3', '-B', 'Scripts/qualify_html_browser.py']:
passed = passed and 'JSONHTML_BROWSER passed engines=3 cases=76 skips=0' in log.splitlines()
if 'Scripts/test_qualification_harness.py' in command:
- passed = passed and 'HARNESS_TESTS passed cases=31' in log.splitlines()
+ passed = passed and 'HARNESS_TESTS passed cases=32' in log.splitlines()
+ if command[:3] == ['python3', '-B', 'Scripts/qualify_transform_selection.py']:
+ data = json.loads(log.strip().splitlines()[-1])
+ passed = passed and data['status'] == 'passed' and len(data['commands']) == 20 and all(x['passed'] for x in data['commands']) and data['witnesses_per_configuration'] == 10 and data['negative_fixture_invocations'] == 14 and data['compiled_guide_fences'] == 1
if command[:3] == ['python3', '-B', 'Scripts/qualify_transform_bindings.py']:
data = json.loads(log.strip().splitlines()[-1])
passed = passed and data['status'] == 'passed' and len(data['commands']) == 8 and all(x['passed'] for x in data['commands']) and data['witnesses_per_configuration'] == 6 and data['negative_fixture_invocations'] == 4 and data['compiled_guide_fences'] == 1
diff --git a/Scripts/qualify_transform_selection.py b/Scripts/qualify_transform_selection.py
new file mode 100644
index 0000000..175154d
--- /dev/null
+++ b/Scripts/qualify_transform_selection.py
@@ -0,0 +1,63 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Public-only selection consumer and ownership/completion compiler boundaries."""
+import argparse, hashlib, json, pathlib, platform, re, shutil, subprocess, sys, tempfile, textwrap
+from qualify_transform_ownership import ordinary_rejection
+ROOT = pathlib.Path(__file__).resolve().parents[1]
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__); parser.add_argument('--output', required=True)
+ args = parser.parse_args(); output = (ROOT/args.output).resolve(); output.mkdir(parents=True, exist_ok=True)
+ guide=(ROOT/'Sources/JSONTransform/Documentation.docc/IndexedSelection.md').read_text()
+ fences=re.findall(r'```swift\n(.*?)\n```',guide,re.S)
+ consumer_text=(ROOT/'Tools/TransformSelectionQualification/Sources/TransformSelectionConsumer/Main.swift').read_text()
+ body=re.search(r'func selectionGuideExample\(\) throws \{\n(.*?)\n\}\nprivate',consumer_text,re.S).group(1)
+ assert len(fences)==1 and textwrap.dedent(body).strip()=='\n'.join(line for line in fences[0].splitlines() if not line.startswith('import ')).strip()
+ commands = []
+ def run(command, name):
+ result = subprocess.run(command,cwd=ROOT,capture_output=True,text=True)
+ data=result.stdout+result.stderr; path=output/(name+'.log');path.write_text(data)
+ row=dict(command=command,exit_code=result.returncode,log=path.name,sha256=hashlib.sha256(path.read_bytes()).hexdigest())
+ commands.append(row); return result,row
+ with tempfile.TemporaryDirectory(prefix='swift-json-selection-consumer-') as temporary:
+ consumer=pathlib.Path(temporary)
+ for p in (ROOT/'Tools/TransformSelectionQualification').rglob('*'):
+ if p.is_file() and not any(x in p.parts for x in ('.build','.swiftpm')):
+ target=consumer/p.relative_to(ROOT/'Tools/TransformSelectionQualification');target.parent.mkdir(parents=True,exist_ok=True);target.write_bytes(p.read_bytes())
+ manifest=consumer/'Package.swift';manifest.write_text(manifest.read_text().replace('path: "../.."','path: '+json.dumps(str(ROOT))))
+ inputs=[dict(path=p.relative_to(consumer).as_posix(),sha256=hashlib.sha256(p.read_bytes()).hexdigest()) for p in sorted(consumer.rglob('*')) if p.is_file()]
+ for config in ['debug','release']:
+ result,row=run(['swift','run','--package-path',str(consumer),'-c',config,'--jobs','2','TransformSelectionConsumer'],config+'-consumer')
+ row['passed']=result.returncode==0 and result.stdout.splitlines()==['TRANSFORM_SELECTION passed witnesses=10']
+ if not row['passed']:break
+ result,row=run(['swift','build','--package-path',str(consumer),'-c',config,'--show-bin-path'],config+'-binpath')
+ row['passed']=result.returncode==0
+ if not row['passed']:break
+ modules=pathlib.Path(result.stdout.strip()); binary=modules/'TransformSelectionConsumer'
+ target=output/(config+'-TransformSelectionConsumer');shutil.copyfile(binary,target)
+ row['binary_sha256']=hashlib.sha256(target.read_bytes()).hexdigest()
+ base=['swiftc','-swift-version','6','-I',str(modules),'-module-cache-path',str(output/'ModuleCache'),'-c']
+ if sys.platform=='darwin':
+ sdk=subprocess.check_output(['xcrun','--sdk','macosx','--show-sdk-path'],text=True).strip()
+ base+=['-sdk',sdk,'-target',platform.machine()+'-apple-macos27.0','-F',str(modules/'PackageFrameworks')]
+ fixtures = [
+ ('position','import JSONTransform\nlet x = JSONTransformSelectionPosition()\n',False,'inaccessible'),
+ ('completion','import JSONTransform\nlet x = JSONTransformSelectionCompletion()\n',False,'inaccessible'),
+ ('report','import JSONTransform\nfunc forge(_ c: JSONTransformSelectionCompletion) { _ = JSONTransformSelectionReport(result: (), completion: c) }\n',False,'inaccessible'),
+ ('scope','import JSONTransform\nlet x = JSONTransformSelectionScope()\n',False,'inaccessible'),
+ ('send-scope','import JSONTransform\nfunc send(_: T) {}\nfunc check(_ s: JSONTransformSelectionScope) { send(s) }\n',False,'does not conform to'),
+ ('send-position','import JSONTransform\nfunc send(_: T) {}\nfunc check(_ p: JSONTransformSelectionPosition) { send(p) }\n',False,'does not conform to'),
+ ('send-completion','import JSONTransform\nfunc send(_: T) {}\nfunc check(_ c: JSONTransformSelectionCompletion) { send(c) }\n',False,'does not conform to'),
+ ('guarded-alias','import JSONTransform\nfunc check(_ p: JSONTransformProgram) throws -> (() throws -> JSONTransformSelectionPosition?)? { var escaped: (() throws -> JSONTransformSelectionPosition?)?; _ = try p.withSelection(population: []) { s in escaped = { try s.next() }; let eof = try s.next(); precondition(eof == nil) }; return escaped }\n',True,None),
+ ]
+ for name,source,positive,diagnostic in fixtures:
+ path=output/(config+'-'+name+'.swift');path.write_text(source)
+ result,row=run(base+[str(path),'-o',str(output/(config+'-'+name+'.o'))],config+'-'+name)
+ row['positive']=positive;row['expected_diagnostic']=diagnostic;row['passed']=result.returncode==0 if positive else ordinary_rejection(result.returncode,result.stderr,diagnostic)
+ passed=len(commands)==20 and all(x['passed'] for x in commands)
+ receipt=dict(status='passed' if passed else 'failed',commands=commands,source_inputs=inputs,compiled_guide_fences=1,
+ configurations=2,witnesses_per_configuration=10,negative_fixture_invocations=14,
+ scope='Outside-Git public-only path consumer; no private imports or released-version resolution.')
+ (output/'receipt.json').write_text(json.dumps(receipt,indent=2)+'\n')
+ print(json.dumps(receipt));return 0 if passed else 1
+if __name__=='__main__':raise SystemExit(main())
diff --git a/Scripts/test_qualification_harness.py b/Scripts/test_qualification_harness.py
index ed4e9f0..feaa36b 100644
--- a/Scripts/test_qualification_harness.py
+++ b/Scripts/test_qualification_harness.py
@@ -28,18 +28,31 @@ def store_markers():
'STORE_RUNTIME R%02d passed' % n for n in range(1,11)])
class HarnessTests(unittest.TestCase):
+ def test_selection_filter_preserves_exact_identity_and_completion_evidence(self):
+ ids = [n for n in DISCOVERY['transform_function_ids'] if '.selection' in n]
+ self.assertEqual(len(ids), 16)
+ command = ['swift','test','--test-product','JSONTransformTests','--filter','selection']
+ events = dict(bundle_counts=[16],module_function_counts={'JSONTransformTests':16},function_names=ids,parameter_case_invocations=0)
+ self.assertTrue(check_result(command,0,'',events)[0])
+ self.assertFalse(check_result(command,0,'',dict(events,function_names=ids[:-1]))[0])
+ consumer = ['python3','-B','Scripts/qualify_transform_selection.py']
+ receipt = dict(status='passed',commands=[dict(passed=True)]*20,witnesses_per_configuration=10,negative_fixture_invocations=14,compiled_guide_fences=1)
+ self.assertTrue(check_result(consumer,0,json.dumps(receipt))[0])
+ for broken in [dict(receipt,commands=receipt['commands'][:-1]),dict(receipt,negative_fixture_invocations=0),dict(receipt,compiled_guide_fences=0)]:
+ self.assertFalse(check_result(consumer,0,json.dumps(broken))[0])
+
def test_transform_binding_discovery_rejects_incomplete_duplicate_and_foreign_events(self):
ids = DISCOVERY['transform_function_ids']
command = ['swift', 'test', '--test-product', 'JSONTransformTests']
- events = dict(bundle_counts=[41], module_function_counts={'JSONTransformTests':41},
+ events = dict(bundle_counts=[57], module_function_counts={'JSONTransformTests':57},
function_names=ids, parameter_case_invocations=0)
self.assertTrue(check_result(command, 0, '', events)[0])
for broken in [dict(events,function_names=ids[:-1]),dict(events,function_names=ids+ids[:1]),
- dict(events,module_function_counts={'JSONTests':41}),dict(events,parameter_case_invocations=1)]:
+ dict(events,module_function_counts={'JSONTests':57}),dict(events,parameter_case_invocations=1)]:
self.assertFalse(check_result(command, 0, '', broken)[0])
- shared = [n for n in ids if 'bindingShared' in n or 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth' in n]
+ shared = [n for n in ids if 'bindingShared' in n or 'selectionShared' in n or 'jtxPlanIsSendableAndExecutionsHaveIndependentHealth' in n]
thread = command + ['--sanitize', 'thread']
- self.assertTrue(check_result(thread, 0, '', dict(events,bundle_counts=[3],module_function_counts={'JSONTransformTests':3},function_names=shared))[0])
+ self.assertTrue(check_result(thread, 0, '', dict(events,bundle_counts=[4],module_function_counts={'JSONTransformTests':4},function_names=shared))[0])
def test_acceptance18_discovery_preserves_exact_public_pipeline_identities(self):
names = DISCOVERY['structural_acceptance_function_ids']
@@ -452,8 +465,8 @@ def test_exact_discovery_rejects_missing_duplicate_and_changed_parameters(self):
identity['function_names'][0] = 'JSONTests.unrelatedReplacement()'
self.assertFalse(check_restored(identity))
harness = ['python3', '-B', 'Scripts/test_qualification_harness.py']
- self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=31')[0])
- for stale in ['HARNESS_TESTS passed cases=30', 'HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', 'HARNESS_TESTS passed cases=28', '']:
+ self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=32')[0])
+ for stale in ['HARNESS_TESTS passed cases=31', 'HARNESS_TESTS passed cases=30', 'HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', 'HARNESS_TESTS passed cases=28', '']:
self.assertFalse(check_result(harness, 0, stale)[0])
lifetime = ['python3', '-B', 'Scripts/qualify_schema_example_lifetime.py']
marker = 'SCHEMA_EXAMPLES_CORPUS passed mode=owned scopes=128 values=6 exact=1'
diff --git a/Sources/JSONCLIEngine/CLIBuildIdentity.swift b/Sources/JSONCLIEngine/CLIBuildIdentity.swift
index a2f826c..ab25073 100644
--- a/Sources/JSONCLIEngine/CLIBuildIdentity.swift
+++ b/Sources/JSONCLIEngine/CLIBuildIdentity.swift
@@ -2,6 +2,6 @@
// Generated by Scripts/generate_cli_identity.py; excludes this file.
// SHA-256 of the ordered Package.swift + Sources content manifest, not a Git commit.
package enum CLIBuildIdentity {
- package static let revision = "sha256:2e6e326d160776afbff186a2c68d01968dc866159ede10038593be415098d91d"
+ package static let revision = "sha256:0a49346d574588626ee993fab93ac2fb38b4a392ea7bacb750e04e9e89855f61"
package static let kind = "source-sha256"
}
diff --git a/Sources/JSONTransform/Documentation.docc/IndexedSelection.md b/Sources/JSONTransform/Documentation.docc/IndexedSelection.md
new file mode 100644
index 0000000..c2f9000
--- /dev/null
+++ b/Sources/JSONTransform/Documentation.docc/IndexedSelection.md
@@ -0,0 +1,83 @@
+# Select explicit population positions
+
+Evaluate the existing query language over finite indices, preserving original
+positions even when values are identical. Selection convention V1 supplies
+`[0,...,N-1]` as input and the complete population in the reserved `$values`
+binding. Declare `values` at compilation; supply exactly the remaining names
+through explicit bindings. Stored values acquire no engine metadata.
+
+```swift
+import JSON
+import JSONTransform
+
+let definition = try JSONTransformProgramDefinition.prepare(
+ source: ".[] | select($values[.].enabled)", variables: ["values"]).result
+let plan = try definition.compile().result
+let row = try JSON.parse(#"{"enabled":true,"amount":9007199254740993}"#)
+let report = try plan.withSelection(population: [row, row]) { scope in
+ var positions: [JSONTransformSelectionPosition] = []
+ while let position = try scope.next() { positions.append(position) }
+ return positions
+}
+let ordinals = try report.result.map { try report.completion.ordinal(of: $0) }
+precondition(ordinals == [0, 1])
+precondition(report.completion.populationCount == 2)
+precondition(report.completion.outputCount == 2)
+```
+
+The evaluator is the ordinary Transform VM. Queries must emit exact nonnegative
+UInt64 ordinals within the population and the native Int range. Mathematical
+integer spellings such as 42.0 and negative zero convert exactly. Fractions,
+negative values, overflow, other JSON kinds and out-of-range outputs fail without
+Double conversion. Order and repeats are preserved. Sort tie rules belong in
+the query. The engine supplies no arbitrary transformed-value provenance.
+
+## Completion and ownership
+
+Pull nil explicitly. Returning after a prefix, even an empty prefix, throws
+``JSONTransformSelectionFailure/incompleteSelection``. A late invalid output,
+resource failure, cancellation, provider failure or reentry latches outside query
+catch handling; a caller catch cannot create a completion witness. Ordinary query
+language catches continue to work. A first operational failure takes precedence
+over a later caller-body error. An otherwise uncaught body error keeps its identity.
+
+Only ``JSONTransformProgram/withSelection(population:bindings:limits:source:_:)``
+creates a report and completion after EOF. Their initializers and position/scope
+construction are inaccessible to public callers. Completion verifies a position's
+invocation identity through ``JSONTransformSelectionCompletion/ordinal(of:)``;
+identical population values from another invocation still throw foreignPosition.
+That constant metadata inspection is caller work and does not reopen the VM or
+retroactively change a completed invocation.
+
+A position retains the admitted population and its output-slot lease after scope
+exit. A completion retains identity/report storage, not the population. Copies
+share reservations. Escaped scopes and closures cannot resume pulls after close.
+Use ``JSONTransformSelectionScope/report()`` to settle deferred ARC refunds and
+observe actual native counts after success, failure or cancellation. Work never
+refunds. Completion statistics are the successful close-time snapshot, not an
+automatically updating post-ARC view. Keep the appropriate handles alive while
+using accounted aliases. Scope, positions and completion are non-Sendable;
+immutable compiled plans are Sendable and support independent concurrent scopes.
+
+## Policy and providers
+
+``JSONTransformSelectionLimits`` defaults to 65,536 population values and balanced
+Transform/native limits. Population admission, index slots, all VM pulls, native
+numeric validation, output slots, provider admission and completion storage share
+one invocation ledger. A query producing no results still admits the whole
+population. The original 49 core caps remain additional finite policies.
+Logical weights are not heap allocation sizes or process RSS.
+
+An optional explicit source callback serves the existing input/inputs operators.
+Nil provider output means EOF; JSON null is a value and fails if emitted as an
+ordinal. Providers may influence selection but cannot enlarge the declared
+population. Their own instructions and already allocated values are caller work.
+Provider values and calls triggered by evaluation remain subject to cumulative
+native/Transform admission and input/work limits. Provider failure/cancellation
+stops further callbacks; a caught reentrant pull also fails the invocation.
+During the callback, report/used/remainingWorkUnits expose a prepaid entry snapshot.
+
+This is a finite selection channel, not a stable Content ID, locator, read grant,
+persistent execution, unbounded sort/stream guarantee or downstream admission.
+The selection convention revision is separate from the definition/dialect revision
+and the complete effective compilation/execution policies.
diff --git a/Sources/JSONTransform/Documentation.docc/JSONTransform.md b/Sources/JSONTransform/Documentation.docc/JSONTransform.md
index 0dfbfc2..c17abaf 100644
--- a/Sources/JSONTransform/Documentation.docc/JSONTransform.md
+++ b/Sources/JSONTransform/Documentation.docc/JSONTransform.md
@@ -3,7 +3,8 @@
Compile the existing exact query dialect and pull bounded native JSON results.
The library was introduced in 1.7.0. This 1.8 candidate adds exact source definitions
-and successful compile reports; indexed selection remains a test-only experiment.
+successful compile reports and guarded finite indexed selection with owner-bound
+positions and explicit completion witnesses.
Source identities and qualification stay separate. Publication requires a separate maintainer decision. It shares one private compiler
and VM with `swift-json query`; it does not introduce a new interpreter or numeric
authority. Plans are immutable and Sendable. Each synchronous invocation has one
@@ -29,3 +30,14 @@ finite cumulative meter and a guarded, non-Sendable execution handle.
- ``JSONTransformLimits``
- ``JSONTransformFailure``
+
+### Indexed selection
+
+-
+
+- ``JSONTransformSelectionLimits``
+- ``JSONTransformSelectionScope``
+- ``JSONTransformSelectionPosition``
+- ``JSONTransformSelectionCompletion``
+- ``JSONTransformSelectionReport``
+- ``JSONTransformSelectionFailure``
diff --git a/Sources/JSONTransform/Documentation.docc/SourceDefinitions.md b/Sources/JSONTransform/Documentation.docc/SourceDefinitions.md
index fda7777..af53fea 100644
--- a/Sources/JSONTransform/Documentation.docc/SourceDefinitions.md
+++ b/Sources/JSONTransform/Documentation.docc/SourceDefinitions.md
@@ -37,5 +37,5 @@ new numeric or query engine is introduced.
Use ``JSONTransformProgram/compileWithReport(_:variables:limits:)`` when compiling
ordinary source with successful native work statistics. These reports are data;
-they are not unforgeable selection-completion witnesses. The indexed selection
-experiment remains test-only in this sprint, with no public selection API.
+they are not unforgeable selection-completion witnesses. The separate guide defines the finite public selection
+convention, owner-bound positions and EOF witnesses.
diff --git a/Sources/JSONTransform/JSONTransformSelection.swift b/Sources/JSONTransform/JSONTransformSelection.swift
new file mode 100644
index 0000000..21a4958
--- /dev/null
+++ b/Sources/JSONTransform/JSONTransformSelection.swift
@@ -0,0 +1,382 @@
+// SPDX-License-Identifier: Apache-2.0
+import JSON
+import JSONFilterRuntime
+
+/// Finite policy for indexed-selection convention V1, additional to native limits.
+public struct JSONTransformSelectionLimits: Sendable {
+ /// The shared admission, projection, VM, provider and output policy.
+ public var transform: JSONTransformLimits = .init()
+ /// Inclusive population count; default 65,536. Zero admits an empty population.
+ public var maximumPopulationValues: UInt64 = 65_536
+ var checkpointObserver: (@Sendable (UInt64) -> Void)?
+ var projectionObserver: (@Sendable (UInt64) -> Void)?
+ /// Creates the finite balanced selection policy.
+ public init() {}
+}
+
+/// A selection-boundary refusal, distinct from language and operational engine errors.
+public enum JSONTransformSelectionFailure: Error, Sendable, Equatable {
+ /// An output was not an exact nonnegative UInt64 index in the population.
+ case invalidPosition
+ /// A position belongs to a different invocation, even with identical values.
+ case foreignPosition
+ /// The caller returned before pulling explicit evaluator EOF.
+ case incompleteSelection
+}
+
+private final class SelectionIdentity {
+ let lease: TransformValueLease
+ init(_ lease: TransformValueLease) { self.lease = lease }
+}
+private final class SelectionPopulation {
+ let value: FilterYield
+ let identity: SelectionIdentity
+ let metadata: TransformValueLease
+ init(_ value: FilterYield, identity: SelectionIdentity, metadata: TransformValueLease) {
+ self.value = value; self.identity = identity; self.metadata = metadata
+ }
+}
+
+/// An unforgeable invocation-local ordinal, retaining the population and its accounting.
+/// Copies share leases. The handle is deliberately non-Sendable; extracted ordinals
+/// and caller mappings are not engine provenance, stable IDs or read grants.
+public struct JSONTransformSelectionPosition {
+ /// Exact zero-based original population index; order and repeats are preserved.
+ public let ordinal: UInt64
+ private let population: SelectionPopulation
+ private let reservation: TransformValueLease
+ fileprivate init(_ ordinal: UInt64, population: SelectionPopulation, reservation: TransformValueLease) {
+ self.ordinal = ordinal; self.population = population; self.reservation = reservation
+ }
+ fileprivate var identity: SelectionIdentity { population.identity }
+}
+
+/// An owner-produced EOF witness and successful accounting snapshot; no public initializer.
+/// The witness retains only identity/report storage, not the population. Escaped
+/// positions retain that population separately. Logical reservations are not RSS.
+public struct JSONTransformSelectionCompletion {
+ /// Indexed-selection convention revision, independent of query and definition versions.
+ public var conventionRevision: UInt32 { 1 }
+ /// Completely admitted population count, including duplicate-valued members.
+ public let populationCount: UInt64
+ /// Successfully published positions, including repeats.
+ public let outputCount: UInt64
+ /// Successful cumulative native snapshot after closing VM/provider state.
+ public let statistics: JSONOperationStatistics
+ private let identity: SelectionIdentity
+ private let reservation: TransformValueLease
+ fileprivate init(identity: SelectionIdentity, reservation: TransformValueLease,
+ populationCount: UInt64, outputCount: UInt64, statistics: JSONOperationStatistics) {
+ self.identity = identity; self.reservation = reservation
+ self.populationCount = populationCount; self.outputCount = outputCount; self.statistics = statistics
+ }
+ /// Verifies invocation identity and extracts an ordinal after successful completion.
+ /// This constant metadata inspection is caller work, not another VM invocation.
+ /// - Parameter position: A position emitted by this invocation.
+ /// - Returns: Its exact original population index.
+ /// - Throws: ForeignPosition for a handle from another invocation.
+ public func ordinal(of position: JSONTransformSelectionPosition) throws -> UInt64 {
+ guard identity === position.identity else { throw JSONTransformSelectionFailure.foreignPosition }
+ return position.ordinal
+ }
+}
+
+/// The caller result paired with an unforgeable successful selection completion.
+public struct JSONTransformSelectionReport {
+ /// The value returned by the synchronous caller body.
+ public let result: Result
+ /// Successful EOF, invocation identity and actual native accounting.
+ public let completion: JSONTransformSelectionCompletion
+ fileprivate init(result: Result, completion: JSONTransformSelectionCompletion) {
+ self.result = result; self.completion = completion
+ }
+}
+
+private final class SelectionHealth {
+ struct Observation {
+ let statistics: JSONOperationStatistics
+ let remaining: UInt64
+ let counters: [UInt64]
+ }
+ var pulling = false
+ var failure: (any Error)?
+ var observation: Observation?
+ func latch(_ error: any Error) {
+ if failure == nil {
+ failure = error is JSONTransformSelectionFailure ? error : JSONTransformFailure.classify(error)
+ }
+ }
+ func check(context: inout TransformRuntimeContext) throws {
+ if let failure { context.latch(failure); throw failure }
+ }
+ func observe(context: inout TransformRuntimeContext) throws {
+ try context.work(66)
+ let cap = min(context.limits.core[.workUnits], context.limits[.totalWorkUnits])
+ observation = .init(statistics: context.meter.statistics(),
+ remaining: cap - min(cap, context.used(.totalWorkUnits)),
+ counters: TransformRuntimeMetric.allCases.map { context.used($0) })
+ }
+}
+private struct SelectionInput: FilterInput {
+ var source: (() throws -> JSON?)?
+ let health: SelectionHealth
+ var ended = false
+ mutating func next(context: inout TransformRuntimeContext) throws -> FilterYield? {
+ try health.check(context: &context); try context.checkpoint()
+ guard !ended, let source else { return nil }
+ try health.observe(context: &context)
+ defer { health.observation = nil }
+ let value: JSON?
+ do { value = try source() }
+ catch {
+ health.latch(JSONTransformFailure(error is CancellationError ? .cancelled : .provider,
+ error is CancellationError ? "cancelled" : "providerFailed"))
+ try health.check(context: &context); throw error
+ }
+ try health.check(context: &context); try context.checkpoint()
+ guard let value else { ended = true; self.source = nil; return nil }
+ try context.charge(.totalRecords)
+ return try FilterValues.produced(value, context: &context)
+ }
+}
+
+/// A guarded, non-Sendable indexed-selection pull scope with one cumulative ledger.
+/// V1 input is [0,...,N-1]; `$values` is the complete original population. Additional
+/// bindings are explicit. Provider values may influence queries but never enlarge
+/// the population. Returning without pulling nil cannot certify a prefix.
+public final class JSONTransformSelectionScope {
+ private var context: TransformRuntimeContext
+ private var vm: FilterVM?
+ private var population: SelectionPopulation?
+ private var input: SelectionInput
+ private let health: SelectionHealth
+ private var planBytes: UInt64 = 0, bindingBytes: UInt64 = 0, initialBindings: UInt64 = 0
+ private var ended = false, closed = false
+ private var published: UInt64 = 0
+ private let count: UInt64
+ private var cleanupFailure: (any Error)?
+ private static var stateBytes: UInt64 { 2048 }
+ private init(context: consuming TransformRuntimeContext, count: UInt64, source: (() throws -> JSON?)?) {
+ self.context = consume context; self.count = count
+ let health = SelectionHealth(); self.health = health
+ input = .init(source: source, health: health)
+ }
+ static func make(plan: JSONTransformProgram, values: [JSON], bindings: [String: JSON],
+ limits: JSONTransformSelectionLimits, source: (() throws -> JSON?)?,
+ observing: ((JSONTransformSelectionScope) -> Void)? = nil) throws -> JSONTransformSelectionScope {
+ do {
+ guard limits.maximumPopulationValues < UInt64(Int.max) else {
+ throw JSONTransformFailure(.configuration, "invalidLimits")
+ }
+ var context = try TransformRuntimeContext(limits: limits.transform.runtime)
+ try context.domain(.core)
+ context.meter.checkpointObserver = limits.checkpointObserver
+ try context.meter.retain(stateBytes)
+ let scope = JSONTransformSelectionScope(context: consume context, count: UInt64(values.count), source: source)
+ observing?(scope)
+ do {
+ try scope.prepare(plan, values: values, bindings: bindings, maximum: limits.maximumPopulationValues, projectionObserver: limits.projectionObserver)
+ return scope
+ } catch {
+ scope.health.latch(error)
+ scope.closePreservingFailure()
+ throw scope.health.failure ?? error
+ }
+ } catch {
+ if error is JSONTransformSelectionFailure { throw error }
+ throw JSONTransformFailure.classify(error)
+ }
+ }
+ private func prepare(_ plan: JSONTransformProgram, values: [JSON], bindings: [String: JSON], maximum: UInt64,
+ projectionObserver: (@Sendable (UInt64) -> Void)?) throws {
+ try context.checkpoint()
+ guard count <= maximum else {
+ throw JSONTransformFailure(.resource, "limitExceeded", metric: "populationValues", limit: maximum, attemptedUsage: count)
+ }
+ guard plan.variables.contains("values"), bindings.count == plan.variables.count - 1 else {
+ throw JSONTransformFailure(.configuration, "invalidBindings")
+ }
+ try context.meter.retain(plan.planReservation); planBytes = plan.planReservation
+ let identity = SelectionIdentity(try makeLease(64))
+ let admitted = try FilterValues.array(values, context: &context)
+ let metadata = try makeLease(48)
+ population = SelectionPopulation(admitted, identity: identity, metadata: metadata)
+ var supplied: [String: FilterYield] = [:]
+ try admitName("values")
+ supplied["values"] = admitted
+ for (name, value) in bindings {
+ try admitName(name)
+ guard name != "values", plan.variables.contains(name) else { throw JSONTransformFailure(.configuration, "invalidBindings") }
+ supplied[name] = try FilterValues.produced(value, context: &context)
+ }
+ var indices: [JSON] = [], scratch: UInt64 = 0, slots: UInt64 = 0
+ defer {
+ do { try context.meter.release(scratch); try context.release(.collectionValues, slots) }
+ catch { recordCleanup(error) }
+ }
+ for ordinal in values.indices {
+ try context.work(20)
+ try context.charge(.collectionValues); slots += 1
+ try context.meter.retain(64); scratch += 64
+ indices.append(.number(JSONNumber(integer: ordinal)))
+ projectionObserver?(UInt64(indices.count))
+ }
+ let root = try FilterValues.array(indices, admittedCollectionCount: true, context: &context)
+ slots = 0 // The root's lease owns these slots now.
+ indices = []; try context.meter.release(scratch); scratch = 0
+ try context.charge(.totalRecords)
+ vm = try FilterVM(program: plan.program, input: root, bindings: supplied, context: &context)
+ }
+ private func makeLease(_ bytes: UInt64) throws -> TransformValueLease {
+ try context.meter.retain(bytes)
+ do { return try TransformValueLease(ownedRetained: bytes, context: &context) }
+ catch { try context.meter.release(bytes); throw error }
+ }
+ private func admitName(_ name: String) throws {
+ try context.charge(.bindings); initialBindings += 1
+ var first = true
+ for byte in name.utf8 {
+ try context.work()
+ let letter = byte == 95 || (65...90).contains(byte) || (97...122).contains(byte)
+ guard letter || (!first && (48...57).contains(byte)) else { throw JSONTransformFailure(.configuration, "invalidBindings") }
+ first = false
+ }
+ guard !first else { throw JSONTransformFailure(.configuration, "invalidBindings") }
+ let before = context.meter.used(.retainedBytes)
+ do {
+ try JSONAdmission.text(name, key: true, meter: &context.meter, owned: true)
+ try context.meter.retain(64)
+ bindingBytes += context.meter.used(.retainedBytes) - before
+ } catch {
+ try context.meter.release(context.meter.used(.retainedBytes) - before); throw error
+ }
+ }
+ /// Pulls the next valid original-population position, or nil at explicit EOF.
+ /// Invalid outputs, resource/provider failures, cancellation and reentry latch;
+ /// query catches and later caller catches cannot restore successful completion.
+ public func next() throws -> JSONTransformSelectionPosition? {
+ guard !health.pulling else {
+ health.latch(JSONTransformFailure(.configuration, "reentrantSelection"))
+ throw health.failure ?? JSONTransformFailure(.configuration, "reentrantSelection")
+ }
+ health.pulling = true; defer { health.pulling = false }
+ do {
+ try context.reclaimValueLeases(); try health.check(context: &context)
+ try context.checkpoint()
+ guard !closed else { throw JSONTransformFailure(.configuration, "scopeClosed") }
+ guard !ended, var active = vm else { return nil }
+ vm = nil; defer { vm = active }
+ guard let raw = try active.next(input: &input, context: &context) else { ended = true; return nil }
+ let before = context.meter.used(.retainedBytes)
+ do { try context.meter.resetValueBounds(); try JSONAdmission.value(raw.value, meter: &context.meter) }
+ catch { try context.meter.release(context.meter.used(.retainedBytes) - before); throw error }
+ try context.charge(.totalResults)
+ guard case .number(let number) = raw.value else { throw JSONTransformSelectionFailure.invalidPosition }
+ let ordinal: UInt64
+ do { ordinal = try number.integer(as: UInt64.self, meter: &context.meter) }
+ catch let failure as JSONFailure where failure.category == .conversion { throw JSONTransformSelectionFailure.invalidPosition }
+ guard let native = Int(exactly: ordinal), native >= 0, ordinal < count, let population else {
+ throw JSONTransformSelectionFailure.invalidPosition
+ }
+ let lease = try makeLease(32)
+ published += 1
+ return .init(ordinal, population: population, reservation: lease)
+ } catch {
+ health.latch(error); throw health.failure ?? error
+ }
+ }
+ /// Drains deferred lease refunds and returns actual native accounting, even after failure/close.
+ /// During a provider callback this returns its admitted entry snapshot without
+ /// borrowing the VM's context. It never reopens data operations or refunds work.
+ /// - Throws: An accounting cleanup failure; a prior operational cause takes precedence.
+ public func report() throws -> JSONOperationStatistics {
+ if health.pulling {
+ if let observation = health.observation { return observation.statistics }
+ health.latch(JSONTransformFailure(.configuration, "reentrantSelection"))
+ throw health.failure ?? JSONTransformFailure(.configuration, "reentrantSelection")
+ }
+ do { try context.reclaimValueLeases() }
+ catch { recordCleanup(error); throw health.failure ?? error }
+ if let cleanupFailure { throw health.failure ?? cleanupFailure }
+ return context.meter.statistics()
+ }
+ /// Remaining cumulative owned work; provider entry uses an admitted snapshot.
+ public var remainingWorkUnits: UInt64 {
+ if health.pulling { return health.observation?.remaining ?? 0 }
+ let cap = min(context.limits.core[.workUnits], context.limits[.totalWorkUnits])
+ return cap - min(cap, context.used(.totalWorkUnits))
+ }
+ /// Reads a cumulative/live transform counter; call report to settle deferred refunds.
+ /// - Parameter metric: The existing transform counter, without a public mutable meter.
+ public func used(_ metric: JSONTransformLimits.Metric) -> UInt64 {
+ if health.pulling { return health.observation?.counters[metric.runtime.index] ?? 0 }
+ return context.used(metric.runtime)
+ }
+ /// Latches cancellation without borrowing a context loaned to the provider.
+ public func cancel() {
+ guard !closed else { return }
+ health.latch(JSONTransformFailure(.cancelled, "cancelled"))
+ if !health.pulling, let failure = health.failure { context.latch(failure) }
+ }
+ fileprivate var firstFailure: (any Error)? { health.failure }
+ fileprivate func complete() throws -> JSONTransformSelectionCompletion {
+ do {
+ try context.checkpoint(); try health.check(context: &context)
+ if let cleanupFailure { throw cleanupFailure }
+ guard ended, let identity = population?.identity else { throw JSONTransformSelectionFailure.incompleteSelection }
+ try context.work(50)
+ let lease = try makeLease(1024)
+ try close()
+ return .init(identity: identity, reservation: lease, populationCount: count,
+ outputCount: published, statistics: try report())
+ } catch { health.latch(error); throw health.failure ?? error }
+ }
+ private func recordCleanup(_ error: any Error) {
+ if cleanupFailure == nil { cleanupFailure = error }
+ health.latch(error)
+ }
+ fileprivate func closePreservingFailure() {
+ do { try close() } catch { recordCleanup(error) }
+ }
+ private func close() throws {
+ guard !closed else { return }; closed = true
+ input.source = nil
+ if var active = vm { vm = nil; try active.close(context: &context) }
+ population = nil
+ try context.reclaimValueLeases()
+ try context.meter.release(planBytes + bindingBytes + Self.stateBytes)
+ try context.release(.bindings, initialBindings)
+ planBytes = 0; bindingBytes = 0; initialBindings = 0
+ }
+}
+
+extension JSONTransformProgram {
+ /// Runs indexed-selection convention V1 on one fully admitted finite population.
+ /// Input is an owned index array; `$values` is the corresponding value array.
+ /// The plan must declare values, and bindings supply exactly the other names.
+ /// Provider work/bytes already owned by the caller stay outside engine quotas.
+ /// - Parameters:
+ /// - population: Explicit original order, including duplicate-valued members.
+ /// - bindings: Additional declared bindings; values is reserved and cannot be supplied here.
+ /// - limits: One finite ledger for admission, projection, evaluation and outputs.
+ /// - source: Optional explicit input/inputs provider; nil means no extra inputs.
+ /// - body: Synchronous caller work; pull nil to certify complete selection.
+ /// - Returns: Caller result and an owner-produced EOF/accounting witness.
+ /// - Throws: Selection refusals, typed operational failures or uncaught caller errors.
+ public func withSelection(population: [JSON], bindings: [String: JSON] = [:],
+ limits: JSONTransformSelectionLimits = .init(), source: (() throws -> JSON?)? = nil,
+ _ body: (JSONTransformSelectionScope) throws -> Result) throws -> JSONTransformSelectionReport {
+ let scope = try JSONTransformSelectionScope.make(plan: self, values: population, bindings: bindings, limits: limits, source: source)
+ let result: Result
+ do { result = try body(scope) }
+ catch {
+ let cause = scope.firstFailure ?? error
+ scope.closePreservingFailure(); throw cause
+ }
+ do { return .init(result: result, completion: try scope.complete()) }
+ catch {
+ let cause = scope.firstFailure ?? error
+ scope.closePreservingFailure(); throw cause
+ }
+ }
+}
diff --git a/Tests/JSONTransformTests/SelectionTests.swift b/Tests/JSONTransformTests/SelectionTests.swift
new file mode 100644
index 0000000..c3fc006
--- /dev/null
+++ b/Tests/JSONTransformTests/SelectionTests.swift
@@ -0,0 +1,338 @@
+// SPDX-License-Identifier: Apache-2.0
+import Testing
+import Synchronization
+import JSON
+@testable import JSONTransform
+
+private func selectionPlan(_ source: String, extra: Set = []) throws -> JSONTransformProgram {
+ try .compile(source, variables: extra.union(["values"]))
+}
+private func select(_ source: String, _ values: [JSON], limits: JSONTransformSelectionLimits = .init()) throws -> JSONTransformSelectionReport<[JSONTransformSelectionPosition]> {
+ try selectionPlan(source).withSelection(population: values, limits: limits) { scope in
+ var positions: [JSONTransformSelectionPosition] = []
+ while let p = try scope.next() { positions.append(p) }
+ return positions
+ }
+}
+private enum SelectionTestError: Error, Equatable { case sentinel }
+private func refusal(_ expected: JSONTransformSelectionFailure, _ body: () throws -> Void) {
+ do { try body(); Issue.record("Expected selection refusal \(expected)") }
+ catch let e as JSONTransformSelectionFailure { #expect(e == expected) }
+ catch { Issue.record("Wrong failure \(error)") }
+}
+
+@Test func selectionS01ExplicitOrdinalsPreserveDuplicateValuesOrderRepeatsAndTies() throws {
+ let row = try JSON.parse(#"{"score":1,"é":9007199254740993,"é":0.1}"#)
+ for (source, indices) in [(".[] | select(. == 1)", [UInt64(1)]), ("1,0,1", [1,0,1]),
+ ("sort_by([$values[.].score, .])[]", [0,1]),
+ (".[] | select($values[.][\"é\"] == 9007199254740993)", [0,1])] {
+ let report = try select(source, [row,row])
+ #expect(report.result.map(\.ordinal) == indices)
+ #expect(try report.result.map { try report.completion.ordinal(of: $0) } == indices)
+ #expect(report.completion.populationCount == 2 && report.completion.outputCount == UInt64(indices.count))
+ #expect(report.completion.conventionRevision == 1 && report.completion.statistics.domain == .core)
+ }
+}
+
+@Test func selectionS02EmptyEOFNullAndIncompletePrefixAreDistinct() throws {
+ var zero = JSONTransformSelectionLimits(); zero.maximumPopulationValues = 0
+ let empty = try select("empty", [], limits: zero)
+ #expect(empty.result.isEmpty && empty.completion.outputCount == 0 && empty.completion.populationCount == 0)
+ refusal(.incompleteSelection) { _ = try selectionPlan("empty").withSelection(population: []) { _ in () } }
+ refusal(.incompleteSelection) {
+ _ = try selectionPlan("0,0").withSelection(population: [42]) { s in _ = try s.next() }
+ }
+ refusal(.invalidPosition) { _ = try select("null", [.null]) }
+ let presentNull = try select("0", [.null])
+ #expect(presentNull.result.first?.ordinal == 0 && presentNull.completion.outputCount == 1)
+}
+
+@Test func selectionS03EveryInvalidOutputIsExactAndTerminalOutsideQueryCatch() throws {
+ for bad in ["null","false","\"0\"","[]","{}","-1","0.1","18446744073709551616","18446744073709551615","9223372036854775808","1"] {
+ var escaped: JSONTransformSelectionScope?
+ refusal(.invalidPosition) {
+ _ = try selectionPlan("try (0, " + bad + ") catch 0").withSelection(population: [42]) { s in
+ escaped = s; let first = try s.next(); #expect(first?.ordinal == 0)
+ for _ in 0..<2 { refusal(.invalidPosition) { _ = try s.next() } }
+ }
+ }
+ let scope = try #require(escaped)
+ refusal(.invalidPosition) { _ = try scope.next() }
+ #expect(try scope.report()[.retainedBytes] == 0)
+ }
+ #expect(try select("42.0", Array(repeating: .null, count: 43)).result.first?.ordinal == 42)
+ for source in ["0.0", "-0", "0e100"] { #expect(try select(source, [42]).result.first?.ordinal == 0) }
+}
+
+@Test func selectionS04CompletionRejectsForeignInvocationsAndPositionsRetainPopulation() throws {
+ let plan = try selectionPlan("0")
+ var scope: JSONTransformSelectionScope?
+ var a: JSONTransformSelectionReport? = try plan.withSelection(population: [.string(String(repeating: "x", count: 8192))]) { s in
+ scope = s; let p = try s.next(); #expect(try s.next() == nil); return p
+ }
+ var position = a?.result, alias = position
+ var completion = a?.completion
+ let b = try select("0", [.string(String(repeating: "x", count: 8192))])
+ let p = try #require(position), witness = try #require(completion)
+ refusal(.foreignPosition) { _ = try b.completion.ordinal(of: p) }
+ #expect(try witness.ordinal(of: p) == 0)
+ // Keep only aliases in a nested scope so last ARC releases are observable below.
+ a = nil; position = nil
+ let owner = try #require(scope)
+ let withAlias = try withExtendedLifetime(alias) { try owner.report()[.retainedBytes] }
+ #expect(withAlias > 8192)
+ alias = nil; completion = nil
+ // p/witness still own their real reservations until this function exits;
+ // the separate S05 case proves the exact last-alias transition.
+ #expect(try owner.report()[.retainedBytes] > 8192)
+ do { _ = try owner.next(); Issue.record("Escaped closed scope resumed") }
+ catch let e as JSONTransformFailure { #expect(e.category == .configuration && e.code == "scopeClosed") }
+}
+
+@Test func selectionS05FinalAliasAndCompletionReleaseSettleToZeroWithoutDataPulls() throws {
+ var scope: JSONTransformSelectionScope?
+ var position: JSONTransformSelectionPosition?
+ var completion: JSONTransformSelectionCompletion?
+ do {
+ let report = try selectionPlan("0").withSelection(population: [.string(String(repeating: "p", count: 8192))]) { s in
+ scope = s; position = try s.next(); #expect(try s.next() == nil)
+ }
+ completion = report.completion
+ }
+ let owner = try #require(scope)
+ let bytes = try withExtendedLifetime(position) { try owner.report()[.retainedBytes] }
+ #expect(bytes > 8192)
+ position = nil
+ let identityOnly = try withExtendedLifetime(completion) { try owner.report()[.retainedBytes] }
+ #expect(identityOnly > 0 && identityOnly < 8192)
+ completion = nil
+ let settled = try owner.report()
+ #expect(settled[.retainedBytes] == 0 && owner.used(.collectionValues) == 0 && owner.used(.bindings) == 0)
+ #expect(settled[.workUnits] > 0)
+}
+
+@Test func selectionS06BindingsAreExplicitReservedAndScalarChecked() throws {
+ let plan = try selectionPlan(".[] | select($values[.] >= $minimum)", extra: ["minimum"])
+ let report = try plan.withSelection(population: [1,2,3], bindings: ["minimum": 2]) { s in
+ var ids: [UInt64] = []; while let p = try s.next() { ids.append(p.ordinal) }; return ids
+ }
+ #expect(report.result == [1,2])
+ for bindings: [String:JSON] in [[:],["values": .null],["other": 2],["K": 2]] {
+ do { _ = try plan.withSelection(population: [1], bindings: bindings) { _ in () }; Issue.record("Accepted binding override/mismatch") }
+ catch let e as JSONTransformFailure { #expect(e.category == .configuration && e.code == "invalidBindings") }
+ }
+ do { _ = try JSONTransformProgram.compile("0").withSelection(population: [1]) { _ in () }; Issue.record("Missing reserved declaration accepted") }
+ catch let e as JSONTransformFailure { #expect(e.code == "invalidBindings") }
+}
+
+@Test func selectionS07PopulationPolicyIsInclusiveFiniteAndCannotSkipCompleteAdmission() throws {
+ var policy = JSONTransformSelectionLimits(); policy.maximumPopulationValues = 2
+ _ = try select("empty", [1,2], limits: policy)
+ policy.maximumPopulationValues = 1
+ do { _ = try select("empty", [1,2], limits: policy); Issue.record("Population cap escaped") }
+ catch let e as JSONTransformFailure { #expect(e.category == .resource && e.metric == "populationValues" && e.limit == 1 && e.attemptedUsage == 2) }
+ for cap in [UInt64.max, UInt64(Int.max)] {
+ policy = .init(); policy.maximumPopulationValues = cap
+ do { _ = try select("empty", [], limits: policy); Issue.record("Infinite/native overflow policy accepted") }
+ catch let e as JSONTransformFailure { #expect(e.category == .configuration && e.code == "invalidLimits") }
+ }
+ policy = .init(); policy.transform.json[.stringBytes] = 1
+ do { _ = try select("empty", [0,.string("too long")], limits: policy); Issue.record("Unused population tail bypassed admission") }
+ catch let e as JSONTransformFailure { #expect(e.category == .resource && e.metric == "stringBytes") }
+}
+
+@Test func selectionS08ProjectionEvaluationAndCompletionShareExactCumulativeWorkAndRetention() throws {
+ let source = "sort_by([$values[.],.])[]", values: [JSON] = [3,1,2]
+ let baseline = try select(source, values)
+ var policy = JSONTransformSelectionLimits()
+ let work = baseline.completion.statistics[.workUnits], peak = baseline.completion.statistics.peakLogicalRetainedBytes
+ policy.transform[.workUnits] = work
+ #expect(try select(source, values, limits: policy).result.map(\.ordinal) == [1,2,0])
+ policy.transform[.workUnits] = work - 1
+ do { _ = try select(source, values, limits: policy); Issue.record("Completion work reset") }
+ catch let e as JSONTransformFailure { #expect(e.category == .resource && e.metric == "workUnits") }
+ policy = .init(); policy.transform[.retainedBytes] = peak
+ _ = try select(source, values, limits: policy)
+ policy.transform[.retainedBytes] = peak - 1
+ do { _ = try select(source, values, limits: policy); Issue.record("Retention escaped") }
+ catch let e as JSONTransformFailure { #expect(e.category == .resource && e.metric == "retainedBytes") }
+ policy = .init(); policy.transform[.collectionValues] = 5
+ do { _ = try select("empty", values, limits: policy); Issue.record("Projection slots not pre-admitted") }
+ catch let e as JSONTransformFailure { #expect(e.category == .resource && e.metric == "collectionValues") }
+}
+
+@Test func selectionS09QuotaAndCancellationRefundTerminalAliasesAndPreserveFirstCause() throws {
+ for cancel in [false,true] {
+ var scope: JSONTransformSelectionScope?, output: JSONTransformSelectionPosition?
+ var policy = JSONTransformSelectionLimits(); policy.transform[.results] = 1
+ do {
+ _ = try selectionPlan("0,0").withSelection(population: [.string(String(repeating: "q", count: 4096))], limits: policy) { s in
+ scope = s; output = try s.next()
+ if cancel { s.cancel() }
+ for _ in 0..<2 {
+ do { _ = try s.next(); Issue.record("Terminal error swallowed") }
+ catch let e as JSONTransformFailure { #expect(e.category == (cancel ? .cancelled : .resource)) }
+ }
+ throw SelectionTestError.sentinel
+ }
+ Issue.record("Failed invocation certified")
+ } catch let e as JSONTransformFailure { #expect(e.category == (cancel ? .cancelled : .resource)) }
+ let s = try #require(scope)
+ let bytes = try withExtendedLifetime(output) { try s.report()[.retainedBytes] }
+ #expect(bytes > 4096); output = nil
+ #expect(try s.report()[.retainedBytes] == 0)
+ }
+}
+
+@Test func selectionS10ProviderEOFNullErrorAndNoFurtherPullsRemainDistinct() throws {
+ var calls = 0
+ let empty = try selectionPlan("inputs").withSelection(population: [], source: { calls += 1; return nil }) { s in let eof = try s.next(); #expect(eof == nil) }
+ #expect(calls == 1 && empty.completion.outputCount == 0)
+ calls = 0
+ refusal(.invalidPosition) {
+ _ = try selectionPlan("input").withSelection(population: [1], source: { calls += 1; return .null }) { s in
+ for _ in 0..<2 { refusal(.invalidPosition) { _ = try s.next() } }
+ }
+ }
+ #expect(calls == 1)
+ calls = 0
+ do {
+ _ = try selectionPlan("try input catch 0").withSelection(population: [1], source: { calls += 1; throw SelectionTestError.sentinel }) { s in
+ for _ in 0..<2 {
+ do { _ = try s.next(); Issue.record("Provider failure swallowed") }
+ catch let e as JSONTransformFailure { #expect(e.category == .provider && e.code == "providerFailed") }
+ }
+ }
+ Issue.record("Provider error certified")
+ } catch let e as JSONTransformFailure { #expect(e.category == .provider) }
+ #expect(calls == 1)
+}
+
+@Test func selectionS11ReentryAndProviderCancellationUseSafeEntrySnapshots() throws {
+ for cancel in [false,true] {
+ var escaped: JSONTransformSelectionScope?, calls = 0
+ do {
+ _ = try selectionPlan("input").withSelection(population: [42], source: {
+ calls += 1
+ if let s = escaped {
+ let report = try s.report(); #expect(report[.workUnits] == s.used(.workUnits))
+ #expect(s.remainingWorkUnits > 0)
+ if cancel { s.cancel() }
+ else {
+ do { _ = try s.next(); Issue.record("Provider reentry accepted") }
+ catch let e as JSONTransformFailure { #expect(e.code == "reentrantSelection") }
+ }
+ }
+ return 0
+ }) { s in
+ escaped = s
+ do { _ = try s.next(); Issue.record("Provider ignored terminal health") }
+ catch let e as JSONTransformFailure { #expect(e.category == (cancel ? .cancelled : .configuration)) }
+ }
+ Issue.record("Caught provider failure certified")
+ } catch let e as JSONTransformFailure { #expect(e.category == (cancel ? .cancelled : .configuration)) }
+ #expect(calls == 1)
+ #expect(try #require(escaped).report()[.retainedBytes] == 0)
+ }
+}
+
+@Test func selectionS12CallerErrorAndEscapedClosuresReleaseProviderWithoutCompletion() throws {
+ final class Source { var calls = 0; func next() -> JSON? { calls += 1; return 0 } }
+ weak var weakSource: Source?
+ var scope: JSONTransformSelectionScope?, closure: (() throws -> JSONTransformSelectionPosition?)?
+ do {
+ let source = Source(); weakSource = source
+ do {
+ _ = try selectionPlan("input").withSelection(population: [42], source: { source.next() }) { s in
+ scope = s; closure = { try s.next() }; throw SelectionTestError.sentinel
+ }
+ Issue.record("Caller error lost")
+ } catch let e as SelectionTestError { #expect(e == .sentinel) }
+ }
+ #expect(weakSource == nil)
+ do { _ = try closure?(); Issue.record("Escaped closure resumed") }
+ catch let e as JSONTransformFailure { #expect(e.code == "scopeClosed") }
+ #expect(try #require(scope).report()[.retainedBytes] == 0)
+}
+
+@Test func selectionSharedImmutablePlanHasIndependentPopulationAndFailureHealth() async throws {
+ let plan = try selectionPlan(".[]")
+ let results = try await withThrowingTaskGroup(of: [UInt64].self) { group in
+ for size in 0..<6 {
+ group.addTask {
+ var policy = JSONTransformSelectionLimits()
+ if size == 5 { policy.transform[.results] = 0 }
+ do {
+ let report = try plan.withSelection(population: Array(repeating: .null, count: size), limits: policy) { s in
+ var ids: [UInt64] = []; while let p = try s.next() { ids.append(p.ordinal) }; return ids
+ }
+ #expect(size != 5 && report.completion.populationCount == UInt64(size)); return report.result
+ } catch let e as JSONTransformFailure { #expect(size == 5 && e.category == .resource); return [99] }
+ }
+ }
+ var result: [[UInt64]] = []; for try await row in group { result.append(row) }; return result
+ }
+ #expect(results.count == 6 && results.contains([99]) && results.contains([0,1,2,3]))
+}
+
+@Test func selectionS13InnerProjectionTaskCancellationReleasesUnpublishedPreparation() async throws {
+ let plan = try selectionPlan(".[]")
+ let handle = Mutex?>(nil), ready = Mutex?>(nil)
+ let observed = Mutex(0)
+ let task = Task {
+ await withCheckedContinuation { continuation in ready.withLock { $0 = continuation } }
+ var policy = JSONTransformSelectionLimits()
+ policy.projectionObserver = { count in
+ if count == 4 { observed.withLock { $0 = count }; handle.withLock { $0?.cancel() } }
+ }
+ var failed: JSONTransformSelectionScope?
+ do {
+ _ = try JSONTransformSelectionScope.make(plan: plan, values: Array(repeating: .null, count: 32), bindings: [:], limits: policy, source: nil, observing: { failed = $0 })
+ Issue.record("Projection ignored task cancellation")
+ } catch let e as JSONTransformFailure { #expect(e.category == .cancelled) }
+ return try #require(failed).report()[.retainedBytes]
+ }
+ handle.withLock { $0 = task }
+ var continuation: CheckedContinuation?
+ while continuation == nil { continuation = ready.withLock { $0 }; if continuation == nil { await Task.yield() } }
+ continuation?.resume()
+ #expect(try await task.value == 0)
+ #expect(observed.withLock { $0 } == 4)
+}
+
+@Test func selectionS14FailedPreparationAndCompletionAllocationsRefundExactBoundaries() throws {
+ let plan = try selectionPlan("empty")
+ var failed: JSONTransformSelectionScope?
+ var policy = JSONTransformSelectionLimits(); policy.transform[.collectionValues] = 3
+ do {
+ _ = try JSONTransformSelectionScope.make(plan: plan, values: [1,2], bindings: [:], limits: policy, source: nil, observing: { failed = $0 })
+ Issue.record("Partial projection admitted")
+ } catch let e as JSONTransformFailure { #expect(e.category == .resource && e.metric == "collectionValues") }
+ #expect(try #require(failed).report()[.retainedBytes] == 0)
+ let base = try select("empty", [])
+ policy = .init(); policy.transform[.workUnits] = base.completion.statistics[.workUnits] - 1
+ var closed: JSONTransformSelectionScope?
+ do {
+ _ = try selectionPlan("empty").withSelection(population: [], limits: policy) { s in
+ closed = s; let eof = try s.next(); #expect(eof == nil)
+ }
+ Issue.record("Completion exceeded work budget")
+ } catch let e as JSONTransformFailure { #expect(e.category == .resource) }
+ #expect(try #require(closed).report()[.retainedBytes] == 0)
+}
+
+@Test func selectionS15LanguageCatchesRemainOrdinaryAndUncaughtFailuresStayTerminal() throws {
+ #expect(try select("try error(\"expected\") catch 0", [42]).result.first?.ordinal == 0)
+ var scope: JSONTransformSelectionScope?
+ do {
+ _ = try selectionPlan("0,error(\"expected\")").withSelection(population: [42]) { s in
+ scope = s; let p = try s.next(); #expect(p?.ordinal == 0)
+ for _ in 0..<2 {
+ do { _ = try s.next(); Issue.record("Language failure reopened") }
+ catch let e as JSONTransformFailure { #expect(e.category == .language) }
+ }
+ }
+ Issue.record("Uncaught language failure certified")
+ } catch let e as JSONTransformFailure { #expect(e.category == .language) }
+ #expect(try #require(scope).report()[.retainedBytes] == 0)
+}
diff --git a/Tools/TransformSelectionQualification/Package.swift b/Tools/TransformSelectionQualification/Package.swift
new file mode 100644
index 0000000..3985a93
--- /dev/null
+++ b/Tools/TransformSelectionQualification/Package.swift
@@ -0,0 +1,8 @@
+// swift-tools-version: 6.4
+// SPDX-License-Identifier: Apache-2.0
+import PackageDescription
+let package = Package(name: "TransformSelectionQualification", platforms: [.macOS("27.0"), .iOS("27.0")],
+ dependencies: [.package(name: "Swift-JSON", path: "../..")],
+ targets: [.executableTarget(name: "TransformSelectionConsumer", dependencies: [
+ .product(name: "JSON", package: "Swift-JSON"), .product(name: "JSONTransform", package: "Swift-JSON")
+ ])], swiftLanguageModes: [.v6])
diff --git a/Tools/TransformSelectionQualification/README.md b/Tools/TransformSelectionQualification/README.md
new file mode 100644
index 0000000..02a5783
--- /dev/null
+++ b/Tools/TransformSelectionQualification/README.md
@@ -0,0 +1,10 @@
+# Public indexed-selection consumer
+
+Run `swift run --package-path Tools/TransformSelectionQualification TransformSelectionConsumer`.
+Only JSON and JSONTransform are imported. Ten witnesses execute the exact DocC
+example, order/repeats, empty completion, late invalidity, foreign ownership,
+explicit bindings, final lease release, provider terminality, cumulative work
+and cancellation. The qualification script copies this package outside Git,
+runs Debug/Release and checks ordinary full-codegen public boundary refusals.
+A path dependency is preparation evidence, not version-tag resolution or
+Content S06/S13 admission. Logical reservations are not RSS or caller work caps.
diff --git a/Tools/TransformSelectionQualification/Sources/TransformSelectionConsumer/Main.swift b/Tools/TransformSelectionQualification/Sources/TransformSelectionConsumer/Main.swift
new file mode 100644
index 0000000..d6a214d
--- /dev/null
+++ b/Tools/TransformSelectionQualification/Sources/TransformSelectionConsumer/Main.swift
@@ -0,0 +1,103 @@
+// SPDX-License-Identifier: Apache-2.0
+import JSON
+import JSONTransform
+
+func selectionGuideExample() throws {
+ let definition = try JSONTransformProgramDefinition.prepare(
+ source: ".[] | select($values[.].enabled)", variables: ["values"]).result
+ let plan = try definition.compile().result
+ let row = try JSON.parse(#"{"enabled":true,"amount":9007199254740993}"#)
+ let report = try plan.withSelection(population: [row, row]) { scope in
+ var positions: [JSONTransformSelectionPosition] = []
+ while let position = try scope.next() { positions.append(position) }
+ return positions
+ }
+ let ordinals = try report.result.map { try report.completion.ordinal(of: $0) }
+ precondition(ordinals == [0, 1])
+ precondition(report.completion.populationCount == 2)
+ precondition(report.completion.outputCount == 2)
+}
+private func run(_ source: String, _ population: [JSON], limits: JSONTransformSelectionLimits = .init()) throws -> JSONTransformSelectionReport<[JSONTransformSelectionPosition]> {
+ try JSONTransformProgram.compile(source, variables: ["values"]).withSelection(population: population, limits: limits) { s in
+ var positions: [JSONTransformSelectionPosition] = []
+ while let p = try s.next() { positions.append(p) }; return positions
+ }
+}
+private enum ConsumerError: Error { case provider }
+@main enum Consumer {
+ static func main() throws {
+ var witnesses: [String] = []
+ try selectionGuideExample(); witnesses.append("compiled-guide")
+ let ordered = try run("1,0,1", [42,42])
+ precondition(ordered.result.map(\.ordinal) == [1,0,1]); witnesses.append("order-repeats")
+ let empty = try run("empty", [])
+ precondition(empty.completion.outputCount == 0 && empty.completion.populationCount == 0); witnesses.append("empty-eof")
+ do {
+ _ = try JSONTransformProgram.compile("try (0,null) catch 0", variables: ["values"]).withSelection(population: [42]) { s in
+ let first = try s.next(); precondition(first?.ordinal == 0)
+ do { _ = try s.next(); preconditionFailure("Invalid position accepted") }
+ catch let e as JSONTransformSelectionFailure { precondition(e == .invalidPosition) }
+ }
+ preconditionFailure("Caught boundary error certified")
+ } catch let e as JSONTransformSelectionFailure { precondition(e == .invalidPosition) }
+ witnesses.append("late-invalid")
+ let other = try run("0", [42,42])
+ do { _ = try other.completion.ordinal(of: ordered.result[0]); preconditionFailure("Foreign position accepted") }
+ catch let e as JSONTransformSelectionFailure { precondition(e == .foreignPosition) }
+ witnesses.append("foreign-owner")
+ let parameterized = try JSONTransformProgram.compile(".[] | select($values[.] >= $minimum)", variables: ["values","minimum"])
+ let selection = try parameterized.withSelection(population: [1,2,3], bindings: ["minimum":2]) { s in
+ var ids: [UInt64] = []; while let p = try s.next() { ids.append(p.ordinal) }; return ids
+ }
+ precondition(selection.result == [1,2]); witnesses.append("bindings")
+ var escaped: JSONTransformSelectionScope?
+ var position: JSONTransformSelectionPosition?
+ var completion: JSONTransformSelectionCompletion?
+ do {
+ let report = try JSONTransformProgram.compile("0", variables: ["values"]).withSelection(population: [.string(String(repeating:"x",count:8192))]) { s in
+ escaped = s; position = try s.next(); let eof = try s.next(); precondition(eof == nil)
+ }
+ completion = report.completion
+ }
+ let owner = escaped!
+ let retained = try withExtendedLifetime(position) { try owner.report()[.retainedBytes] }
+ precondition(retained > 8192); position = nil
+ let identityOnly = try withExtendedLifetime(completion) { try owner.report()[.retainedBytes] }
+ precondition(identityOnly > 0 && identityOnly < 8192); completion = nil
+ let settled = try owner.report(); precondition(settled[.retainedBytes] == 0)
+ do { _ = try owner.next(); preconditionFailure("Escaped scope resumed") }
+ catch let e as JSONTransformFailure { precondition(e.code == "scopeClosed") }
+ witnesses.append("terminal-refunds")
+ var calls = 0
+ do {
+ _ = try JSONTransformProgram.compile("try input catch 0", variables: ["values"]).withSelection(population: [42], source: { calls += 1; throw ConsumerError.provider }) { s in
+ for _ in 0..<2 {
+ do { _ = try s.next(); preconditionFailure("Provider error swallowed") }
+ catch let e as JSONTransformFailure { precondition(e.category == .provider) }
+ }
+ }
+ preconditionFailure("Provider error certified")
+ } catch let e as JSONTransformFailure { precondition(e.category == .provider) }
+ precondition(calls == 1); witnesses.append("provider-terminal")
+ let measured = try run(".[]", [1,2,3])
+ var limits = JSONTransformSelectionLimits(); limits.transform[.workUnits] = measured.completion.statistics[.workUnits]
+ _ = try run(".[]", [1,2,3], limits: limits)
+ limits.transform[.workUnits] -= 1
+ do { _ = try run(".[]", [1,2,3], limits: limits); preconditionFailure("Budget reset") }
+ catch let e as JSONTransformFailure { precondition(e.category == .resource) }
+ witnesses.append("cumulative-work")
+ var failed: JSONTransformSelectionScope?
+ do {
+ _ = try JSONTransformProgram.compile("0", variables: ["values"]).withSelection(population: [42]) { s in
+ failed = s; s.cancel()
+ do { _ = try s.next(); preconditionFailure("Cancel ignored") }
+ catch let e as JSONTransformFailure { precondition(e.category == .cancelled) }
+ }
+ preconditionFailure("Cancelled invocation certified")
+ } catch let e as JSONTransformFailure { precondition(e.category == .cancelled) }
+ let terminal = try failed!.report(); precondition(terminal[.retainedBytes] == 0)
+ witnesses.append("cancellation")
+ precondition(witnesses.count == 10)
+ print("TRANSFORM_SELECTION passed witnesses=10")
+ }
+}
From c9a5a4cd0c71a6c14fd743622d62026798b4b79d Mon Sep 17 00:00:00 2001
From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com>
Date: Thu, 29 Oct 2026 20:03:05 +0100
Subject: [PATCH 5/6] Run composition in dedicated CI execution lanes
---
.github/workflows/linux.yml | 25 ++++++--
.github/workflows/native.yml | 26 ++++++--
Documentation/Qualification/MaintenanceCI.md | 21 +++++++
Scripts/ci.py | 2 +-
Scripts/test_qualification_harness.py | 64 +++++++++++++++++++-
5 files changed, 124 insertions(+), 14 deletions(-)
diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
index 4f286e5..9a2349c 100644
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -11,13 +11,13 @@ concurrency:
cancel-in-progress: true
jobs:
qualification:
- name: Linux / ${{ matrix.architecture }} / ${{ matrix.lane }}
+ name: Linux execution / ${{ matrix.architecture }} / ${{ matrix.lane }}
runs-on: ${{ matrix.runner }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
- lane: [linux, linux-safety]
+ lane: [linux, linux-safety, composition]
architecture: [x86_64, arm64]
include:
- architecture: x86_64
@@ -47,9 +47,6 @@ jobs:
git config --global --add safe.directory "$GITHUB_WORKSPACE"
test "$(git rev-parse HEAD)" = "$GITHUB_SHA"
- run: python3 -B Scripts/ci.py --lane '${{ matrix.lane }}'
- - name: Public composition and reference imports
- if: matrix.lane == 'linux'
- run: python3 -B Scripts/ci.py --lane composition
- name: Preserve Linux receipts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
@@ -60,3 +57,21 @@ jobs:
path: |
.build/ci/
!.build/ci/**/reproduction/extracted/**
+ required:
+ # Both architectures and all three lanes must succeed. Preserve the four
+ # existing context names without converting an incomplete matrix to green.
+ name: Linux / ${{ matrix.architecture }} / ${{ matrix.lane }}
+ needs: qualification
+ if: ${{ always() }}
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ strategy:
+ fail-fast: false
+ matrix:
+ lane: [linux, linux-safety]
+ architecture: [x86_64, arm64]
+ steps:
+ - name: Require complete qualification
+ env:
+ QUALIFICATION_RESULT: ${{ needs.qualification.result }}
+ run: test "$QUALIFICATION_RESULT" = success
diff --git a/.github/workflows/native.yml b/.github/workflows/native.yml
index 2f1e707..efed818 100644
--- a/.github/workflows/native.yml
+++ b/.github/workflows/native.yml
@@ -11,13 +11,13 @@ concurrency:
cancel-in-progress: true
jobs:
qualification:
- name: Native / ${{ matrix.lane }}
+ name: Native execution / ${{ matrix.lane }}
runs-on: xcode-27
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
- lane: [native, safety, docs, archive]
+ lane: [native, safety, docs, archive, composition]
env:
DEVELOPER_DIR: /Applications/Xcode_27.2.app/Contents/Developer
PYTHONDONTWRITEBYTECODE: '1'
@@ -34,9 +34,6 @@ jobs:
- run: python3 -B Scripts/ci.py --lane '${{ matrix.lane }}'
env:
SWIFT_JSON_RELEASE_METADATA: ${{ github.workspace }}/.build/handoff/pages/latest-release.json
- - name: Public composition and reference imports
- if: matrix.lane == 'native'
- run: python3 -B Scripts/ci.py --lane composition
- name: Preserve native receipts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
@@ -63,10 +60,27 @@ jobs:
with:
path: .build/handoff/pages/site
include-hidden-files: true # Preserve the verified .nojekyll file; only the generated site is uploaded.
+ required:
+ # Preserve protected context names. Every execution lane, including
+ # composition, must succeed; a failed/cancelled/skipped dependency fails here.
+ name: Native / ${{ matrix.lane }}
+ needs: qualification
+ if: ${{ always() }}
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ strategy:
+ fail-fast: false
+ matrix:
+ lane: [native, safety, docs, archive]
+ steps:
+ - name: Require complete qualification
+ env:
+ QUALIFICATION_RESULT: ${{ needs.qualification.result }}
+ run: test "$QUALIFICATION_RESULT" = success
publish:
name: Documentation / publish
if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
- needs: qualification
+ needs: [qualification, required]
# Native compilation and all qualification happened in the jobs above.
# This job only publishes their exact static artifact.
runs-on: ubuntu-latest
diff --git a/Documentation/Qualification/MaintenanceCI.md b/Documentation/Qualification/MaintenanceCI.md
index 0e4d8fc..9058f8c 100644
--- a/Documentation/Qualification/MaintenanceCI.md
+++ b/Documentation/Qualification/MaintenanceCI.md
@@ -19,6 +19,27 @@ keeps the existing main-only condition; maintenance pushes must not replace the
development website. A skipped Pages publication on a maintenance branch is
expected. Required product qualification must still execute and pass.
+Composition has a separate execution matrix lane on macOS and on each Linux
+architecture. The native, safety, docs, archive and composition commands retain
+their existing 60-minute execution limits; no command or fixture is omitted.
+The existing `Native / ...` and `Linux / ...` check names are final gates over
+their complete workflow execution matrix. They run even after a dependency fails
+and accept only its `success` result. Failure, cancellation, a skipped execution
+or a missing result cannot produce a successful required context. Consequently,
+one failed execution makes all of that workflow's final gates fail. Inspect the
+`Native execution / ...` or `Linux execution / ...` job and its receipts for the
+specific cause. Pages also waits for the final native gates, with its main-only
+condition unchanged. No repository protection setting is changed by scheduling.
+
+This separation follows the 1.8 PR74 execution at `48dca4d`: the normal native and
+both Linux lanes passed, but the native and x86 Linux jobs reached their one-hour
+job limits during the appended composition step. Native's nine-command composition
+receipt had completed just before cancellation; x86 had completed only five of
+nine commands. Preserve those cancelled checks and partial receipts. They are not
+retroactively green, and the successor must qualify its actual source. Moving
+the unchanged composition commands to a separate job does not waive them or
+extend a timeout. It does not qualify held runtime/performance/allocation gates.
+
If an expected automatic run is absent, inspect the branch/event/commit and
workflow filters. Existing manual dispatch may qualify the exact source while
investigation proceeds, but it does not prove that automatic triggering works.
diff --git a/Scripts/ci.py b/Scripts/ci.py
index bea99b3..542f298 100644
--- a/Scripts/ci.py
+++ b/Scripts/ci.py
@@ -533,7 +533,7 @@ def check_result(command, code, log, events=None):
if command[:3] == ['python3', '-B', 'Scripts/qualify_html_browser.py']:
passed = passed and 'JSONHTML_BROWSER passed engines=3 cases=76 skips=0' in log.splitlines()
if 'Scripts/test_qualification_harness.py' in command:
- passed = passed and 'HARNESS_TESTS passed cases=32' in log.splitlines()
+ passed = passed and 'HARNESS_TESTS passed cases=35' in log.splitlines()
if command[:3] == ['python3', '-B', 'Scripts/qualify_transform_selection.py']:
data = json.loads(log.strip().splitlines()[-1])
passed = passed and data['status'] == 'passed' and len(data['commands']) == 20 and all(x['passed'] for x in data['commands']) and data['witnesses_per_configuration'] == 10 and data['negative_fixture_invocations'] == 14 and data['compiled_guide_fences'] == 1
diff --git a/Scripts/test_qualification_harness.py b/Scripts/test_qualification_harness.py
index feaa36b..2a71616 100644
--- a/Scripts/test_qualification_harness.py
+++ b/Scripts/test_qualification_harness.py
@@ -4,6 +4,8 @@
import copy
import json
import pathlib
+import os
+import re
import subprocess
from qualify_release_1_1 import checkout_head
from generate_cli_identity import rendered
@@ -28,6 +30,64 @@ def store_markers():
'STORE_RUNTIME R%02d passed' % n for n in range(1,11)])
class HarnessTests(unittest.TestCase):
+ def workflow_jobs(self, filename):
+ text = (pathlib.Path(__file__).resolve().parents[1]/'.github/workflows'/filename).read_text()
+ # Inspect this repository's authored job blocks, not a general YAML parser.
+ jobs = dict(re.findall(r'^ (\w+):\n(.*?)(?=^ \w+:\n|\Z)', text.split('jobs:\n',1)[1], re.M|re.S))
+ return text, jobs
+
+ def test_required_workflow_contexts_wait_for_complete_execution_matrices(self):
+ for filename, execution, required, name in [
+ ('native.yml','[native, safety, docs, archive, composition]','[native, safety, docs, archive]','Native / ${{ matrix.lane }}'),
+ ('linux.yml','[linux, linux-safety, composition]','[linux, linux-safety]','Linux / ${{ matrix.architecture }} / ${{ matrix.lane }}'),
+ ]:
+ with self.subTest(workflow=filename):
+ _, jobs = self.workflow_jobs(filename)
+ self.assertIn('lane: '+execution, jobs['qualification'])
+ self.assertIn("run: python3 -B Scripts/ci.py --lane '${{ matrix.lane }}'", jobs['qualification'])
+ self.assertNotIn("if: matrix.lane == 'composition'", jobs['qualification'])
+ self.assertNotIn('--lane composition', jobs['qualification'])
+ gate = jobs['required']
+ self.assertIn('name: '+name+'\n', gate)
+ self.assertIn('needs: qualification\n', gate)
+ self.assertIn('if: ${{ always() }}\n', gate)
+ self.assertIn('lane: '+required, gate)
+ if filename=='linux.yml':
+ for block in [jobs['qualification'], gate]:
+ self.assertIn('architecture: [x86_64, arm64]', block)
+
+ def test_workflow_gates_reject_failed_cancelled_skipped_and_missing_results(self):
+ for filename in ['native.yml','linux.yml']:
+ text, jobs = self.workflow_jobs(filename)
+ self.assertNotIn('continue-on-error', text)
+ gate = jobs['required']
+ self.assertIn('QUALIFICATION_RESULT: ${{ needs.qualification.result }}', gate)
+ script = re.findall(r'^ run: (.+)$', gate, re.M)
+ self.assertEqual(script, ['test "$QUALIFICATION_RESULT" = success'])
+ for result in ['success','failure','cancelled','skipped','', 'Success','success failure']:
+ run = subprocess.run(['bash','-c',script[0]],env=dict(os.environ,QUALIFICATION_RESULT=result),capture_output=True)
+ self.assertEqual(run.returncode==0, result=='success', (filename,result))
+ env = dict(os.environ); env.pop('QUALIFICATION_RESULT',None)
+ self.assertNotEqual(subprocess.run(['bash','-c',script[0]],env=env,capture_output=True).returncode,0)
+
+ def test_composition_scheduling_preserves_timeouts_pins_artifacts_and_pages_guard(self):
+ for filename in ['native.yml','linux.yml']:
+ text, jobs = self.workflow_jobs(filename)
+ self.assertIn('timeout-minutes: 60', jobs['qualification'])
+ self.assertIn('fail-fast: false', jobs['qualification'])
+ self.assertIn('persist-credentials: false', jobs['qualification'])
+ self.assertIn('retention-days: 30', jobs['qualification'])
+ self.assertIn('if-no-files-found: error', jobs['qualification'])
+ self.assertIn('actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803', text)
+ self.assertIn('actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f', text)
+ self.assertIn("branches: [main, 'release/1.6']", text)
+ _, native = self.workflow_jobs('native.yml')
+ self.assertIn("if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'", native['publish'])
+ self.assertIn('needs: [qualification, required]', native['publish'])
+ _, linux = self.workflow_jobs('linux.yml')
+ self.assertNotIn('publish', linux)
+ self.assertIn('swift@sha256:31d14d727f4451f25e29bac9c42563a098c39ac66415e8bad970f48038e38aca', linux['qualification'])
+
def test_selection_filter_preserves_exact_identity_and_completion_evidence(self):
ids = [n for n in DISCOVERY['transform_function_ids'] if '.selection' in n]
self.assertEqual(len(ids), 16)
@@ -465,8 +525,8 @@ def test_exact_discovery_rejects_missing_duplicate_and_changed_parameters(self):
identity['function_names'][0] = 'JSONTests.unrelatedReplacement()'
self.assertFalse(check_restored(identity))
harness = ['python3', '-B', 'Scripts/test_qualification_harness.py']
- self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=32')[0])
- for stale in ['HARNESS_TESTS passed cases=31', 'HARNESS_TESTS passed cases=30', 'HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', 'HARNESS_TESTS passed cases=28', '']:
+ self.assertTrue(check_result(harness, 0, 'HARNESS_TESTS passed cases=35')[0])
+ for stale in ['HARNESS_TESTS passed cases=32', 'HARNESS_TESTS passed cases=31', 'HARNESS_TESTS passed cases=30', 'HARNESS_TESTS passed cases=10', 'HARNESS_TESTS passed cases=11', 'HARNESS_TESTS passed cases=12', 'HARNESS_TESTS passed cases=13', 'HARNESS_TESTS passed cases=14', 'HARNESS_TESTS passed cases=15', 'HARNESS_TESTS passed cases=18', 'HARNESS_TESTS passed cases=19', 'HARNESS_TESTS passed cases=20', 'HARNESS_TESTS passed cases=21', 'HARNESS_TESTS passed cases=25', 'HARNESS_TESTS passed cases=26', 'HARNESS_TESTS passed cases=28', '']:
self.assertFalse(check_result(harness, 0, stale)[0])
lifetime = ['python3', '-B', 'Scripts/qualify_schema_example_lifetime.py']
marker = 'SCHEMA_EXAMPLES_CORPUS passed mode=owned scopes=128 values=6 exact=1'
From deb82a485bb701ec4c79394deae840e67766d832 Mon Sep 17 00:00:00 2001
From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com>
Date: Tue, 10 Nov 2026 00:35:56 +0100
Subject: [PATCH 6/6] Record maintainer admission of the public configured
store API
The maintainer approved the read-only configured store/follow surface as
public API on 2026-11-10, superseding the package-only store policy on
main. Record that in AGENTS.md (now the 1.9 candidate), the roadmap's
1.9.0 row (the store is no longer "if separately admitted"; admission does
not waive the release gates), the read-only store decision in the
qualification record and the bounded-workflows contract. This admits the
API shape only; merge and release remain separate decisions.
Co-Authored-By: Claude Opus 5.5
---
AGENTS.md | 5 +++--
.../StructuralIntelligence/BoundedWorkflows1.8.md | 2 ++
Documentation/Qualification/Release1.8.md | 5 +++++
Documentation/ReleaseRoadmap.md | 2 +-
4 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index b6c6108..ffe2f5f 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -50,9 +50,10 @@ Scalar scanning and immutable plan-local regex sharing are selected private
optimizations. Source-attached N3 snapshots and P1 tree/sparse-table projections live in JSONInspection.
NAV10 set/matrix and NAV11 directed-simple-graph APIs are development capabilities
inside JSONInspection; no graph product, mutation or serializer is introduced.
-The 1.8 candidate promotes the checked immutable configured store inside
+The 1.9 candidate promotes the checked immutable configured store inside
JSONInspection and adds ordered path/snapshot/store value batches, exact-ID lookup
-and bounded complete BFS follow. Preserve the original construction phases and
+and bounded complete BFS follow. The maintainer admitted this public read-only
+store/follow API on 2026-11-10; merge and release remain separate decisions. Preserve the original construction phases and
one invocation-owned ledger per operation. Positions remain unforgeable and
source-instance-local. No persistence or mutable store is exposed. See
Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md.
diff --git a/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md b/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
index 77c6890..55b3d1b 100644
--- a/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
+++ b/Documentation/Architecture/StructuralIntelligence/BoundedWorkflows1.8.md
@@ -4,6 +4,8 @@ Implementation contract for the maintainer-authorized 1.8 feature branch from
published1.7.0 (8557bade). This is an additive implementation, not merge or release
approval. Original NAV12 construction fixtures, failures and qualification retain
their source-bound historical meaning; public promotion adds its own evidence.
+The maintainer admitted the public read-only store/follow API on 2026-11-10; the
+candidate now targets 1.9.0 after the published 1.8.0 located-streaming release.
## Batch reads
diff --git a/Documentation/Qualification/Release1.8.md b/Documentation/Qualification/Release1.8.md
index daff11a..0f9f30f 100644
--- a/Documentation/Qualification/Release1.8.md
+++ b/Documentation/Qualification/Release1.8.md
@@ -154,6 +154,11 @@ external private-read-scope refusal; the other original seven negatives and all
65 original vectors remain. Store/position forging is still forbidden. This is
an intentional additive promotion; it adds no product or runtime dependency.
+Maintainer admission: on 2026-11-10 the maintainer approved making this read-only
+configured store/follow surface public API, superseding the earlier package-only
+policy. This admits the API shape for the 1.9 line; it is not merge or release
+approval, and the remaining source-bound gates still apply.
+
Fixed metadata access and supplied callbacks are trusted caller work. Each later
query is independent of construction and other queries. Logical retained bytes
are conservative owned reservations, not process memory. No persistence, mutable
diff --git a/Documentation/ReleaseRoadmap.md b/Documentation/ReleaseRoadmap.md
index 2f9e7e7..2aed94f 100644
--- a/Documentation/ReleaseRoadmap.md
+++ b/Documentation/ReleaseRoadmap.md
@@ -60,7 +60,7 @@ there is no public store/follow API or store product. Root dependencies stay emp
| --- | --- | --- |
| **1.7.0 — integrated foundations** | Release the complete current public consolidation together: Navigation/Inspection snapshots, projections and graph, Transform and both DSL products, alongside all stable products. Keep store implementation private. | Freeze current-main successor; qualify exact source, public consumers/API/DocC, archives/assets and supported platforms; resolve the new candidate's performance dispositions explicitly. |
| **1.8.0 — located root-array streaming** | Add JSONLocatedTopLevelArraySequence: root-array elements with zero-based half-open UTF-8 source spans over the existing shared grammar/driver, for consumers that stage original record bytes. | Additive public-consumer tests, facade parity on errors/limits/statistics, DocC and complete supported-platform CI on the exact source. |
-| **1.9.0 — bounded structural workflows** | Add independently reviewed, additive Inspection conveniences: aggregate batch reads and, if separately admitted and qualified, the minimal immutable public configured-store/follow interface. Improve Navigation/Inspection guides and error/resource ergonomics. | One-ledger aggregate proof, finite request/result/frontier profiles, ordinary compiler refusals, adversarial failure/cancellation and complete supported-platform evidence. Store can remain private if its gate is not complete. |
+| **1.9.0 — bounded structural workflows** | Add independently reviewed, additive Inspection conveniences: aggregate batch reads and the minimal immutable public configured-store/follow interface (public API admitted by the maintainer on 2026-11-10). Improve Navigation/Inspection guides and error/resource ergonomics. | One-ledger aggregate proof, finite request/result/frontier profiles, ordinary compiler refusals, adversarial failure/cancellation and complete supported-platform evidence. Admission of the store API does not waive these gates. |
| **1.10.0 — durable workflows and operational maturity** | Candidate wave for versioned exact snapshot/store persistence and source-bound restoration, after a dedicated design admission. Complete capacity/scaling characterization and consumer migration guidance. | Freeze the logical envelope, corruption/version/source-identity semantics, bounded decode/rebuild and disk teardown/reopen tests. No persistence or source-span claim from a bare JSON dump. |
| **2.0.0 — supported family and compatibility contract** | Ship all accepted products/capabilities under one coherent, documented compatibility, resource and platform contract. Complete end-to-end applications, migration tooling and the release qualification matrix. | Scope freeze, complete public API review and cross-product qualification, explicit treatment of every remaining hold, reproducible clean candidate and a separate owner tag/publication instruction. |