Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,14 @@
default; `JSONTextSequenceOptions.MalformedTexts.skip` skips them and counts
them on the iterator, but never skips resource failures. `JSONTextSequenceWriter`
writes RS, strict JSON and LF per value, as one sequence or in awaited chunks.
- Add `JSONEnvelope`, a versioned record of one payload in RFC 8785 canonical
bytes. Format version 1 is a canonical object with `digest`, `format`
(`swift-json-envelope`), `formatVersion` (1), `payload`, `type` and `version`.
The digest is the SHA-256 of the payload's canonical bytes; decoding checks the
marker, fields and digest and reports `JSONEnvelopeFailure` for invalid or
damaged envelopes. Equal envelopes encode to equal bytes, and equal payloads
have equal digests for content addressing. Tests pin bytes and digests computed
independently.

## Unreleased — 1.9.1 hygiene

Expand Down
6 changes: 3 additions & 3 deletions Documentation/Qualification/TestDiscovery.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"schema_version": 2,
"root_functions": {
"JSONTests": 198,
"JSONTests": 203,
"JSONSchemaTests": 79,
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 42,
Expand Down Expand Up @@ -73,7 +73,7 @@
"JSONSchemaDSL"
],
"portable_root_functions": {
"JSONTests": 198,
"JSONTests": 203,
"JSONSchemaTests": 79,
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 42,
Expand All @@ -89,7 +89,7 @@
"JSONDSLTests": 47
},
"wasi_root_functions": {
"JSONTests": 198,
"JSONTests": 203,
"JSONSchemaTests": 79,
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 42,
Expand Down
211 changes: 211 additions & 0 deletions Sources/JSON/Codec/JSONEnvelope.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
// SPDX-License-Identifier: Apache-2.0
// Original versioned, digest-checked envelope over RFC 8785 canonical JSON.
import Foundation

/// A versioned, self-checking record of one JSON payload, written as RFC 8785 canonical JSON.
///
/// An envelope names what its payload is (``type``) and which schema version of
/// that type it follows (``version``), so a store can migrate old records. Its
/// wire form is one canonical object with six members:
///
/// ```json
/// {"digest":"sha256:…","format":"swift-json-envelope","formatVersion":1,"payload":{…},"type":"mcp.task","version":3}
/// ```
///
/// `digest` is the SHA-256 of the payload's canonical bytes. Decoding recomputes
/// it, so a damaged payload is detected, and equal payloads always have equal
/// digests, so the digest can serve as a content address. Because the whole
/// envelope is canonical, equal envelopes encode to equal bytes.
///
/// Format version 1 is fixed: the member names, the marker and the digest
/// algorithm do not change. A later format would use a new `formatVersion`.
public struct JSONEnvelope: Sendable, Equatable {
/// The format marker, `swift-json-envelope`.
public static let format = "swift-json-envelope"
/// The format version this type reads and writes, `1`.
public static let formatVersion = 1
/// The largest accepted ``type`` length in UTF-8 bytes.
public static let maximumTypeBytes = 256
/// The accepted ``version`` range: the positive integers that binary64 represents exactly.
public static let versions: ClosedRange<Int> = 1...9_007_199_254_740_991

/// The caller's name for the kind of payload, such as `mcp.task`.
public let type: String
/// The schema version of ``type`` that the payload follows.
public let version: Int
/// The enveloped value.
public let payload: JSON

/// Creates an envelope after checking its type and version.
///
/// - Throws: ``JSONEnvelopeFailure/invalidType`` for an empty type or one longer
/// than ``maximumTypeBytes``; ``JSONEnvelopeFailure/invalidVersion`` for a version
/// outside ``versions``.
///
/// - Parameters:
/// - type: A non-empty name for the kind of payload, at most 256 UTF-8 bytes.
/// - version: The payload's schema version, from 1 through 2^53 − 1.
/// - payload: The value to envelope.
public init(type: String, version: Int, payload: JSON) throws {
guard !type.isEmpty, type.utf8.count <= Self.maximumTypeBytes else { throw JSONEnvelopeFailure.invalidType }
guard Self.versions.contains(version) else { throw JSONEnvelopeFailure.invalidVersion }
self.type = type; self.version = version; self.payload = payload
}

/// Writes the envelope's canonical bytes and returns them with the payload digest.
///
/// - Returns: The canonical envelope bytes and the `sha256:` digest of the payload.
/// - Throws: Inexact conversion for a number the policy refuses, resource failure or cancellation.
///
/// - Parameters:
/// - options: Limits and the canonical number policy.
public func encode(options: JSONEnvelopeOptions = .init()) throws -> JSONEnvelopeEncoding {
try encodeWithReport(options: options).result
}
/// Writes the envelope with one cumulative work ledger.
///
/// - Returns: The encoding and the ledger's charged counts.
/// - Throws: Inexact conversion for a number the policy refuses, resource failure or cancellation.
///
/// - Parameters:
/// - options: Limits and the canonical number policy.
public func encodeWithReport(options: JSONEnvelopeOptions = .init()) throws -> JSONOperationReport<JSONEnvelopeEncoding> {
var meter = try JSONWorkMeter(limits: options.limits, domain: .writing)
let writer = JSONCanonicalWriter(limits: options.limits, numberPolicy: options.numberPolicy)
let payloadBytes = try writer.write(payload, meter: &meter)
let digest = try Self.digest(of: payloadBytes, meter: &meter)
try meter.resetValueBounds(); let typeBytes = try writer.write(.string(type), meter: &meter)
try meter.resetValueBounds(); let versionBytes = try writer.write(.number(JSONNumber(integer: version)), meter: &meter)
// Members in RFC 8785 order; every name is ASCII, so UTF-16 order is byte order.
var bytes: [UInt8] = []
func append<C: Collection>(_ part: C, meter: inout JSONWorkMeter) throws where C.Element == UInt8 {
for byte in part { try meter.work(); try meter.retain(1); bytes.append(byte) }
}
func literal(_ text: String, meter: inout JSONWorkMeter) throws {
try meter.charge(.outputBytes, UInt64(text.utf8.count)); try append(text.utf8, meter: &meter)
}
try literal("{\"digest\":\"\(digest)\",\"format\":\"\(Self.format)\",\"formatVersion\":\(Self.formatVersion),\"payload\":", meter: &meter)
try append(payloadBytes, meter: &meter)
try literal(",\"type\":", meter: &meter); try append(typeBytes, meter: &meter)
try literal(",\"version\":", meter: &meter); try append(versionBytes, meter: &meter)
try literal("}", meter: &meter)
try meter.checkpoint()
return .init(result: .init(data: Data(bytes), digest: digest), statistics: meter.statistics())
}

/// Reads an envelope and verifies its format, fields and payload digest.
///
/// The input may be any strict JSON spelling of the envelope object; canonical
/// bytes are not required. Use the same number policy that encoded it.
///
/// - Returns: The checked envelope.
/// - Throws: ``JSONEnvelopeFailure`` for a well-formed JSON value that is not a
/// valid version 1 envelope or whose digest does not match; `JSONFailure` for
/// syntax, a refused number, resource failure; or cancellation.
///
/// - Parameters:
/// - data: The envelope bytes.
/// - options: Limits and the canonical number policy used for the digest.
public static func decode(_ data: Data, options: JSONEnvelopeOptions = .init()) throws -> JSONEnvelope {
try decodeWithReport(data, options: options).result
}
/// Reads and verifies an envelope with one cumulative work ledger.
///
/// See ``decode(_:options:)``.
///
/// - Returns: The checked envelope and the ledger's charged counts.
/// - Throws: ``JSONEnvelopeFailure``, `JSONFailure` or cancellation, as for ``decode(_:options:)``.
///
/// - Parameters:
/// - data: The envelope bytes.
/// - options: Limits and the canonical number policy used for the digest.
public static func decodeWithReport(_ data: Data, options: JSONEnvelopeOptions = .init()) throws -> JSONOperationReport<JSONEnvelope> {
var meter = try JSONWorkMeter(limits: options.limits, domain: .core)
try meter.require(.inputBytes, UInt64(data.count))
var input: [UInt8] = []
for byte in data { try meter.work(); try meter.retain(1); input.append(byte) }
let root = try JSONParser().parse(input, meter: &meter)
try meter.release(UInt64(input.count)); input = []

guard case .object(let object) = root, object.count == 6 else { throw JSONEnvelopeFailure.invalidFormat }
func member(_ name: String, meter: inout JSONWorkMeter) throws -> JSON {
guard let value = try object.lookup(name, meter: &meter) else { throw JSONEnvelopeFailure.invalidFormat }
return value
}
guard case .string(format) = try member("format", meter: &meter) else { throw JSONEnvelopeFailure.invalidFormat }
guard case .number(let formatVersion) = try member("formatVersion", meter: &meter) else { throw JSONEnvelopeFailure.invalidFormat }
guard try integer(formatVersion, meter: &meter) == Self.formatVersion else {
throw JSONEnvelopeFailure.unsupportedFormatVersion
}
guard case .string(let type) = try member("type", meter: &meter) else { throw JSONEnvelopeFailure.invalidType }
guard case .number(let versionNumber) = try member("version", meter: &meter),
let version = try integer(versionNumber, meter: &meter) else { throw JSONEnvelopeFailure.invalidVersion }
guard case .string(let digest) = try member("digest", meter: &meter), isDigest(digest) else { throw JSONEnvelopeFailure.invalidDigest }
let payload = try member("payload", meter: &meter)
let envelope = try JSONEnvelope(type: type, version: version, payload: payload)

try meter.resetValueBounds()
let writer = JSONCanonicalWriter(limits: options.limits, numberPolicy: options.numberPolicy)
let payloadBytes = try writer.write(payload, meter: &meter)
guard try Self.digest(of: payloadBytes, meter: &meter) == digest else { throw JSONEnvelopeFailure.digestMismatch }
try meter.checkpoint()
return .init(result: envelope, statistics: meter.statistics())
}

/// The integer value, or nil when the number is not an integer that fits `Int`.
private static func integer(_ number: JSONNumber, meter: inout JSONWorkMeter) throws -> Int? {
do { return try number.integer(as: Int.self, meter: &meter) }
catch let failure as JSONFailure where failure.category == .conversion { return nil }
}
private static func digest(of bytes: [UInt8], meter: inout JSONWorkMeter) throws -> String {
"sha256:" + (try JSONSHA256Core.hexadecimal(bytes, accounting: &meter))
}
private static func isDigest(_ text: String) -> Bool {
let bytes = Array(text.utf8)
guard bytes.count == 71, bytes.starts(with: Array("sha256:".utf8)) else { return false }
return bytes.dropFirst(7).allSatisfy { (0x30...0x39).contains($0) || (0x61...0x66).contains($0) }
}
}

/// Limits and the number policy for encoding and decoding a ``JSONEnvelope``.
public struct JSONEnvelopeOptions: Sendable {
/// Finite capacities and work quotas applied to each call.
public var limits: JSONLimits
/// The canonical number policy for the payload; decoding must use the policy that encoded it.
public var numberPolicy: JSONCanonicalWriter.NumberPolicy
/// Creates the configuration; finite limits are checked when an operation begins.
///
/// - Parameters:
/// - limits: Finite inclusive capacities and work quotas for each call.
/// - numberPolicy: Refuse (the default) or round numbers that binary64 cannot represent exactly.
public init(limits: JSONLimits = .balanced, numberPolicy: JSONCanonicalWriter.NumberPolicy = .preserving) {
self.limits = limits; self.numberPolicy = numberPolicy
}
}

/// The canonical bytes of a ``JSONEnvelope`` and the digest of its payload.
public struct JSONEnvelopeEncoding: Sendable, Equatable {
/// The canonical envelope bytes.
public let data: Data
/// The payload digest, `sha256:` followed by 64 lowercase hexadecimal digits.
public let digest: String
}

/// Why a well-formed JSON value is not a valid version 1 ``JSONEnvelope``.
///
/// Syntax, resource and number-conversion failures are reported as `JSONFailure`.
/// No case carries input text.
public enum JSONEnvelopeFailure: Error, Sendable, Equatable {
/// The root is not an object with exactly the six envelope members, or the format marker is wrong.
case invalidFormat
/// The format version is not 1.
case unsupportedFormatVersion
/// The type is not a non-empty string of at most 256 UTF-8 bytes.
case invalidType
/// The version is not an integer from 1 through 2^53 − 1.
case invalidVersion
/// The digest is not `sha256:` followed by 64 lowercase hexadecimal digits.
case invalidDigest
/// The payload's canonical digest differs from the recorded digest.
case digestMismatch
}
9 changes: 8 additions & 1 deletion Sources/JSON/Documentation.docc/JSON.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ preserved/normalized decimal spelling. Scalar sorting is deterministic JSON
output, not a cryptographic canonicalization standard. For that, use
``JSONCanonicalWriter``, which writes RFC 8785 (JCS) canonical bytes: UTF-16 key
order, short string escapes and ECMAScript binary64 numbers. By default it refuses
a number whose binary64 spelling would change its exact value.
a number whose binary64 spelling would change its exact value. ``JSONEnvelope``
builds on it: a typed, versioned payload in canonical bytes with a SHA-256
digest that decoding verifies. See <doc:VersionedEnvelopes>.

``JSONValueEncoder`` and ``JSONValueDecoder`` implement native value-based
Codable. Direct JSON/JSONObject round-trips preserve raw scalar keys and numbers.
Expand Down Expand Up @@ -60,6 +62,7 @@ belongs exclusively to each invocation.
- <doc:QuickEntryPoints>
- <doc:CommandLine>
- <doc:ExactPersistence>
- <doc:VersionedEnvelopes>
- <doc:Composition>

### Exact model
Expand All @@ -75,6 +78,10 @@ belongs exclusively to each invocation.
- ``JSONWriter``
- ``JSONWritingOptions``
- ``JSONCanonicalWriter``
- ``JSONEnvelope``
- ``JSONEnvelopeOptions``
- ``JSONEnvelopeEncoding``
- ``JSONEnvelopeFailure``
- ``JSONIndentationUnit``
- ``JSONValueEncoder``
- ``JSONValueDecoder``
Expand Down
61 changes: 61 additions & 0 deletions Sources/JSON/Documentation.docc/VersionedEnvelopes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Versioned envelopes

Store a typed, versioned payload as canonical bytes with a content digest.

## Overview

A ``JSONEnvelope`` wraps one payload with the caller's name for its kind
(``JSONEnvelope/type``) and the schema version of that kind
(``JSONEnvelope/version``). It is meant for records that a store keeps for a
long time, where later code must recognize what a record is, migrate older
versions and notice damage.

```swift
let envelope = try JSONEnvelope(type: "mcp.task", version: 3, payload: task)
let encoding = try envelope.encode()
store.save(encoding.data, key: encoding.digest)

let restored = try JSONEnvelope.decode(store.load(key))
```

### Wire format

Format version 1 is one RFC 8785 canonical object with exactly six members:

```json
{"digest":"sha256:87245ae1c062ec8fd5e2ab1ac60fbc44f6822d6176fe850381db1fd6e97c7810","format":"swift-json-envelope","formatVersion":1,"payload":{"id":7,"state":"done"},"type":"mcp.task","version":3}
```

- `format` is always `swift-json-envelope`; `formatVersion` is always `1`.
- `type` is a non-empty string of at most 256 UTF-8 bytes; `version` is an
integer from 1 through 2^53 − 1.
- `digest` is `sha256:` followed by the lowercase hexadecimal SHA-256 of the
payload's canonical bytes.

Because the whole envelope is canonical, equal envelopes encode to equal bytes.
Equal payloads, compared with the exact model's equality, have equal digests, so
the digest works as a content address independent of the type and version.

### Decoding

``JSONEnvelope/decode(_:options:)`` accepts any strict JSON spelling of the
object, recomputes the payload digest and compares it with the recorded one. A
well-formed value that is not a valid envelope throws a ``JSONEnvelopeFailure``:
a wrong marker or member set, an unsupported format version, an invalid type,
version or digest field, or a digest mismatch. Syntax, resource and number
failures stay `JSONFailure`.

### Numbers

The payload is canonicalized with ``JSONCanonicalWriter``. By default
(``JSONCanonicalWriter/NumberPolicy/preserving``) a number that binary64 cannot
spell exactly, such as `0.10000000000000001`, is refused rather than changed.
With ``JSONCanonicalWriter/NumberPolicy/rounding`` such numbers are rounded
before hashing, and decoding must use the same policy.

## Topics

- ``JSONEnvelope``
- ``JSONEnvelopeOptions``
- ``JSONEnvelopeEncoding``
- ``JSONEnvelopeFailure``
2 changes: 1 addition & 1 deletion Sources/JSONCLIEngine/CLIBuildIdentity.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
// Generated by Scripts/generate_cli_identity.py; excludes this file.
// SHA-256 of the ordered Package.swift + Sources content manifest (DocC catalogs excluded), not a Git commit.
package enum CLIBuildIdentity {
package static let revision = "sha256:c0c4442af4a6e52b5e232fc41405acbf92616b366832ae0df0bfb843e18a3773"
package static let revision = "sha256:fd026d72a5ef7b9f545d4ab4b08edd1460af5d0630551034a16f066824e1fa39"
package static let kind = "source-sha256"
}
Loading
Loading