Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Changelog

## Unreleased — 1.10.0 exact interchange (in progress)

- Add `JSONCanonicalWriter`, which writes RFC 8785 JSON Canonicalization Scheme
(JCS) bytes: compact output, keys ordered by UTF-16 code units, short string
escapes and ECMAScript spelling of binary64 numbers. Equal values produce equal
bytes for hashing, signing and content addressing. `NumberPolicy.preserving`
(the default) refuses a number whose binary64 spelling would denote a different
exact decimal; `.rounding` rounds to the nearest binary64 value as RFC 8785 does.
Tests cover the RFC's Appendix B number vectors, its sample document and its
key-sorting example. The writer does not hash or sign. The existing writer and
its scalar key order are unchanged.

## Unreleased — 1.9.1 hygiene

- Add the `platform-floor` native CI lane. It checks the macOS 15 / iOS 18
Expand Down
9 changes: 9 additions & 0 deletions Documentation/Architecture/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,8 @@ or shape-inferencing result builders. See [numbers and Unicode](NumbersAndUnicod
| `JSONWriter.write(_:)` | Owned Data containing complete strict UTF-8 JSON |
| `JSONWriter.writeString(_:)` | Owned String with its additional retention charged |
| `JSONWriter.writeWithReport(_:)` | JSONOperationReport<Data> |
| `JSONCanonicalWriter.write(_:)` | Owned Data containing RFC 8785 canonical UTF-8 JSON |
| `JSONCanonicalWriter.writeWithReport(_:)` | JSONOperationReport<Data> |

Parsing accepts exactly one root followed by JSON whitespace and EOF. It rejects
BOM, malformed UTF-8, unpaired surrogate escapes, raw controls, malformed numbers,
Expand All @@ -127,6 +129,13 @@ or scalar key order, and normalized or preserved number text. Preserved lexemes
were admitted with their grammar and identity. Sorting makes no cryptographic
canonicalization claim.

JSONCanonicalWriter writes RFC 8785 JSON Canonicalization Scheme bytes: compact,
keys ordered by UTF-16 code units, short string escapes and ECMAScript binary64
numbers. Its default `.preserving` number policy refuses, with an inexact-conversion
failure, a number whose binary64 spelling would denote a different exact decimal;
`.rounding` follows RFC 8785 and rounds. It produces bytes only, with no hashing,
signing or exported cryptography API.

JSONDocument owns optional original bytes and bounded value/member-key spans.
Spans are half-open zero-based byte ranges; diagnostic line/UTF-8 byte column and
physical record coordinates are one-based when present. Typed source paths keep
Expand Down
6 changes: 3 additions & 3 deletions Documentation/Qualification/TestDiscovery.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"schema_version": 2,
"root_functions": {
"JSONTests": 190,
"JSONTests": 198,
"JSONSchemaTests": 79,
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 36,
Expand Down Expand Up @@ -73,7 +73,7 @@
"JSONSchemaDSL"
],
"portable_root_functions": {
"JSONTests": 190,
"JSONTests": 198,
"JSONSchemaTests": 79,
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 36,
Expand All @@ -89,7 +89,7 @@
"JSONDSLTests": 47
},
"wasi_root_functions": {
"JSONTests": 190,
"JSONTests": 198,
"JSONSchemaTests": 79,
"JSONRegexRuntimeTests": 16,
"JSONStreamingTests": 36,
Expand Down
80 changes: 80 additions & 0 deletions Sources/JSON/Codec/JSONCanonicalWriter.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// SPDX-License-Identifier: Apache-2.0
// Original implementation of RFC 8785, JSON Canonicalization Scheme (JCS).
import Foundation

/// A writer for RFC 8785 canonical JSON, the JSON Canonicalization Scheme (JCS).
///
/// The output is compact UTF-8 with object members ordered by the UTF-16 code
/// units of their keys, strings using the short escapes `\b`, `\t`, `\n`, `\f`,
/// `\r`, `\"` and `\\` (and `\u00xx` for other control characters), and numbers
/// spelled as ECMAScript formats binary64 values. Equal values produce equal
/// bytes, so the result can be hashed, signed or used as a content address.
///
/// RFC 8785 treats every number as an IEEE 754 binary64 value. A ``JSONNumber``
/// is an exact decimal, so the ``NumberPolicy`` decides what happens when the
/// binary64 spelling would denote a different value. The default,
/// ``NumberPolicy/preserving``, refuses such a number instead of changing it.
///
/// This type produces bytes only. It performs no hashing or signing, and its key
/// order differs from ``JSONWritingOptions/KeyOrder/scalar`` for keys that mix
/// supplementary-plane characters with characters from U+E000 through U+FFFF.
public struct JSONCanonicalWriter: Sendable {
/// How a number that binary64 cannot represent exactly is handled.
public enum NumberPolicy: Sendable {
/// Refuse a number whose canonical binary64 spelling would denote a
/// different exact decimal, with an inexact-conversion failure.
///
/// A short decimal such as `0.1` is accepted, because its canonical
/// spelling `0.1` denotes the same exact value. `9007199254740993` and
/// `0.10000000000000001` are refused.
case preserving
/// Round to the nearest binary64 value before formatting, as RFC 8785
/// does for I-JSON input. Different exact decimals can then produce the
/// same bytes. Values outside the finite binary64 range still fail.
case rounding
}
/// Finite capacities and work quotas applied to each call.
public let limits: JSONLimits
/// The treatment of numbers that binary64 cannot represent exactly.
public let numberPolicy: NumberPolicy
/// Creates an immutable canonical writer. Limits are checked when writing begins.
///
/// - Parameters:
/// - limits: Finite inclusive capacities and work quotas for each call.
/// - numberPolicy: Refuse (the default) or round numbers that binary64 cannot represent exactly.
public init(limits: JSONLimits = .balanced, numberPolicy: NumberPolicy = .preserving) {
self.limits = limits; self.numberPolicy = numberPolicy
}
/// Writes the RFC 8785 canonical bytes of a complete value.
///
/// - Returns: Owned canonical UTF-8 bytes.
/// - Throws: Inexact conversion for a number the policy refuses or for a value
/// outside the finite binary64 range; resource failure or cancellation.
///
/// - Parameters:
/// - value: The exact root to admit and canonicalize.
public func write(_ value: JSON) throws -> Data { try writeWithReport(value).result }
/// Writes RFC 8785 canonical bytes with successful operation accounting.
///
/// - Returns: Owned canonical bytes and counts; output bytes are charged before growth.
/// - Throws: Inexact conversion for a number the policy refuses or for a value
/// outside the finite binary64 range; resource failure or cancellation.
///
/// - Parameters:
/// - value: The exact root to admit and canonicalize.
public func writeWithReport(_ value: JSON) throws -> JSONOperationReport<Data> {
var meter = try JSONWorkMeter(limits: limits, domain: .writing)
let bytes = try write(value, meter: &meter)
for _ in bytes { try meter.work(); try meter.retain(1) }
return .init(result: Data(bytes), statistics: meter.statistics())
}
package func write(_ value: JSON, meter: inout JSONWorkMeter) throws -> [UInt8] {
try JSONAdmission.value(value, meter: &meter)
var cursor = JSONWritingCursor(value: value, limits: limits, canonical: numberPolicy)
var bytes: [UInt8] = []
while let byte = try cursor.next(meter: &meter) {
try meter.charge(.outputBytes); try meter.retain(1); bytes.append(byte)
}
return bytes
}
}
36 changes: 31 additions & 5 deletions Sources/JSON/Codec/JSONWriter.swift
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,15 @@ package struct JSONWritingCursor {
}
private var stack: [Step]
private let options: JSONWritingOptions
/// RFC 8785 mode: UTF-16 key order, short escapes and ECMAScript numbers.
private let canonical: JSONCanonicalWriter.NumberPolicy?
private var initialized = false
private var beginsValue = false
package init(value: JSON, options: JSONWritingOptions) { stack = [.value(value, 0)]; self.options = options }
package init(value: JSON, options: JSONWritingOptions) { stack = [.value(value, 0)]; self.options = options; canonical = nil }
package init(value: JSON, limits: JSONLimits, canonical policy: JSONCanonicalWriter.NumberPolicy) {
stack = [.value(value, 0)]; options = .init(limits: limits); canonical = policy
}
private var sortsKeys: Bool { canonical != nil || options.keyOrder == .scalar }
private mutating func push(_ step: Step, meter: inout JSONWorkMeter) throws {
try meter.retain(JSONLogicalWeights.frame); stack.append(step)
}
Expand All @@ -103,8 +109,21 @@ package struct JSONWritingCursor {
if x == nil { return false }
}
}
/// RFC 8785 section 3.2.3 orders keys by their UTF-16 code units.
private func utf16Less(_ a: JSONMember, _ b: JSONMember, meter: inout JSONWorkMeter) throws -> Bool {
var l = a.key.string.utf16.makeIterator(), r = b.key.string.utf16.makeIterator()
while true {
try meter.work()
let x = l.next(), y = r.next()
if x != y { if let x, let y { return x < y }; return x == nil }
if x == nil { return false }
}
}
private func less(_ a: JSONMember, _ b: JSONMember, meter: inout JSONWorkMeter) throws -> Bool {
canonical == nil ? try scalarLess(a, b, meter: &meter) : try utf16Less(a, b, meter: &meter)
}
private func ordered(_ object: JSONObject, meter: inout JSONWorkMeter) throws -> [JSONMember] {
if options.keyOrder == .insertion { return object.members }
if !sortsKeys { return object.members }
var values: [JSONMember] = [], scratch: [JSONMember] = []
for member in object {
try meter.work(); try meter.retain(JSONLogicalWeights.member * 2)
Expand All @@ -118,7 +137,7 @@ package struct JSONWritingCursor {
var left = start, right = middle
for target in start..<end {
try meter.work()
if try left < middle && (right == end || !scalarLess(values[right], values[left], meter: &meter)) {
if try left < middle && (right == end || !less(values[right], values[left], meter: &meter)) {
scratch[target] = values[left]; left += 1
} else { scratch[target] = values[right]; right += 1 }
}
Expand Down Expand Up @@ -157,6 +176,10 @@ package struct JSONWritingCursor {
try push(.escaped(iterator, kind), meter: &meter)
switch byte {
case 34, 92: try push(.byte(byte, kind), meter: &meter); return token(92, kind)
case 8 where canonical != nil, 9 where canonical != nil, 10 where canonical != nil,
12 where canonical != nil, 13 where canonical != nil:
let short: UInt8 = switch byte { case 8: 98; case 9: 116; case 10: 110; case 12: 102; default: 114 }
try push(.byte(short, kind), meter: &meter); return token(92, kind)
case 0...31:
let hex: [UInt8] = Array("0123456789abcdef".utf8)
try push(.byte(hex[Int(byte & 15)], kind), meter: &meter); try push(.byte(hex[Int(byte >> 4)], kind), meter: &meter)
Expand All @@ -171,7 +194,10 @@ package struct JSONWritingCursor {
case .boolean(let boolean): try push(.text((boolean ? "true" : "false").utf8.makeIterator(), .boolean), meter: &meter)
case .string(let text): try quoted(text, meter: &meter)
case .number(let number):
if options.numberStyle == .preserved, let lexeme = number.originalLexeme { try push(.text(lexeme.utf8.makeIterator(), .number), meter: &meter) }
if let policy = canonical {
let text = try number.canonicalText(policy: policy, meter: &meter)
try push(.text(text.utf8.makeIterator(), .number), meter: &meter)
} else if options.numberStyle == .preserved, let lexeme = number.originalLexeme { try push(.text(lexeme.utf8.makeIterator(), .number), meter: &meter) }
else {
let c = number.components
if c.exponent != 0 { try push(.text(String(c.exponent).utf8.makeIterator(), .number), meter: &meter); try push(.byte(101, .number), meter: &meter) }
Expand Down Expand Up @@ -199,7 +225,7 @@ package struct JSONWritingCursor {
if index > 0 { try push(.byte(44), meter: &meter) }
}
try push(.byte(123), meter: &meter)
if options.keyOrder == .scalar { try meter.release(UInt64(members.count) * JSONLogicalWeights.member) }
if sortsKeys { try meter.release(UInt64(members.count) * JSONLogicalWeights.member) }
}
}
}
Expand Down
6 changes: 5 additions & 1 deletion Sources/JSON/Documentation.docc/JSON.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ and uses order-independent equality. A duplicate decoded key is a syntax error.
JSONStreaming. Parsing requires one complete root, trailing whitespace and EOF.
The writer supports compact/pretty output, insertion/scalar key order and
preserved/normalized decimal spelling. Scalar sorting is deterministic JSON
output, not a cryptographic canonicalization standard.
output, not a cryptographic canonicalization standard. For that, use
``JSONCanonicalWriter``, which writes RFC 8785 (JCS) canonical bytes: UTF-16 key
order, short string escapes and ECMAScript binary64 numbers. By default it refuses
a number whose binary64 spelling would change its exact value.

``JSONValueEncoder`` and ``JSONValueDecoder`` implement native value-based
Codable. Direct JSON/JSONObject round-trips preserve raw scalar keys and numbers.
Expand Down Expand Up @@ -71,6 +74,7 @@ belongs exclusively to each invocation.
- ``JSONParser``
- ``JSONWriter``
- ``JSONWritingOptions``
- ``JSONCanonicalWriter``
- ``JSONIndentationUnit``
- ``JSONValueEncoder``
- ``JSONValueDecoder``
Expand Down
75 changes: 75 additions & 0 deletions Sources/JSON/Number/JSONNumberCanonical.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
// SPDX-License-Identifier: Apache-2.0
// Original implementation of the RFC 8785 section 3.2.2.3 number serialization,
// which is ECMAScript Number::toString applied to a binary64 value.

extension JSONNumber {
/// The ECMAScript spelling of this number's binary64 value.
///
/// Swift's `description` of a finite Double supplies the shortest digits that
/// round-trip, choosing the closest such digits, which is the digit string
/// ECMAScript also requires. Only the layout around those digits is rebuilt here.
package func canonicalText(policy: JSONCanonicalWriter.NumberPolicy, meter: inout JSONWorkMeter) throws -> String {
try meter.work(subsystem: .numericWorkUnits)
if isZero { return "0" }
let value = try doubleValue(policy: .rounded, meter: &meter, chargesOutput: false)
// A finite Double's description is at most 24 ASCII bytes; reserve a fixed
// bound around the synchronous foreign rendering.
let foreign: UInt64 = 32
try meter.retain(foreign); try meter.checkpoint()
let shortest = value.magnitude.description
try meter.checkpoint()
var digits: [UInt8] = [], point: Int? = nil, exponent = 0, exponentNegative = false, inExponent = false
try meter.retain(foreign)
for byte in shortest.utf8 {
try meter.work(subsystem: .numericWorkUnits)
switch byte {
case 48...57 where inExponent: exponent = exponent * 10 + Int(byte - 48)
case 48...57: digits.append(byte - 48)
case 46: point = digits.count
case 101, 69: inExponent = true
case 45: exponentNegative = true
case 43: break
default: throw JSONFailure(.conversion, code: .inexactConversion)
}
}
var position = (point ?? digits.count) + (exponentNegative ? -exponent : exponent)
while digits.count > 1 && digits[0] == 0 { try meter.work(subsystem: .numericWorkUnits); digits.removeFirst(); position -= 1 }
while digits.count > 1 && digits[digits.count - 1] == 0 { try meter.work(subsystem: .numericWorkUnits); digits.removeLast() }
var output: [UInt8] = []
func append(_ byte: UInt8, meter: inout JSONWorkMeter) throws {
try meter.work(subsystem: .numericWorkUnits); try meter.retain(1); output.append(byte)
}
func appendDigits(_ range: Range<Int>, meter: inout JSONWorkMeter) throws {
for index in range { try append(digits[index] + 48, meter: &meter) }
}
if digits == [0] {
try append(48, meter: &meter)
} else {
if value < 0 { try append(45, meter: &meter) }
let k = digits.count, n = position
if k <= n && n <= 21 {
try appendDigits(0..<k, meter: &meter)
for _ in 0..<(n - k) { try append(48, meter: &meter) }
} else if 0 < n && n <= 21 {
try appendDigits(0..<n, meter: &meter); try append(46, meter: &meter); try appendDigits(n..<k, meter: &meter)
} else if -6 < n && n <= 0 {
try append(48, meter: &meter); try append(46, meter: &meter)
for _ in 0..<(-n) { try append(48, meter: &meter) }
try appendDigits(0..<k, meter: &meter)
} else {
try append(digits[0] + 48, meter: &meter)
if k > 1 { try append(46, meter: &meter); try appendDigits(1..<k, meter: &meter) }
try append(101, meter: &meter); try append(n - 1 < 0 ? 45 : 43, meter: &meter)
for byte in String((n - 1).magnitude).utf8 { try append(byte, meter: &meter) }
}
}
try meter.release(foreign * 2)
let text = String(decoding: output, as: UTF8.self)
if policy == .preserving {
let spelled = try Self.parse(output, meter: &meter)
guard spelled == self else { throw JSONFailure(.conversion, code: .inexactConversion) }
}
try meter.release(UInt64(output.count))
return text
}
}
9 changes: 5 additions & 4 deletions Sources/JSON/Number/JSONNumberConversions.swift
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@ extension JSONNumber {
/// digits and a nonzero sticky digit preserves which side of every midpoint
/// the original exact decimal occupies. Binary32's midpoint bound is smaller.
/// The system converter therefore sees <=822 bytes, independent of input size.
package func binaryConversionText(meter: inout JSONWorkMeter) throws -> String {
/// Canonical writing passes `chargesOutput: false`: this text is scratch, not output.
package func binaryConversionText(meter: inout JSONWorkMeter, chargesOutput: Bool = true) throws -> String {
let c = components
let kept = min(800,c.digits.count), shortened = c.digits.count > kept
let coefficientCount = kept + (shortened ? 1 : 0)
Expand All @@ -101,7 +102,7 @@ extension JSONNumber {
guard !overflow else { throw JSONFailure(.resource,code: .counterOverflow) }
var output: [UInt8] = []
func append(_ byte: UInt8,meter: inout JSONWorkMeter) throws {
try meter.work(subsystem: .numericWorkUnits); try meter.charge(.outputBytes)
try meter.work(subsystem: .numericWorkUnits); if chargesOutput { try meter.charge(.outputBytes) }
try meter.retain(1); output.append(byte)
}
if c.negative { try append(45,meter: &meter) }
Expand Down Expand Up @@ -149,8 +150,8 @@ extension JSONNumber {
var meter = try JSONWorkMeter(limits: limits, domain: .core)
return try doubleValue(policy: policy, meter: &meter)
}
package func doubleValue(policy: JSONNumberConversionPolicy, meter: inout JSONWorkMeter) throws -> Double {
let text = try binaryConversionText(meter: &meter)
package func doubleValue(policy: JSONNumberConversionPolicy, meter: inout JSONWorkMeter, chargesOutput: Bool = true) throws -> Double {
let text = try binaryConversionText(meter: &meter, chargesOutput: chargesOutput)
try meter.checkpoint()
guard let value = Double(text), value.isFinite else { throw JSONFailure(.conversion, code: .inexactConversion) }
try meter.checkpoint()
Expand Down
Loading
Loading