Skip to content

Finish new format design and update ts encoder#2

Open
creationix wants to merge 3 commits into
mainfrom
rx-1.0
Open

Finish new format design and update ts encoder#2
creationix wants to merge 3 commits into
mainfrom
rx-1.0

Conversation

@creationix
Copy link
Copy Markdown
Owner

No description provided.

Copilot AI review requested due to automatic review settings May 26, 2026 19:44
- New decoder for paired-delimiter spec: containers without an index decode
  eagerly to native objects/arrays; containers with an index return a lazy
  Proxy that defers child decoding until property access.
- Add depth-based index thresholds (minIndexDepth / maxIndexDepth) so callers
  can force-on / force-off random access at specific nesting levels. Replaces
  the entry-count threshold with a body-byte threshold for the heuristic.
- Add bytes tag `@` (URL-safe b64 body) and tighten the chain semantics:
  any bytes segment promotes the chain result to bytes.
- Schema sharing now requires shape count > 1 in a pre-scan pass; singleton
  shapes use inline keys, large objects keep their index for O(log n) lookup.
- Drop rxb entirely (binary variant + reader + tests + CLI integration).
- Trim rx.ts public surface to {stringify, encode, tune, EncodeOptions,
  StringifyOptions, TuneOptions, Refs}. EncodeOptions extends TuneOptions
  so the same six knobs work as one-off overrides and as tune() defaults.
  Removed the export-let global constants (incompatible with isolatedModules).
- Add zig-cli prototype: JSON → RX converter with a generic Source interface
  (tape source today, host-value source later).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the RX text format encoder and documentation toward a new “paired delimiter” design (containers as []/{}/<>, schema as .), and adds new encoder tuning controls for index emission.

Changes:

  • Switch container and chain encoding to paired delimiters and revise index emission to be byte-size/depth driven.
  • Add schema-sharing prescan counts and emit inline schema + schema pointers for repeated object shapes.
  • Extend the CLI and spec docs with the new index depth knobs and updated format description.

Reviewed changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.

File Description
rx.ts Implements the new paired-delimiter encoder, depth-based index forcing, and schema sharing changes.
rx-cli.ts Adds CLI flags for index depth tuning and adjusts convert --to rx to use stringify() with new options.
docs/rx-format.md Rewrites the RX format specification to describe the new paired-delimiter model, schema, and indexes.
Comments suppressed due to low confidence (1)

rx-cli.ts:634

  • rx convert --to rx now uses stringify() (new paired-delimiter encoding), but .rx inputs are still parsed via open() from rx-read.ts, which currently only understands the legacy :/; container tags. This means the CLI can generate .rx files it cannot read back (and other subcommands reading .rx will fail). Update the reader to the new format (or keep convert emitting the legacy format until the reader is updated).
function applyPath(value: unknown, segments: string[]): unknown {
	let current: unknown = value;
	const trail: string[] = [];
	for (const seg of segments) {
		if (Array.isArray(current)) {
			const idx = /^\d+$/.test(seg) ? parseInt(seg, 10) : NaN;
			if (!Number.isInteger(idx)) {
				fail("show", `path [${[...trail, seg].join(", ")}] — '${seg}' is not a numeric index (array at [${trail.join(", ")}] has length ${current.length})`);
			}
			if (idx < 0 || idx >= current.length) {
				fail("show", `path [${[...trail, seg].join(", ")}] — index ${idx} out of range in ${current.length}-element array`);
			}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread rx.ts
Comment on lines +53 to +57
maxIndexDepth: Infinity,
stringChainThreshold: 24,
stringChainDelimiter: "/.",
dedupComplexityLimit: 32,
};
Comment thread docs/rx-format.md
Comment on lines +26 to +38
| Tag | Name | Layout | Varint meaning |
|---------|---------|----------------------|-------------------------|
| `+` | Integer | `+[varint]` | zigzag signed |
| `*` | Decimal | `[+base]*[varint]` | zigzag exponent |
| `,` | String | `[utf8],[varint]` | byte length |
| `'` | Ref | `'[name]` | b64 name (not a number) |
| `^` | Pointer | `^[varint]` | byte offset delta |
| `[` `]` | Array | `[children]` | — (paired) |
| `{` `}` | Object | `{children}` | — (paired) |
| `<` `>` | Chain | `<segments>` | — (paired) |
| `.` | Schema | `[keys].[varint]` | byte length |
| `#` | Index | `[entries]#[varint]` | packed count+width |
| `@` | Bytes | `[b64body]@[varint]` | byte length of body |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants