Skip to content

Compact REPL syntax discovery and include named structural references - #1229

Merged
taras merged 4 commits into
mainfrom
codex/repl-syntax-index-1178
Nov 11, 2026
Merged

taras merged 4 commits into
mainfrom
codex/repl-syntax-index-1178

Conversation

@taras

@taras taras commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Why

Bare REPL.Syntax sends a full reference when Sidekick needs the available names first. A retained 85-name fixture produces 82,885 characters before the agent chooses what to inspect. Named inspection also refuses structural constructs that appear in the vocabulary.

What changes

Before: discovery includes full contracts, and named inspection rejects engine constructs such as If and installed parent/child syntax.

After: bare REPL.Syntax returns categories, exact names, short purposes and an actionable named lookup. The same fixture produces 6,105 characters with all 85 names retained. Named inspection returns complete ordinary, engine and installed structural references.

Named engine availability also follows the real generated context. A narrowed Evaluate fragment can describe If as available and then execute it; installed constructs remain unavailable there. Root-only and parent/context rules remain qualified. Bare Syntax, CLI Markdown and version-1 JSON keep their existing output.

How it works

The compact index projects the existing vocabulary. Named references retain shared selection, rendering and full-identity checks. Generated fragment/root owners supply immutable engine-reference data through the existing internal SyntaxReference.available seam; only a named lookup uses it. Inspection shares the owner metadata projection and does not invoke installed handlers or broaden execution admission.

Review guide

Start with repl-profile.ts, the Syntax reference/selection/renderers and the generated fragment/root owners. Check lookup-local specialization, omitted versus empty data, middleware/read-once behavior, installed identity, fresh profile/include/Plugin selection and ordinary-component precedence. The named structural-reference amendment is explicitly accepted in #1178.

How to verify it

  • Original focused evidence: 52 tests / 274 steps cover compact completeness, purposes, full named references, refusal, fresh reads and installed contracts. Both affected journey files pass under prepared Bun: 98 tests / 1,104 assertions.
  • Correction 2 focused Deno evidence: 49 tests / 276 steps. Real protected Evaluate, producer projection and generated roots now exercise the independent reviewer's counterexample. Immutable specialization, middleware, sibling/scope restoration, installed negatives and pinned base output are covered.
  • Planned typed consumers and docs validation pass. A fresh corrected-source build passes; the exact compiled regression selects one test / one step and passes against source and the rebuilt binary outside the checkout. Candidate hashes match signed head 50e26a423efdca71d42deaf57ddcc3a47a98a8b8.

Risks and limitations

Current signed head: 50e26a423efdca71d42deaf57ddcc3a47a98a8b8; feature base: 2bc5ef29b2966af27c7ba292dd42a759f6f69f79. Original Architect and original Planner feedback are PASS on the correction. Independent Codex Astra review is also PASS against all four acceptance clauses. The retained Fable session could not resume because its saved full model ID is absent from the offered catalog; its refusal and the normal fallback transition are preserved. Every corrected-head technical check passes after one unchanged-head retry of each recorded Deno DR6 and Delivery fallback failure. First and passing logs remain in #1200 and #1176; neither flake is claimed fixed. Main health remains the delivery blocker.

The original Fable review on parent 7c3f0574 is retained as CHANGES_REQUIRED. Correction 2 repairs its real narrowed-availability finding. An optional architecture diagnostic exhausted its response allowance without a canonical result; it remains failed and is not approval. The original Planner independently accepted the finite repair plan, followed by fresh corrected-source feedback. Two of the existing three source-correction slots are used; no allowance was reset.

Earlier blocked results and first failures remain intact: broad selection stopped at registry DNS/cache metadata; build preparation later lacked the global Babel cache; compilation then encountered sandbox DNS while downloading Deno's runtime. Authorized preparation restored the exact frozen inputs, and the same original Codex Implementor completed the successful build/test. The incomplete broad run is not counted as passing evidence. No dependency version, runtime or Implementor session was substituted.

Parent CI's Node History final-selection failure is recorded in #1054 with its first log retained; no cause fix is claimed. Character measurements describe one fixture, with no token, latency or model-efficiency claim. Earlier Main-health failures remain recorded under #1224. Main has advanced to c900ebfd through #1230; its new CI run is in progress, so current Main health is pending. This draft remains unmerged.

Scope confirmation

  • Every changed file supports compact discovery, named references or their tests and documentation.
  • Selection, canonical output and execution boundaries remain covered.
  • Unrelated work is preserved; no dependency changes are included.
  • Original results, candidate identities, finite allowances and first failures are retained.

Closes #1178.

@taras

taras commented Nov 10, 2026

Copy link
Copy Markdown
Owner Author

Spec review at 50e26a423efdca71d42deaf57ddcc3a47a98a8b8, against #1178 and our requirements for one authoritative home and a contract sufficient to reimplement.

  1. Reconcile generated named availability with the existing reference contract. The new paragraph includes supported engine syntax in named lookup while leaving bare generated structural symbols empty. But lines 3088–3099 still describe two inputs, lines 3167–3175 derive named availability from effective symbols, and SYN25b at line 13694 says the narrowed reference adds nothing. These permit opposite answers for named If in a fragment. Preserve the accepted bare-output behavior, but state the distinction between its presentation and the named availability input, including how symbols middleware affects each. Update the conformance text with explicit fragment/root positives and unavailable installed/context-dependent forms.

  2. Keep placement rules in the generated-composition owner. The new Syntax paragraph defines fragment/root exclusions, while §5.3.3 still broadly declares every built-in construct available. Put the qualified placement rule in that owner and link to it from Syntax. This should explain reference availability without turning documentation into an execution grant.

  3. Complete the REPL index contract. Lines 3218–3224 promise bounded purposes but leave their derivation, bound and missing-description/origin-only fallback unspecified. State the observable requirements, or link their owner. Also retain an explicit link to the shared full named-reference/rendering contract: replacing “selection and renderer” with “structured symbols and selection” leaves that relationship implicit.

The preservation of ordinary bare Syntax/CLI/version-1 JSON, full identity checks, atomic unknown-name refusal and non-executing inspection is clear. Any additional policy choice should be recorded explicitly rather than inferred from current code. This feedback concerns the specification, not merge readiness.

taras added a commit that referenced this pull request Nov 10, 2026
Address the maintainer review on #1229. Put generated placement in the composition contract. Separate bare presentation from named availability, including symbols middleware and fragment and root conformance. Specify index purpose derivation, fallback and bound, and link the shared named renderer.

Evidence: syntax API, Syntax component, Evaluate, generated root, Sidekick controls and syntax CLI suites passed (50 tests, 320 steps).
@taras

taras commented Nov 10, 2026

Copy link
Copy Markdown
Owner Author

Addressed the spec review in 465649580091945755553b79745dd14d2f283920.

  1. The Syntax contract separates bare presentation, enclosing named selection, and named availability. It explains which symbols middleware sees in each read and retains full identity comparison. SYN47 covers fragment/root positives, context-dependent exclusions, unavailable installed forms, and bare output before and after a named read.
  2. The generated-composition section owns placement. A fragment can use engine constructs such as If and Loop where valid, a root also supplies Output under its ordinary placement rules, and neither supplies Content, Return, or installed structural constructs. Named Syntax links to that contract and remains descriptive.
  3. The REPL index specifies where the short purpose comes from, how it handles missing descriptions and origin-only TypeScript, and its 160 UTF-16 code-unit limit before Markdown escaping. Named REPL lookup links to core's full reference selection and renderer.

The update changes only the two owning specs. The Syntax API/component, Evaluate, generated-root, Sidekick controls and syntax CLI suites passed (50 tests, 320 steps). The original Fable session could not resume. The existing independent Codex Astra reviewer passed the exact update under the recorded fallback policy. CI is tracked separately on the updated head.

@taras
taras marked this pull request as ready for review November 10, 2026 22:33
taras added a commit that referenced this pull request Nov 10, 2026
Address the maintainer review on #1229. Put generated placement in the composition contract. Separate bare presentation from named availability, including symbols middleware and fragment and root conformance. Specify index purpose derivation, fallback and bound, and link the shared named renderer.

Evidence: syntax API, Syntax component, Evaluate, generated root, Sidekick controls and syntax CLI suites passed (50 tests, 320 steps).
@taras
taras force-pushed the codex/repl-syntax-index-1178 branch from 4656495 to 796a345 Compare November 10, 2026 22:34
taras added a commit that referenced this pull request Nov 11, 2026
Address the maintainer review on #1229. Put generated placement in the composition contract. Separate bare presentation from named availability, including symbols middleware and fragment and root conformance. Specify index purpose derivation, fallback and bound, and link the shared named renderer.

Evidence: syntax API, Syntax component, Evaluate, generated root, Sidekick controls and syntax CLI suites passed (50 tests, 320 steps).
@taras
taras force-pushed the codex/repl-syntax-index-1178 branch from 796a345 to cb6853e Compare November 11, 2026 01:04
Address the maintainer review on #1229. Put generated placement in the composition contract. Separate bare presentation from named availability, including symbols middleware and fragment and root conformance. Specify index purpose derivation, fallback and bound, and link the shared named renderer.

Evidence: syntax API, Syntax component, Evaluate, generated root, Sidekick controls and syntax CLI suites passed (50 tests, 320 steps).
@taras
taras force-pushed the codex/repl-syntax-index-1178 branch from cb6853e to cbe5021 Compare November 11, 2026 04:02
@taras
taras merged commit ce8bb81 into main Nov 11, 2026
59 of 62 checks passed
@taras
taras deleted the codex/repl-syntax-index-1178 branch November 11, 2026 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make REPL.Syntax a compact index with named structural lookups

1 participant