Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .changeset/docs-gen-nested-brace-escape.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
"@objectstack/spec": patch
---

fix(spec): docs-gen no longer cuts a nested `{…}` / `<…>` in half (#5452)

The reference-docs generator wraps a delimited fragment of `.describe()` prose
in an inline-code span so MDX renders it literally instead of parsing it as a
JS expression or a JSX tag. It located the fragment's closing delimiter with
`indexOf` — the **first** closer, not the **matching** one — so any nested pair
was wrapped only up to its inner closer and the outer one fell outside the
span.

The published symptom: `{{var}}` in a description was emitted as
`` `{{var}` `` followed by a stray `}`. Readers saw `{{var}` plus an orphan
brace on precisely the rows that teach template-variable syntax, where the
paired double brace *is* the thing being documented. Nesting is not an exotic
input in this corpus — template interpolation and filter-map examples both
produce it.

The matcher now counts nesting depth, so the whole pair lands inside one code
span. Five rows across four regenerated reference pages change:

- `references/ai/model-registry.mdx` — `PromptTemplate.system` / `.user`,
both `{{var}}`
- `references/automation/flow.mdx` — `flow.nodes[].outputSchema`,
`{{nodeId.field}}`
- `references/ai/solution-blueprint.mdx` — the roll-up `filter` example,
`{ status: { $in: [...] } }`
- `references/api/analytics.mdx` — the retired `query` envelope,
`{ cube, query: {...} }`

The issue reported three; the last two were cut in the same place but start
with a single brace, so the `` `{{ `` grep that found the others could never
have seen them.

Unchanged: a single `{…}` pair, a `{<id>}` nest, and a lone unmatched `<` / `{`
(entity-escaped, e.g. a SemVer range `>=4.0 <5`) all escape exactly as before.
No package export or runtime behaviour changes — the fix is in
`scripts/build-docs.ts`, whose escaping moved to `scripts/lib/escape-mdx.ts` so
it can be pinned directly rather than by grepping emitted `.mdx`.
4 changes: 2 additions & 2 deletions content/docs/references/ai/model-registry.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -168,8 +168,8 @@ const result = ModelCapabilitySchema.parse(data);
| **id** | `string` | ✅ | Unique template identifier |
| **name** | `string` | ✅ | Template name (snake_case) |
| **label** | `string` | ✅ | Display name |
| **system** | `string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | optional | System prompt — supports `{{var}`} interpolation |
| **user** | `string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | ✅ | User prompt template — supports `{{var}`} interpolation |
| **system** | `string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | optional | System prompt — supports `{{var}}` interpolation |
| **user** | `string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | ✅ | User prompt template — supports `{{var}}` interpolation |
| **assistant** | `string` | optional | Assistant message prefix |
| **variables** | `{ name: string; type?: Enum<'string' \| 'number' \| 'boolean' \| 'object' \| 'array'>; required?: boolean; defaultValue?: any; … }[]` | optional | Template variables |
| **modelId** | `string` | optional | Recommended model ID |
Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/ai/solution-blueprint.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ const result = BlueprintAppSchema.parse(data);
| **field** | `string` | optional | Numeric field on the CHILD object to aggregate. Ignored for "count" (pass "id" or omit it). |
| **relationshipField** | `string` | optional | The child FK field pointing back at this parent. Auto-detected from the child's lookup / master_detail; set it only when the child has more than one reference to this parent. |
| **conditions** | `{ field: string; op: Enum<'lt' \| 'lte' \| 'gt' \| 'gte' \| 'eq' \| 'ne'>; value: number \| string \| boolean }[]` | optional | CONDITIONAL roll-up: aggregate only the child rows matching these comparisons (ANDed). REQUIRED whenever the field name carries a qualifier — "已完成任务数 / 已收货金额 / 待处理工单数", any 已X / 未X / `<某状态>`的 count-or-sum → e.g. [`{ field: "status", op: "eq", value: "completed" }`]. WITHOUT it the roll-up silently counts EVERY child and reports a plausible-looking WRONG number, which is worse than a visible 0. |
| **filter** | `any` | optional | The same predicate as a canonical query filter map (e.g. `{ status: "completed" }`, `{ status: { $in: ["received", "partial"] }` }). Use it when hand-authoring a blueprint; the structured design path uses `conditions` instead. Wins over `conditions` when both are given. |
| **filter** | `any` | optional | The same predicate as a canonical query filter map (e.g. `{ status: "completed" }`, `{ status: { $in: ["received", "partial"] } }`). Use it when hand-authoring a blueprint; the structured design path uses `conditions` instead. Wins over `conditions` when both are given. |


---
Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/api/analytics.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ const result = AnalyticsEndpoint.parse(data);
| **limit** | `number` | optional | |
| **offset** | `number` | optional | |
| **timezone** | `string` | optional | |
| **query** | `any` | optional | [REMOVED] `query` was removed from AnalyticsQueryRequest in @objectstack/spec 17.0.0 (#3878). The `{ cube, query: {...}` } envelope was the dialect of the retired degraded analytics shim (#3891) — the real engine never understood it. Move the query.* fields to the body top level: `{ cube, measures, dimensions?, where?, timeDimensions?, order?, limit?, offset?, timezone? }`. |
| **query** | `any` | optional | [REMOVED] `query` was removed from AnalyticsQueryRequest in @objectstack/spec 17.0.0 (#3878). The `{ cube, query: {...} }` envelope was the dialect of the retired degraded analytics shim (#3891) — the real engine never understood it. Move the query.* fields to the body top level: `{ cube, measures, dimensions?, where?, timeDimensions?, order?, limit?, offset?, timezone? }`. |
| **format** | `any` | optional | [REMOVED] `format` was removed from AnalyticsQueryRequest in @objectstack/spec 17.0.0 (#3878). It was never implemented — every response is the JSON envelope. Delete the key; for CSV/XLSX use the export surface instead. |


Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/automation/flow.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ const result = FlowSchema.parse(data);
| **position** | `{ x: number; y: number }` | optional | |
| **timeoutMs** | `integer` | optional | Maximum execution time for this node in milliseconds |
| **inputSchema** | `Record<string, { type: Enum<'string' \| 'number' \| 'boolean' \| 'object' \| 'array'>; required: boolean; description?: string }>` | optional | Input parameter schema for this node |
| **outputSchema** | `any` | optional | [REMOVED] `flow.nodes[].outputSchema` was removed in @objectstack/spec 17.0.0 (#3896 audit close-out) — it was never validated: the engine does not check node outputs against it, so it documented a contract nothing enforced. Delete the key. Downstream nodes read prior outputs via expressions (`{{nodeId.field}`}) regardless of any declaration. |
| **outputSchema** | `any` | optional | [REMOVED] `flow.nodes[].outputSchema` was removed in @objectstack/spec 17.0.0 (#3896 audit close-out) — it was never validated: the engine does not check node outputs against it, so it documented a contract nothing enforced. Delete the key. Downstream nodes read prior outputs via expressions (`{{nodeId.field}}`) regardless of any declaration. |
| **waitEventConfig** | `{ eventType: Enum<'timer' \| 'signal' \| 'webhook' \| 'manual' \| 'condition'>; timerDuration?: string; signalName?: string; timeoutMs?: any; … }` | optional | Configuration for wait node event resumption |
| **boundaryConfig** | `{ attachedToNodeId: string; eventType: Enum<'error' \| 'timer' \| 'signal' \| 'cancel'>; interrupting: boolean; errorCode?: string; … }` | optional | Configuration for boundary events attached to host nodes |

Expand Down
44 changes: 3 additions & 41 deletions packages/spec/scripts/build-docs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import {
resolveImports,
type CategorySurface,
} from './lib/docs-import-surface';
import { escapeMdxDescription } from './lib/escape-mdx';
import { anchorFor, formatType, type TypeContext } from './lib/format-type';
import { createSink } from './lib/generated-output';
import { schemaNameFromExportKey } from './lib/schema-name';
Expand Down Expand Up @@ -257,47 +258,8 @@ function generateMarkdown(schemaName: string, schema: any, category: string, zod
// Add schema heading
md += `## ${schemaName}\n\n`;

// Escape MDX-unsafe characters in description text. MDX parses `{` as a JS
// expression and `<` as JSX, so any raw `{token}` / `<title>` inside a Zod
// `.describe()` string breaks the docs build. Wrap such fragments in inline
// code so they render literally.
//
// Single pass with backtick tracking: fragments already inside an inline-code
// span are left untouched. A naive two-pass replace double-wraps nested cases
// like `{<id>}` into `` `{`<id>`}` `` — the inner backticks close the span
// early and leak `<id>` as raw JSX (MDX: "Expected a closing tag for `<id>`").
//
// A matched `{…}` / `<…>` pair is wrapped in an inline-code span so it renders
// literally. A *lone* `<` or `{` with no closing partner (e.g. a SemVer range
// `">=4.0 <5"`, or prose like `count < 5`) can't be wrapped, so it is replaced
// with its HTML entity — otherwise MDX reads the `<` as the start of a JSX tag
// and the build dies ("Unexpected character `5` before name").
const escapeMdxDescription = (raw: string): string => {
let out = '';
let inCode = false;
for (let i = 0; i < raw.length; i++) {
const ch = raw[i];
if (ch === '`') {
inCode = !inCode;
out += ch;
continue;
}
if (!inCode && (ch === '{' || ch === '<')) {
const close = ch === '{' ? '}' : '>';
const end = raw.indexOf(close, i + 1);
if (end !== -1) {
out += '`' + raw.slice(i, end + 1) + '`';
i = end;
continue;
}
// Unmatched: escape so MDX doesn't treat it as a JSX/expression opener.
out += ch === '<' ? '&lt;' : '&#123;';
continue;
}
out += ch;
}
return out;
};
// Description text is made MDX-safe by `lib/escape-mdx.ts` — extracted so the
// escaping can be pinned directly instead of by grepping emitted `.mdx`.

// Add description with better formatting
if (mainDef.description) {
Expand Down
157 changes: 157 additions & 0 deletions packages/spec/scripts/escape-mdx.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.

/**
* Pin for how the reference-docs generator escapes `{…}` / `<…>` inside
* `.describe()` prose — #5452.
*
* The escaper wrapped a delimited fragment in an inline-code span, but found
* its closing partner with `indexOf`, i.e. the FIRST closer rather than the
* MATCHING one. Nested delimiters are not an exotic input here — `{{var}}` is
* the template-interpolation syntax the spec's own prose *teaches*, and a
* filter map (`{ status: { $in: [...] } }`) is an ordinary example — so the
* wrap ended at the inner `}` and the outer one fell outside the span:
* `{{var}}` was published as `` `{{var}` `` plus a stray `}`.
*
* The issue counted three sites (`grep -rn '`{{' content/docs/references/`).
* The corpus gate below counts FIVE, because that grep only sees the fragments
* whose nesting starts at character one; `ai/solution-blueprint.mdx` and
* `api/analytics.mdx` were cut in exactly the same place with a single leading
* brace, and no grep for `` `{{ `` could have found them.
*
* MEASURED (reverse verification), both directions run:
* - restoring the old matcher (`raw.indexOf(close, i + 1)` in place of
* `findMatchingClose`) turns all five nested-delimiter unit cases red
* (`5 failed | 7 passed`), each reporting the split shape;
* - and, after re-running `gen:docs` over the restored escaper, the corpus
* gate goes red with 5 offenders — the same 5 that were on `main`.
* The direction is the ordinary one (restore the defect → the new pins go red)
* because these assert a POSITIVE output shape the fix produces, not the
* absence of a finding. The single-delimiter and unmatched cases stay green
* either way, which is exactly why the bug survived: the escaper was correct
* on every shape anyone had thought to look at.
*/

import fs from 'fs';
import path from 'path';
import url from 'url';

import { describe, expect, it } from 'vitest';

import { escapeMdxDescription } from './lib/escape-mdx';

const HERE = path.dirname(url.fileURLToPath(import.meta.url));
const REPO = path.resolve(HERE, '../../..');
const REFERENCES = path.join(REPO, 'content/docs/references');

describe('escapeMdxDescription — nested delimiters (#5452)', () => {
it('wraps `{{var}}` whole, leaving no stray closer outside the span', () => {
expect(escapeMdxDescription('System prompt — supports {{var}} interpolation')).toBe(
'System prompt — supports `{{var}}` interpolation',
);
});

it('wraps a doubled brace inside parentheses (the automation/flow.mdx specimen)', () => {
expect(
escapeMdxDescription(
'Downstream nodes read prior outputs via expressions ({{nodeId.field}}) regardless.',
),
).toBe('Downstream nodes read prior outputs via expressions (`{{nodeId.field}}`) regardless.');
});

it('wraps a singly-nested filter map whole (the ai/solution-blueprint.mdx specimen)', () => {
// The variant `grep '`{{'` could not see: nesting that starts one char in.
expect(escapeMdxDescription('e.g. { status: { $in: ["received"] } }.')).toBe(
'e.g. `{ status: { $in: ["received"] } }`.',
);
});

it('wraps nested angle delimiters whole', () => {
// The span opens at the delimiter, not at the identifier in front of it —
// pre-existing behaviour, and MDX-safe either way. What #5452 changes is
// that BOTH closing `>` land inside the span instead of one leaking out.
expect(escapeMdxDescription('Shaped as Array<Record<string, any>> at rest')).toBe(
'Shaped as Array`<Record<string, any>>` at rest',
);
});

it('wraps each of two independent doubled pairs on one line', () => {
expect(escapeMdxDescription('render {{a.b}} then {{c}} here')).toBe(
'render `{{a.b}}` then `{{c}}` here',
);
});
});

describe('escapeMdxDescription — shapes the fix must not disturb', () => {
it('still wraps a single `{…}` pair', () => {
expect(escapeMdxDescription('a {token} b')).toBe('a `{token}` b');
});

it('still wraps a `{<id>}` nest in ONE span (no inner backticks)', () => {
expect(escapeMdxDescription('path {<id>} here')).toBe('path `{<id>}` here');
});

it('still entity-escapes a lone `<` with no partner (SemVer range)', () => {
expect(escapeMdxDescription('supports >=4.0 <5 only')).toBe('supports >=4.0 &lt;5 only');
});

it('still entity-escapes a lone `{` with no partner', () => {
expect(escapeMdxDescription('an unclosed { here')).toBe('an unclosed &#123; here');
});

it('still leaves fragments already inside an inline-code span untouched', () => {
expect(escapeMdxDescription('see `{{var}}` above')).toBe('see `{{var}}` above');
});
});

/**
* Corpus gate over the generator's committed OUTPUT.
*
* The invariant is brace BALANCE inside an inline-code span, not the literal
* `` `{{ ``-plus-stray-`}` string the issue grepped for. Balance is what the
* defect actually violates — the wrap cut a pair in half — so it catches the
* two sites whose nesting did not start at character one, and it keeps
* catching them when the offending prose is reworded.
*
* Angle delimiters deliberately get NO corpus gate: `<`/`>` are also the
* comparison operators, and validation-rule / SemVer examples legitimately
* carry an unbalanced one inside a code span (`record.amount < 0`,
* `>=1.2.3`) — 11 such spans, all correct. Nesting for angles is pinned by
* the positive unit case above instead. Backslash-escaped delimiters are not
* delimiters: the module-JSDoc path escapes braces as `\{`, so they are
* dropped before counting.
*/
describe('published reference pages keep inline-code braces balanced (#5452)', () => {
const pages: string[] = [];
const walk = (dir: string) => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) walk(full);
else if (entry.name.endsWith('.mdx')) pages.push(full);
}
};
walk(REFERENCES);

it('finds the generated reference corpus', () => {
expect(pages.length).toBeGreaterThan(50);
});

it('has no inline-code span with an unbalanced brace', () => {
const offenders: string[] = [];
for (const file of pages) {
const rel = path.relative(REPO, file);
fs.readFileSync(file, 'utf-8')
.split('\n')
.forEach((line, index) => {
// Odd segments of a backtick split are the inline-code spans.
const segments = line.split('`');
for (let i = 1; i < segments.length; i += 2) {
const span = segments[i].replace(/\\[{}]/g, '');
const opens = span.split('{').length - 1;
const closes = span.split('}').length - 1;
if (opens !== closes) offenders.push(`${rel}:${index + 1} \`${segments[i]}\``);
}
});
}
expect(offenders).toEqual([]);
});
});
Loading
Loading