diff --git a/docs/authors/writing-tutorials.md b/docs/authors/writing-tutorials.md index 2df77b496..e132e4450 100644 --- a/docs/authors/writing-tutorials.md +++ b/docs/authors/writing-tutorials.md @@ -425,6 +425,93 @@ Key behaviors: Full worked walkthrough (self-demonstrating): **[Use Variables to Personalize Your Tutorials](https://developers.sap.com/tutorials/use-variables-in-tutorials)** (source: `sap-tutorials/meta-tutorials`). +### 3.8 Directives ★ NEW + +Directives are a generic structured-content syntax for tutorials. They allow you to mark up passages that will be handled by specialized components, added to the platform in follow-up features. The directive system ships with **no concrete directives yet** — it is the foundation only. Examples here use hypothetical directive names to show the syntax; they will not render as live components until those features land. + +Directives have three forms: + +**Inline directives** (mid-sentence): + +```markdown +This feature is available in `:menu[Production]{edition="prod"}` only. +``` + +A directive spans from the opening `:` through the closing `}`. It has a **name** (kebab-case), an optional **label** in brackets, and optional **attributes** in braces. + +**Leaf directives** (standalone, block-level): + +```markdown +Some text above. + +::code-sample[Download the SDK]{url="https://example.com/sdk.zip" filename="sdk.zip"} + +More text below. +``` + +Leaf directives are alone on a line, prefixed with `::`, and render as a self-contained block. + +**Container directives** (wrapping other content): + +```markdown +:::details[Click to expand] +This content is hidden until the reader clicks the heading. +More paragraphs and **markdown** all work here. +::: +``` + +Container directives open with `:::` and close with `:::`. Content between them is Markdown, rendered normally. Nesting uses extra colons — an outer container uses more colons than its inner content: + +```markdown +::::compare +Outer container (four colons). + +:::figure[Screenshot A]{align="left"} +Inner figure (three colons). +::: + +:::figure[Screenshot B]{align="right"} +Another inner figure. +::: +:::: +``` + +**Attribute syntax:** + +- **Quoted values:** `key="value"` or `key='value'` — use double or single quotes around values with spaces. +- **Unquoted values:** `key=value` (no spaces in the value itself). +- **Bare flags:** `open` → treated as `open=true`. +- **CSS classes:** `.classname` — shorthand for `class="classname"`. The directive's implementation controls which classes are allowed (you will get a build warning if you use a class not in its allowlist). +- **Element IDs:** `#id` — shorthand for `id="id"`. +- **Duplicate keys:** an error; each key must appear once. + +Examples: + +```markdown +:::panel[Expandable section]{open .highlight #my-panel} + +This is a container with `open=true`, class `highlight`, and id `my-panel`. + +::: + +:code[JavaScript]{lang="js" theme='dark'} + +:::details[Learn more]{class="advanced"} +For advanced users only. +::: +``` + +**Escaping and safety:** + +- Attribute values are **always HTML-escaped** by the framework — you never need to escape manually, and HTML injection through attributes is not possible. +- Directives inside code fences, inline code, indented code blocks, HTML blocks, or YAML frontmatter are **never interpreted** — they render literally as text. This keeps your Markdown safe when it happens to contain colons and braces that look like a directive but aren't. + +**Unknown directive names:** + +If you use a directive name that doesn't exist (e.g. `:menu[...]` before the `menu` feature ships), the text is left as-is (rendered literally) and you'll see a **lint warning** `directive-unknown` during the build. This warning never stops a build — it's informational only. The intent is to make it safe to write Markdown that *coincidentally* looks like `:something[...]` without risking a hard failure. + +When a concrete directive is added to the platform, the warning will stop and the directive will render as a styled component. + --- ## 4. The author workflow today diff --git a/docs/developers/reference/tutorials-ims-gotchas.md b/docs/developers/reference/tutorials-ims-gotchas.md index edf6838c3..771e21c3c 100644 --- a/docs/developers/reference/tutorials-ims-gotchas.md +++ b/docs/developers/reference/tutorials-ims-gotchas.md @@ -15,6 +15,8 @@ Cross-references: - **`CAP_BASE_URL` env var** — Used by `scripts/parsers/cap.ts` and migration scripts. Defaults to `http://localhost:4004`. - **Node.js >= 20 required** — Build scripts use native `fetch` (no polyfill). - **Slug fields** — `Missions.slug` and `CompletionPaths.slug` must be populated for the build pipeline to generate mission/group pages. Run `node scripts/migrate-reference-data.js populate-slugs` after data import. +- **Directive pre-pass pipeline position** — The directive grammar parser (`scripts/parsers/directives.ts`) runs in `scripts/parsers/compose.ts` AFTER variable substitution (`wrapTokens`) and BEFORE option-block conversion (`convertOptionBlocks`). This means `{{var:x}}` tokens inside directive labels work (variables are already expanded), and directives can wrap option blocks. The framework reserves Hugo shortcode namespace `tut-*` for directive output; any new directive implementation must NOT generate a shortcode with that prefix or it will collide with the framework. +- **Empty directive registry byte-identical invariant** — When no concrete directives are registered (the foundation-only state), a tutorial with no directive markup parses and renders byte-identical to the pre-directive version. This is enforced by a corpus regression test (`test/unit/directives.test.js`). Before registering a new directive, ensure it ships with: a summary in `DirectiveRegistry`, a `toText()` plain-text fallback, full developer docs, and a regression suite (the conformance test will fail loudly if these are missing). Handoff: [docs/developers/reference/directive-grammar.md](directive-grammar.md). ## Directory layout diff --git a/docs/superpowers/plans/2026-11-05-tutorial-directive-grammar.md b/docs/superpowers/plans/2026-11-05-tutorial-directive-grammar.md new file mode 100644 index 000000000..1ab07ba66 --- /dev/null +++ b/docs/superpowers/plans/2026-11-05-tutorial-directive-grammar.md @@ -0,0 +1,284 @@ +# Tutorial Directive Grammar Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add one generic CommonMark-style directive grammar (`:name[label]{attrs}` inline / `::name` leaf / `:::name` container) to the tutorial markdown pipeline, with a registry, build-time validation, a Hugo `tut-` shortcode rendering contract, and a shared plain-text fallback — shipping zero concrete directives so existing tutorials render byte-identically. + +**Architecture:** A fence-aware pre-pass (`scripts/parsers/directives.ts`) parses directives using the existing `md-grammar.ts` `classifyLines` classifier, validates each against a registry (`directive-registry.ts`), and rewrites them into `tut-` Hugo shortcodes. It slots into `compose.ts` after `wrapTokens` and before `convertOptionBlocks`. A separate `renderDirectivesAsText()` entrypoint gives MCP/AI consumers readable text. The registry starts empty; follow-up changes register directives. + +**Tech Stack:** TypeScript, vitest (`--project unit`), Hugo/Goldmark shortcodes, tsx CLI harnesses, markdown-it (via existing md-grammar). + +**Spec:** `openspec/changes/tutorial-directive-grammar/` (proposal.md, design.md, specs/tutorial-directive-grammar/spec.md, tasks.md) + +## Global Constraints + +- Directive names match `[a-z][a-z0-9-]*` (lowercase kebab-case). Copied verbatim from spec D1. +- No new runtime dependency — hand-written parser, reuse `md-grammar.ts classifyLines`; no remark/unified. (proposal Impact, design Non-Goals) +- Pipeline position is fixed: `expandDirectives` runs **after `wrapTokens`** (compose.ts ~line 154) and **before `convertOptionBlocks`** (compose.ts ~line 184). (spec "Pipeline compatibility", design D3) +- Zero output change for tutorials with no directives — proven by a byte-identical corpus regression with an empty registry. (spec "Legacy markers unchanged") +- All attribute/label values HTML- and quote-escaped before entering shortcode params; shortcodes never `safeHTML` attribute values. (spec "Shortcode rendering contract", design D4) +- Unknown directive names are left byte-for-byte unchanged and emit a `directive/unknown` **warning**, never an error. (spec "False-positive protection", design D2) +- The existing `escapeHugoDelimiters` (in `render-frontmatter.ts`, downstream of compose) must not mangle generated `tut-*` shortcodes — they use the same `{{% %}}`/`{{< >}}` delimiters it already preserves via sentinel. (design D4) +- CDS/backend not touched by this change; foundation is parser + Hugo + MCP/AI read-path only. + +## Review Focus + +- **Attribute-value HTML/quote injection** (`:ui[x]{title="\" onmouseover=alert(1)"}`) → escaped text only, no event-handler attribute reaches HTML. Test in Task 2. +- **Directive syntax inside fenced/indented code and inline code spans** → emitted byte-for-byte unchanged, never parsed. Test in Task 1 (code), Task 4 (via compose integration). +- **Unclosed container at end of step** → `directive/unclosed` error pointing at the opening line, not a silent swallow of the rest of the step. Test in Task 1. +- **Prose that coincidentally looks inline** (`10:30`, `std::vector`, `http://x`, `:smile:`, `:foo{x=1}` with no `[` label) → no match, text unchanged. Test in Task 1. +- **Variable token inside a directive label** (`:menu[dev-{{var:n}}]`) → the live variable span survives because directives run after `wrapTokens`. Test in Task 4. + +--- + +## Task 1: Registry + tokenizer + core parser + +**Files:** +- Create: `scripts/parsers/directive-registry.ts` +- Create: `scripts/parsers/directives.ts` (parser half: `parseDirectives`, `DirectiveParseError`, node types) +- Test: `scripts/parsers/__tests__/directive-registry.test.ts` +- Test: `scripts/parsers/__tests__/directives-parse.test.ts` + +**Interfaces:** +- Consumes: `classifyLines(md): LineClass[]` and `isNonDelimiterLine` from `./md-grammar.js` (LineClass = `{ kind: 'code'|'heading'|'html'|'other'; level: number }`, one per line). +- Produces: + - `interface DirectiveDef { name; kinds: Array<'inline'|'leaf'|'container'>; summary; label?: {required; markdown}; attrs: Record; classes?: string[]; expand?(node, ctx): string; render(node, ctx): string; toText(node, ctx): string; since: string }` + - `registerDirective(def: DirectiveDef): void`, `getRegistry(): ReadonlyMap`, `resetRegistry(): void` (test-only) + - `interface DirectiveNode { name; kind: 'inline'|'leaf'|'container'; label?: string; attrs: Record; id?: string; classes: string[]; body?: string; line: number; column: number }` + - `class DirectiveParseError extends Error { slug; line; column; directive; code } ` — codes: `unknown`(warning) `missing-attr` `invalid-attr` `duplicate-attr` `invalid-kind` `unclosed` `invalid-class` + - `parseDirectives(md: string, opts: { slug: string; registry?: ReadonlyMap }): { nodes: DirectiveNode[]; warnings: Array<{code; line; message}> }` — returns structural parse (no rendering); throws `DirectiveParseError` on schema violations. + - `tokenizeAttrs(raw: string): { attrs; id?; classes: string[] }` (exported for unit test) + +- [ ] **Step 1: Write failing registry tests** in `directive-registry.test.ts`: `registerDirective` then `getRegistry().get(name)` round-trips; duplicate registration of same name throws; `resetRegistry()` empties it; name not matching `/^[a-z][a-z0-9-]*$/` throws. + +- [ ] **Step 2: Write failing tokenizer + parser tests** in `directives-parse.test.ts`. Register a fixture `echo` def (kinds `['inline','leaf','container']`, optional label, attrs `{n:{type:'number'}, flag:{type:'boolean'}}`, classes `['big']`). Assert: + - `tokenizeAttrs('n=2 flag #setup-block .big')` → `{attrs:{n:2, flag:true}, id:'setup-block', classes:['big']}` + - duplicate key `os=mac os=win` → throws `DirectiveParseError` code `duplicate-attr` with the line + - `.class` not in allowlist → code `invalid-class` + - inline `:echo[hi]{n=1}` in prose → one node kind `inline` label `hi` + - leaf `::echo[x]` alone on a line → node kind `leaf` + - container `:::echo[t]\nbody\n:::` → node kind `container`, body `body` + - nested `::::echo\n:::echo\ninner\n:::\n::::` → outer body contains inner container in source order + - false positives: `10:30`, `std::vector`, `http://x`, `:smile:`, `:echo{n=1}` (no `[`) → **zero** nodes + - unknown `:foo[bar]` → zero nodes + one `directive/unknown` warning with line + - fenced code ` ```js\n:::echo\n``` ` → zero nodes (byte-for-byte skip) + - inline code span `` `:echo[x]` `` → zero nodes + - `:::echo` with no closing fence before EOF → throws code `unclosed` at the opening line + - wrong kind: register `only-container` as `kinds:['container']`, use it inline → code `invalid-kind` + - missing required attr → code `missing-attr` + +- [ ] **Step 3: Run tests, verify they fail** — `npx vitest run --project unit scripts/parsers/__tests__/directive-registry.test.ts scripts/parsers/__tests__/directives-parse.test.ts` → FAIL (modules/exports absent). + +- [ ] **Step 4: Implement `directive-registry.ts`** — Map-backed registry with the signatures above; validate name on register; `resetRegistry` for tests. + +- [ ] **Step 5: Implement the tokenizer and `parseDirectives` in `directives.ts`.** Use `classifyLines` to mask `kind==='code'`/`'html'` lines and frontmatter; track inline-code spans within `'other'` lines; apply D2 false-positive guards (preceding char in `([{"'`+line start, required `[` for inline, columns 0–3 for leaf/container respecting list/blockquote prefix); container nesting by colon count; validate each node against its registry def, throwing `DirectiveParseError` (or pushing a warning for unknown). + +- [ ] **Step 6: Run tests, verify they pass.** Same command as Step 3 → PASS. + +- [ ] **Step 7: Commit** — `git add scripts/parsers/directive-registry.ts scripts/parsers/directives.ts scripts/parsers/__tests__/directive-registry.test.ts scripts/parsers/__tests__/directives-parse.test.ts && git commit -m "feat(directives): registry + fence-aware directive parser (openspec: tutorial-directive-grammar)"` + +--- + +## Task 2: Rendering (`expandDirectives`) + escaping + +**Files:** +- Modify: `scripts/parsers/directives.ts` (add `expandDirectives`, `escapeShortcodeParam`) +- Test: `scripts/parsers/__tests__/directives-render.test.ts` + +**Interfaces:** +- Consumes: `parseDirectives`, `DirectiveNode`, `getRegistry` from Task 1. +- Produces: + - `expandDirectives(body: string, ctx: { slug: string; variables?: TutorialVariable[]; registry?: ReadonlyMap }): string` — runs `expand` hooks first (with cycle detection, max-depth guard), then replaces each node with its `render` output. Inline/leaf → `{{< tut- label="…" k="…" >}}`; container → `{{% tut- label="…" %}}\n\n{{% /tut- %}}`; defs declaring `render:'markdown'` return raw markdown. + - `escapeShortcodeParam(v: string): string` — HTML-escapes `& < > "` and the param quote so no value breaks out of `"…"` or injects an attribute. + +- [ ] **Step 1: Write failing tests** in `directives-render.test.ts`, registry reset + fixture `echo` (container → `{{% %}}`, inline → `{{< >}}`) and a `sugar` def with `render:'markdown'`: + - inline `:echo[hi]{n=1}` → `{{< tut-echo label="hi" n="1" >}}` + - container `:::echo[t]\n**b**\n:::` → `{{% tut-echo label="t" %}}\n**b**\n{{% /tut-echo %}}` + - `render:'markdown'` def returns its markdown unwrapped (no shortcode) + - **injection**: `:echo[Save]{title="\" onmouseover=alert(1) x=\""}` → rendered shortcode param contains the value HTML-escaped, with no bare `onmouseover=` token and no unescaped `"` closing the param + - cycle: a fixture `expand` hook that re-emits itself → throws (depth guard), does not hang + +- [ ] **Step 2: Run, verify fail** — `npx vitest run --project unit scripts/parsers/__tests__/directives-render.test.ts` → FAIL. + +- [ ] **Step 3: Implement `escapeShortcodeParam` and `expandDirectives`** per the signatures. Replace nodes back-to-front by source offset to keep positions stable. + +- [ ] **Step 4: Run, verify pass.** + +- [ ] **Step 5: Commit** — `git commit -m "feat(directives): expandDirectives rendering + param escaping"` + +--- + +## Task 3: Plain-text fallback (`renderDirectivesAsText`) + +**Files:** +- Modify: `scripts/parsers/directives.ts` (add `renderDirectivesAsText`) +- Test: `scripts/parsers/__tests__/directives-text.test.ts` + +**Interfaces:** +- Consumes: `parseDirectives`, `getRegistry` from Task 1. +- Produces: `renderDirectivesAsText(markdown: string, opts?: { registry? }): string` — replaces each registered node with its `toText(node, ctx)` output; unknown directives left as-is; code/fence contexts untouched. + +- [ ] **Step 1: Write failing tests** in `directives-text.test.ts`, fixture `details` (container, `toText` → `**