diff --git a/.changeset/powershell.md b/.changeset/powershell.md new file mode 100644 index 0000000..a30db2f --- /dev/null +++ b/.changeset/powershell.md @@ -0,0 +1,5 @@ +--- +"@tanstack/highlight": minor +--- + +Add isolated PowerShell highlighting with pwsh and ps1 aliases and selective imports. diff --git a/README.md b/README.md index 787ad11..2b66cf8 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,7 @@ This is not an editor parser or a TextMate engine. It is a deliberately small do - [Quick Start](docs/quick-start.md) - [Comparison](docs/comparison.md) - [Language Support](docs/language-support.md) +- [PowerShell Showcase](docs/guides/powershell.md) - [Octane Integration](docs/guides/octane.md) - [Guides](docs/guides/language-registration.md) - [API Reference](docs/reference/index.md) @@ -218,7 +219,8 @@ Local browser bundles, minified with esbuild and compressed independently. KB us | Core + TSX | 10.11 KB | 4.26 KB | 3.90 KB | | Octane MDX + TypeScript | 13.86 KB | 5.59 KB | 5.17 KB | | Nine-language docs set | 16.04 KB | 6.20 KB | 5.66 KB | -| All 39 languages | 52.74 KB | 17.73 KB | 15.93 KB | +| PowerShell + core | 7.19 KB | 3.25 KB | 3.01 KB | +| All 40 languages | 55.38 KB | 18.47 KB | 16.65 KB | The following comparison was measured before the 1.0 property-context correction. Re-run the comparison commands below for current timings. diff --git a/docs/config.json b/docs/config.json index 86099b9..29198e0 100644 --- a/docs/config.json +++ b/docs/config.json @@ -33,6 +33,7 @@ { "label": "SSR and Client Rendering", "to": "guides/ssr-and-client" }, { "label": "React Integration", "to": "guides/react" }, { "label": "Octane Integration", "to": "guides/octane" }, + { "label": "PowerShell Showcase", "to": "guides/powershell" }, { "label": "Custom Languages", "to": "guides/custom-languages" }, { "label": "Bundle Size and Performance", "to": "guides/performance" } ] diff --git a/docs/guides/performance.md b/docs/guides/performance.md index 9bf4fce..b19d4e9 100644 --- a/docs/guides/performance.md +++ b/docs/guides/performance.md @@ -8,7 +8,7 @@ CI measures selective browser bundles and highlighting performance on real docum ## Bundle profiles -`pnpm run size` builds twenty-six browser profiles with esbuild and measures minified, gzip, and Brotli bytes independently. It also checks that helper, adapter, and selective language imports retain only the requested modules. +`pnpm run size` builds twenty-seven browser profiles with esbuild and measures minified, gzip, and Brotli bytes independently. It also checks that helper, adapter, and selective language imports retain only the requested modules. | Profile | Registered languages | Current gzip | CI budget | | --- | --- | ---: | ---: | @@ -16,17 +16,19 @@ CI measures selective browser bundles and highlighting performance on real docum | TSX | TSX | 4.26 KB | 4.35 KB | | Octane | TypeScript plus Octane MDX adapter | 5.59 KB | 5.7 KB | | Docs | CSS, HTML, JS, JSON, JSX, Markdown, Shell, TS, TSX | 6.20 KB | 6.3 KB | -| All | All 39 definitions | 17.73 KB | 18.0 KB | +| All | All 40 definitions | 18.47 KB | 18.75 KB | KB uses 1,000 bytes. Core helpers imported from the root tree-shake to the same engine size. The standalone theme helper is 695 gzip bytes. +PowerShell plus core measures 3,252 gzip bytes. Adding PowerShell and correcting short Swift macro declarations grows the all-language entry from 17,681 to 18,471 gzip bytes (+790), measured under identical Node 26.11.1 tooling. The Swift selective profile grows by 13 gzip bytes and stays within its existing budget. All 24 unrelated profiles and their budgets are unchanged; only the all-language budget increases. + Selective profiles are the primary metric. The all-language profile exists to prevent convenience-entry growth from becoming invisible. ## Runtime corpus The committed corpus contains 334 real code fences sampled from TanStack documentation, with up to twenty samples per normalized language. -`pnpm run bench` measures tokenization, HTML, Markdown, HAST, line numbers, long decorated blocks, and dedicated C#, C++, CMake, Dart, Java, Kotlin, Lua, Perl, PHP, Ruby, Rust, and Swift samples. Each profile reports the median of three samples after two warmup passes, with a 1.2 second CI budget. The main highlighting profile processes at least 10,000 blocks. +`pnpm run bench` measures tokenization, HTML, Markdown, HAST, line numbers, long decorated blocks, and dedicated C#, C++, CMake, Dart, Java, Kotlin, Lua, Perl, PHP, PowerShell, Ruby, Rust, and Swift samples. Each profile reports the median of three samples after two warmup passes, with a 1.2 second CI budget. The main highlighting profile processes at least 10,000 blocks. A local before-and-after review used the same minified bundle settings, fixtures, and benchmark harness on macOS arm64 with Node 24.15.0: diff --git a/docs/guides/powershell.md b/docs/guides/powershell.md new file mode 100644 index 0000000..c87dedf --- /dev/null +++ b/docs/guides/powershell.md @@ -0,0 +1,83 @@ +--- +title: PowerShell Showcase +--- + +# PowerShell Showcase + +Register PowerShell directly for report and automation examples. The `pwsh` and `ps1` aliases normalize to `powershell` when it is registered. + +```ts +import { createHighlighter } from '@tanstack/highlight/core' +import { powershell } from '@tanstack/highlight/languages/powershell' + +const highlighter = createHighlighter({ languages: [powershell] }) +const result = highlighter.highlight('Get-Item $HOME', { lang: 'pwsh' }) +``` + +## A report pipeline + +An advanced function combines attributes, splatting, scoped variables, typed records, here-strings, null coalescing, and case-insensitive word operators. The API URL is a placeholder. + +```powershell +#Requires -Version 7.4 +<# Build typed records, then shape a pipeline for display. + Quotes and $variables in this comment stay quiet. #> +function Get-StationReport { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [ValidateSet('coast', 'ridge')] + [string[]] $Station + ) + + begin { + $script:Endpoint = $env:STATION_API ?? 'https://example.com' + $headers = @{ Accept = 'application/json' } + $budget = 64MB + } + process { + foreach ($name in $Station) { + $request = @{ + Uri = "$script:Endpoint/observations/$name" + Headers = $headers + ErrorAction = 'Stop' + } + try { + $reading = Invoke-RestMethod @request + [pscustomobject]@{ + Station = $name + Celsius = [double] $reading.celsius + Online = $true + } + } + catch { + Write-Warning -Message "Station $name unavailable: $_" + } + } + } + end { + $banner = @" +Forecast ready +Budget: $budget +"@ + $literal = @' +$HOME is literal; # this is text, not a comment. +'@ + Write-Verbose -Message ($banner + $literal) + } +} + +${report-title} = 'Today''s observatory' +Get-StationReport -Station coast, ridge | + Where-Object { $_.Online -and $_.Celsius -ge 0 } | + Sort-Object -Property Celsius -Descending | + Select-Object -First 5 -Property Station, Celsius +``` + +## Preview and scope + +Run `pnpm run report:compare` and open `artifacts/shiki-comparison.html`. This showcase appears first in GitHub Light and Aurora X alongside Shiki's reference. Its canonical source is `test/showcases/Get-StationReport.ps1`. + +Interpolation remains inside the string token. Full PowerShell command/argument disambiguation and symbol resolution are outside the lightweight tokenizer's scope. + +Lexical rules follow PowerShell's [quoting rules](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules) and [language specification](https://learn.microsoft.com/en-us/powershell/scripting/lang-spec/chapter-02). diff --git a/docs/installation.md b/docs/installation.md index 30bf9f9..79d9916 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -60,7 +60,7 @@ export const highlighter = createHighlighter({ }) ``` -The root entry is useful for prototypes, server-only scripts, or sites where the roughly 18 KB gzip all-language build is acceptable: +The root entry is useful for prototypes, server-only scripts, or sites where the [all-language bundle size](guides/performance) is acceptable: ```ts import { highlight } from '@tanstack/highlight' diff --git a/docs/language-support.md b/docs/language-support.md index bf63dee..edc5d90 100644 --- a/docs/language-support.md +++ b/docs/language-support.md @@ -35,6 +35,7 @@ Every language is an isolated definition imported from `@tanstack/highlight/lang | Perl | `perl` | `pl` | Sigil and special variables, quote-like operators with nested delimiters, regex vs division, heredocs, POD blocks | | PHP | `php` | - | PHP tags, attributes, quoted strings, heredoc/nowdoc, optional HTML delegation | | Plaintext | `plaintext` | `text`, `txt`, `-->` | Escaping only | +| PowerShell | `powershell` | `pwsh`, `ps1` | Quoted and here-strings, nested expandable expressions, scoped/braced/splat variables, cmdlets, parameters, type literals, case-insensitive keywords and operators | | Python | `python` | `py` | Triple strings, prefixes, decorators, comments | | Ruby | `ruby` | `rb` | Interpolated strings with nested quotes, percent literals, heredocs, regex vs division, symbols and hash keys, block comments | | Rust | `rust` | `rs` | Nested block comments, raw strings with hash counts, byte and C strings, lifetimes vs character literals, attributes, macros | diff --git a/docs/reference/default-entry.md b/docs/reference/default-entry.md index fa507a3..6245199 100644 --- a/docs/reference/default-entry.md +++ b/docs/reference/default-entry.md @@ -43,7 +43,7 @@ Returns the canonical registered language name for a name or alias. Names are tr function listLanguages(): Array ``` -Returns the 39 canonical language names registered in `defaultHighlighter`. +Returns the 40 canonical language names registered in `defaultHighlighter`. ### `tokenize` diff --git a/docs/reference/languages.md b/docs/reference/languages.md index bd26d4b..05efbc3 100644 --- a/docs/reference/languages.md +++ b/docs/reference/languages.md @@ -46,6 +46,7 @@ const highlighter = createHighlighter({ | `perl` | `@tanstack/highlight/languages/perl` | `pl` | | `php` | `@tanstack/highlight/languages/php` | None | | `plaintext` | `@tanstack/highlight/languages/plaintext` | `text`, `txt`, `-->` | +| `powershell` | `@tanstack/highlight/languages/powershell` | `pwsh`, `ps1` | | `python` | `@tanstack/highlight/languages/python` | `py` | | `ruby` | `@tanstack/highlight/languages/ruby` | `rb` | | `rust` | `@tanstack/highlight/languages/rust` | `rs` | @@ -61,6 +62,6 @@ const highlighter = createHighlighter({ | `vue` | `@tanstack/highlight/languages/vue` | None | | `yaml` | `@tanstack/highlight/languages/yaml` | `yml` | -`@tanstack/highlight/languages` re-exports `apache`, `cmake`, `cpp`, `csharp`, `css`, `dart`, `diff`, `dockerfile`, `ejs`, `env`, `go`, `html`, `http`, `java`, `js`, `json`, `jsx`, `kotlin`, `lua`, `markdown`, `mermaid`, `nginx`, `perl`, `php`, `plaintext`, `python`, `ruby`, `rust`, `scheme`, `shell`, `sql`, `svelte`, `swift`, `toml`, `ts`, `tsrx`, `tsx`, `vue`, and `yaml`. The barrel is convenient but individual subpaths make bundle intent explicit. +`@tanstack/highlight/languages` re-exports `apache`, `cmake`, `cpp`, `csharp`, `css`, `dart`, `diff`, `dockerfile`, `ejs`, `env`, `go`, `html`, `http`, `java`, `js`, `json`, `jsx`, `kotlin`, `lua`, `markdown`, `mermaid`, `nginx`, `perl`, `php`, `plaintext`, `powershell`, `python`, `ruby`, `rust`, `scheme`, `shell`, `sql`, `svelte`, `swift`, `toml`, `ts`, `tsrx`, `tsx`, `vue`, and `yaml`. The barrel is convenient but individual subpaths make bundle intent explicit. See the [language support matrix](../language-support) for the context-aware behavior and current scope of each registration. diff --git a/docs/test-strategy.md b/docs/test-strategy.md index 8df900b..1d638f3 100644 --- a/docs/test-strategy.md +++ b/docs/test-strategy.md @@ -19,7 +19,7 @@ The suite protects the package's actual product boundary: valid code commonly pu ## Size Profiles -`pnpm run size` checks twenty-six independent browser profiles, including root helpers, language barrel imports, adapters, and themes. Each has minified, gzip, and Brotli budgets. The main highlighter profiles are: +`pnpm run size` checks twenty-seven independent browser profiles, including root helpers, language barrel imports, adapters, and themes. Each has minified, gzip, and Brotli budgets. The main highlighter profiles are: | Profile | Languages | Gzip budget | | --- | --- | ---: | @@ -27,13 +27,13 @@ The suite protects the package's actual product boundary: valid code commonly pu | TSX | TSX | 4.35 KB | | Octane | TypeScript plus Octane MDX adapter | 5.7 KB | | Docs | CSS, HTML, JS, JSON, JSX, Markdown, Shell, TS, TSX | 6.3 KB | -| All | All 39 definitions | 18.0 KB | +| All | All 40 definitions | 18.75 KB | The selective profiles are the primary product metric. The all-language profile protects the convenience entry from unbounded growth. Bundle graphs reject unexpected language or theme code. Package tests repeat isolation checks through public exports after building. ## Throughput -`pnpm run bench` measures highlighting, tokenization, Markdown, HAST, line numbers, long numbered blocks, long decorated blocks, and dedicated C#, C++, CMake, Dart, Java, Kotlin, Lua, Perl, PHP, Ruby, Rust, and Swift samples. Timings use the median of three samples after two warmup passes. Each profile has a 1.2 second CI budget; the main highlighting profile processes at least 10,000 blocks. +`pnpm run bench` measures highlighting, tokenization, Markdown, HAST, line numbers, long numbered blocks, long decorated blocks, and dedicated C#, C++, CMake, Dart, Java, Kotlin, Lua, Perl, PHP, PowerShell, Ruby, Rust, and Swift samples. Timings use the median of three samples after two warmup passes. Each profile has a 1.2 second CI budget; the main highlighting profile processes at least 10,000 blocks. `pnpm run compare:sugar-high` compares the overlapping JS/TS/JSX/TSX use case. `pnpm run compare:shiki` compares all supported fixtures. These are directional measurements, not claims of equivalent grammar depth. diff --git a/scripts/bench.mjs b/scripts/bench.mjs index 9e56deb..4aff1d6 100644 --- a/scripts/bench.mjs +++ b/scripts/bench.mjs @@ -56,6 +56,12 @@ try { outputBytes: htmlBytes, targetBlocks: 10_000, }, + powershell: { + fixtures: [{ rawLang: 'powershell', code: fs.readFileSync('test/showcases/Get-StationReport.ps1', 'utf8') }], + run: (fixture) => highlight(fixture.code, { lang: fixture.rawLang }), + observe: (result) => result.html.length, + targetBlocks: 2_000, + }, cpp: { fixtures: [{ rawLang: 'cpp', code: '#include \nconstexpr auto text = R"tag(// raw text)tag";\nint main() { std::vector values{1, 2, 3}; return values.size(); }' }], run: (fixture) => highlight(fixture.code, { lang: fixture.rawLang }), diff --git a/scripts/generate-visual-compare.mjs b/scripts/generate-visual-compare.mjs index c822c9a..90238ae 100644 --- a/scripts/generate-visual-compare.mjs +++ b/scripts/generate-visual-compare.mjs @@ -84,6 +84,8 @@ function normalizeShikiLanguage(lang) { 'js-vue': 'javascript', jsonc: 'json', md: 'markdown', + pwsh: 'powershell', + ps1: 'powershell', sh: 'bash', shell: 'bash', text: 'plaintext', @@ -96,8 +98,14 @@ function normalizeShikiLanguage(lang) { } function selectFixtures(fixtures) { - const selected = [] - const seen = new Set() + const selected = [{ + lang: 'powershell', + rawLang: 'powershell', + file: 'test/showcases/Get-StationReport.ps1', + line: 1, + code: fs.readFileSync('test/showcases/Get-StationReport.ps1', 'utf8'), + }] + const seen = new Set(['powershell']) for (const fixture of fixtures) { if (seen.has(fixture.lang)) continue @@ -208,6 +216,13 @@ function buildHtml(samples) { background: #0d1117; color: #e6edf3; } + .dark .shiki, .dark .shiki span { + color: var(--shiki-dark) !important; + background-color: var(--shiki-dark-bg) !important; + font-style: var(--shiki-dark-font-style) !important; + font-weight: var(--shiki-dark-font-weight) !important; + text-decoration: var(--shiki-dark-text-decoration) !important; + } .missing-shiki { border: 1px dashed var(--panel-border); color: var(--muted); diff --git a/scripts/language-utils.mjs b/scripts/language-utils.mjs index b7b4d36..9d166fc 100644 --- a/scripts/language-utils.mjs +++ b/scripts/language-utils.mjs @@ -27,6 +27,7 @@ export const supportedLanguages = [ 'perl', 'php', 'plaintext', + 'powershell', 'python', 'ruby', 'rust', @@ -54,6 +55,8 @@ const aliases = { kt: 'kotlin', kts: 'kotlin', pl: 'perl', + pwsh: 'powershell', + ps1: 'powershell', rb: 'ruby', rs: 'rust', '-->': 'plaintext', diff --git a/scripts/measure-size.mjs b/scripts/measure-size.mjs index 25b996b..ddcd08d 100644 --- a/scripts/measure-size.mjs +++ b/scripts/measure-size.mjs @@ -179,13 +179,22 @@ const profiles = { languages: ['swift'], limits: { minified: 7_450, gzip: 3_550, brotli: 3_250 }, }, + powershell: { + source: ` + import { createHighlighter } from './src/core.ts' + import { powershell } from './src/languages/powershell.ts' + globalThis.highlighter = createHighlighter({ languages: [powershell] }) + `, + languages: ['powershell'], + limits: { minified: 7_400, gzip: 3_400, brotli: 3_150 }, + }, all: { source: ` import { defaultHighlighter } from './src/index.ts' globalThis.highlighter = defaultHighlighter `, languages: 'all', - limits: { minified: 53_150, gzip: 18_000, brotli: 16_200 }, + limits: { minified: 55_800, gzip: 18_750, brotli: 16_900 }, }, reactAdapter: { source: `export * from './src/react.ts'`, diff --git a/scripts/test-package.mjs b/scripts/test-package.mjs index 0a38272..f659e64 100644 --- a/scripts/test-package.mjs +++ b/scripts/test-package.mjs @@ -95,6 +95,11 @@ const isolatedBundles = [ source: `import { createHighlighter } from '@tanstack/highlight/core'; import { tsx } from '${entry}'; globalThis.highlighter = createHighlighter({ languages: [tsx] })`, languages: ['tsx'], })), + ...['@tanstack/highlight/languages/powershell', '@tanstack/highlight/languages'].map((entry) => ({ + name: `${entry} PowerShell`, + source: `import { createHighlighter } from '@tanstack/highlight/core'; import { powershell } from '${entry}'; globalThis.highlighter = createHighlighter({ languages: [powershell] })`, + languages: ['powershell'], + })), ...['react', 'markdown', 'remark', 'rehype', 'octane'].map((entry) => ({ name: `${entry} adapter`, source: `export * from '@tanstack/highlight/${entry}'`, diff --git a/skills/configure-selective-highlighting/references/languages.md b/skills/configure-selective-highlighting/references/languages.md index 5650227..2aa4519 100644 --- a/skills/configure-selective-highlighting/references/languages.md +++ b/skills/configure-selective-highlighting/references/languages.md @@ -29,6 +29,7 @@ Import only the definitions the application registers. | Perl | `perl` | `@tanstack/highlight/languages/perl` | `pl` | | PHP | `php` | `@tanstack/highlight/languages/php` | - | | Plaintext | `plaintext` | `@tanstack/highlight/languages/plaintext` | `text`, `txt`, `-->` | +| PowerShell | `powershell` | `@tanstack/highlight/languages/powershell` | `pwsh`, `ps1` | | Python | `python` | `@tanstack/highlight/languages/python` | `py` | | Ruby | `ruby` | `@tanstack/highlight/languages/ruby` | `rb` | | Rust | `rust` | `@tanstack/highlight/languages/rust` | `rs` | diff --git a/src/index.ts b/src/index.ts index 452f91f..4e2253a 100644 --- a/src/index.ts +++ b/src/index.ts @@ -30,6 +30,7 @@ import { nginx } from './languages/nginx.js' import { perl } from './languages/perl.js' import { php } from './languages/php.js' import { plaintext } from './languages/plaintext.js' +import { powershell } from './languages/powershell.js' import { python } from './languages/python.js' import { ruby } from './languages/ruby.js' import { rust } from './languages/rust.js' @@ -71,6 +72,7 @@ export type HighlightLanguage = | 'perl' | 'php' | 'plaintext' + | 'powershell' | 'python' | 'ruby' | 'rust' @@ -152,6 +154,7 @@ export const allLanguages = [ perl, php, plaintext, + powershell, python, ruby, rust, diff --git a/src/languages/index.ts b/src/languages/index.ts index 07d06f4..6eac78a 100644 --- a/src/languages/index.ts +++ b/src/languages/index.ts @@ -23,6 +23,7 @@ export { nginx } from './nginx.js' export { perl } from './perl.js' export { php } from './php.js' export { plaintext } from './plaintext.js' +export { powershell } from './powershell.js' export { python } from './python.js' export { ruby } from './ruby.js' export { rust } from './rust.js' diff --git a/src/languages/powershell.ts b/src/languages/powershell.ts new file mode 100644 index 0000000..c53c294 --- /dev/null +++ b/src/languages/powershell.ts @@ -0,0 +1,93 @@ +import { defineLanguage, type TokenRange } from '../core.js' +import { patternTokenizer } from '../internal/patterns.js' + +export const powershell = defineLanguage({ + name: 'powershell', + aliases: ['pwsh', 'ps1'], + tokenize: patternTokenizer([ + { collect: collectPowerShellLexicalRanges }, + { className: 'literal', regex: /\$(?:true|false|null)\b/gi }, + { className: 'variable', regex: /[$@](?:[\p{L}\p{N}_?]+:)?[\p{L}\p{N}_?]+|\$[$^]/gu }, + { className: 'type', regex: /\[(?:[A-Za-z_]\w*\.)*[A-Za-z_]\w*(?:\[\])?\]/g }, + { className: 'keyword', regex: /(?|&]=?/g }, + ]), +}) + +function collectPowerShellLexicalRanges(code: string) { + const ranges: Array = [] + for (let index = 0; index < code.length;) { + const start = index + let end = index + let className: TokenRange['className'] = 'string' + if (code[index] === '`') end = Math.min(index + 2, code.length) + else if (code.startsWith('${', index)) { + end = index + 2 + while (end < code.length && code[end] !== '}') { + end += code[end] === '`' ? 2 : 1 + } + end = Math.min(end + 1, code.length) + className = 'variable' + } else if (code.startsWith('<#', index)) { + const close = code.indexOf('#>', index + 2) + end = close < 0 ? code.length : close + 2 + className = 'comment' + } else if (code[index] === '#' && (index === 0 || /[\s(){}\[\];'"|&]/.test(code[index - 1]))) { + end = index + 1 + while (end < code.length && !/[\r\n]/.test(code[end])) end++ + className = 'comment' + } else if (code[index] === '@' && /['"]/.test(code[index + 1] || '') && /^[ \t]*(?:\r\n|\r|\n)/.test(code.slice(index + 2))) { + const quote = code[index + 1] + const close = new RegExp(`^${quote}@`, 'gm') + close.lastIndex = index + 2 + const match = close.exec(code) + end = match ? match.index + 2 : code.length + } else if (code[index] === '"' || code[index] === "'") { + end = quotedEnd(code, index) + } + if (end > start) { + ranges.push({ start, end, className }) + index = end + } else index++ + } + return ranges +} + +// Keep nested quoted strings in expandable-string subexpressions protected. +function quotedEnd(code: string, start: number, depth = 0): number { + const quote = code[start] + let index = start + 1 + while (index < code.length) { + if (quote === '"' && code[index] === '`') index += 2 + else if (code[index] === quote) { + index++ + if (code[index] === quote) index++ + else return index + } else if (quote === '"' && code.startsWith('$(', index) && depth < 24) { + let balance = 1 + index += 2 + while (index < code.length && balance) { + if (code.startsWith('<#', index)) { + const close = code.indexOf('#>', index + 2) + index = close < 0 ? code.length : close + 2 + } else if (code[index] === '#' && /[\s(){}\[\];'"|&]/.test(code[index - 1])) { + while (index < code.length && !/[\r\n]/.test(code[index])) index++ + } else if (code[index] === '`') index += 2 + else if (code[index] === '"' || code[index] === "'") index = quotedEnd(code, index, depth + 1) + else { + if (code[index] === '(') balance++ + if (code[index] === ')') balance-- + index++ + } + } + } else index++ + } + return Math.min(index, code.length) +} diff --git a/src/languages/swift.ts b/src/languages/swift.ts index f5f6a9f..269ef3f 100644 --- a/src/languages/swift.ts +++ b/src/languages/swift.ts @@ -10,7 +10,7 @@ const patterns = [ // Contextual words are keywords only before declarations, types, or as accessors. { className: 'keyword', - regex: /(? = [ + { + lang: 'pwsh', normalized: 'powershell', + code: readFileSync(new URL('./showcases/Get-StationReport.ps1', import.meta.url), 'utf8'), + expectedClasses: ['th-keyword', 'th-type', 'th-variable', 'th-string', 'th-comment', 'th-literal', 'th-command', 'th-number', 'th-function', 'th-operator', 'th-property'], + }, { lang: 'php', normalized: 'php', diff --git a/test/powershell.test.ts b/test/powershell.test.ts new file mode 100644 index 0000000..7e8d401 --- /dev/null +++ b/test/powershell.test.ts @@ -0,0 +1,95 @@ +import { readFileSync } from 'node:fs' +import { describe, expect, it } from 'vitest' +import { createHighlighter } from '../src/core' +import { normalizeLanguage } from '../src/index' +import { powershell } from '../src/languages/powershell' +import { markdown } from '../src/languages/markdown' +import { tokenClasses } from './token-classes' + +const highlighter = createHighlighter({ languages: [powershell, markdown] }) +const classes = (code: string, text: string, lang = 'pwsh') => tokenClasses(highlighter, code, text, lang) + +describe('PowerShell documentation syntax', () => { + it('scans unterminated braced variables without backtracking', () => { + for (const [code, expected] of [ + ['${'.repeat(50_000), ['variable']], + ['"' + '${'.repeat(50_000), ['string']], + ['"${'.repeat(20_000), ['string', 'variable']], + ] as const) { + expect(classes(code, code)).toEqual(expected) + } + }, 1000) + it('preserves braced names while leaving escaped dollar signs literal', () => { + for (const text of ['${my var}', '${env:ProgramFiles(x86)}', '${a`}b}']) { + expect(classes(`${text} = 1`, text)).toEqual(['variable']) + } + expect(classes('`${x}', '${x}')).not.toContain('variable') + }) + it('protects escaped unbraced variables and literals', () => { + for (const [text, className] of [ + ['$HOME', 'variable'], ['$env:HOME', 'variable'], ['$?', 'variable'], + ['$$', 'variable'], ['$true', 'literal'], ['$false', 'literal'], ['$null', 'literal'], + ]) { + expect(classes('Write-Output escaped`' + text, text)).not.toContain(className) + expect(classes('Write-Output ' + text, text)).toEqual([className]) + expect(classes('Write-Output ``' + text, text)).toEqual([className]) + } + expect(classes('Write-Output "escaped`$HOME"', '"escaped`$HOME"')).toEqual(['string']) + expect(classes('Write-Output `', '`')).toEqual(['string']) + }) + it('keeps member names out of keywords while preserving statements', () => { + for (const text of ['End', 'Process', 'Begin', 'Data']) { + expect(classes(`$job.${text}`, text)).toEqual(['property']) + expect(classes(`${text} {}`, text)).toEqual(['keyword']) + } + }) + it('protects quoted escapes, doubled quotes and comment boundaries', () => { + for (const text of ["'Today''s # forecast'", '"quote `" # text"', '"C:\\cache\\"']) expect(classes(`$x = ${text}`, text, 'ps1')).toEqual(['string']) + expect(classes('Write-Output hello#there # comment', '#there', 'pwsh')).not.toContain('comment') + expect(classes('Write-Output hello#there # comment', '# comment', 'pwsh')).toEqual(['comment']) + expect(classes('Write-Output escaped`#hash', '#hash', 'pwsh')).not.toContain('comment') + expect(classes('<# "text" $true #> $false', '<# "text" $true #>', 'pwsh')).toEqual(['comment']) + }) + it('protects nested expandable-string expressions and braced variable names', () => { + const text = '\"Result: $(Get-Item -Path \"coast\").Name\"' + expect(classes(`$x = ${text}`, text, 'pwsh')).toEqual(['string']) + const commented = '"Value: $(1 <# ) " #> + 2)"' + expect(classes(`$x = ${commented}`, commented, 'pwsh')).toEqual(['string']) + const variable = '${name"#with`}"characters}' + expect(classes(`${variable} = 1`, variable, 'pwsh')).toEqual(['variable']) + }) + it('matches here-string terminators only at the start of a line', () => { + for (const quote of ['"', "'"]) { + const text = `@${quote} \t\r\n# literal $x\r\n ${quote}@ still text\r\n${quote}@` + expect(classes(`$x = ${text}\r\nGet-Item`, text, 'powershell')).toEqual(['string']) + expect(classes(`$x = ${text}\r\nGet-Item`, 'Get-Item', 'powershell')).toEqual(['command']) + } + }) + it('recognizes case-insensitive keywords, variables, parameters and operators', () => { + const code = 'FUNCTION Get-Report { PARAM([string[]] $Name) $script:Count = $ENV:HOME; ${a-b} = @params; IF ($TRUE -AND $? -cnotmatch "x") { Get-Item -LiteralPath $Name } }' + for (const word of ['FUNCTION', 'PARAM', 'IF']) expect(classes(code, word, 'pwsh')).toEqual(['keyword']) + for (const word of ['$script:Count', '$ENV:HOME', '${a-b}', '@params', '$?']) expect(classes(code, word, 'pwsh')).toEqual(['variable']) + for (const command of ['ForEach-Object', 'Test-Match', 'Compare-In']) expect(classes(command, command, 'pwsh')).toEqual(['command']) + for (const word of ['-AND', '-cnotmatch']) expect(classes(code, word, 'pwsh')).toEqual(['operator']) + expect(classes(code, '$TRUE', 'pwsh')).toEqual(['literal']) + expect(classes(code, '-LiteralPath', 'pwsh')).toEqual(['property']) + expect(classes(code, '[string[]]', 'pwsh')).toEqual(['type']) + expect(classes(code, 'Get-Report', 'pwsh')).toEqual(['function']) + expect(classes('1..4', '4', 'pwsh')).toEqual(['number']) + for (const number of ['64MB', '0xFF', '0b1010', '1.2e3', '42UL']) expect(classes(`$x = ${number}`, number, 'pwsh')).toEqual(['number']) + }) +}) + +it('registers aliases, isolates imports and delegates Markdown fences', () => { + for (const name of ['powershell', 'pwsh', 'ps1']) expect(normalizeLanguage(name)).toBe('powershell') + expect(createHighlighter({ languages: [markdown] }).normalizeLanguage('pwsh')).toBe('plaintext') + for (const lang of ['powershell', 'pwsh', 'ps1']) { + expect(classes(`\`\`\`${lang}\nGet-Item $HOME\n\`\`\``, 'Get-Item', 'markdown')).toEqual(['command']) + } +}) + +it('keeps the PowerShell guide identical to its canonical showcase', () => { + const guide = readFileSync(new URL('../docs/guides/powershell.md', import.meta.url), 'utf8') + const showcase = readFileSync(new URL('./showcases/Get-StationReport.ps1', import.meta.url), 'utf8') + expect(guide.match(/```powershell\n([\s\S]*?)\n```/)?.[1]).toBe(showcase.trimEnd()) +}) diff --git a/test/real-doc-fixtures.test.ts b/test/real-doc-fixtures.test.ts index bdad25b..27fd6bb 100644 --- a/test/real-doc-fixtures.test.ts +++ b/test/real-doc-fixtures.test.ts @@ -11,7 +11,7 @@ type DocFixture = { } const fixtures = fixtureData.fixtures as Array -const languagesWithoutRealDocFixtures = new Set(['go', 'cpp', 'cmake', 'php', 'csharp', 'dart', 'java', 'kotlin', 'lua', 'perl', 'ruby', 'rust', 'swift']) +const languagesWithoutRealDocFixtures = new Set(['go', 'cpp', 'cmake', 'php', 'csharp', 'dart', 'java', 'kotlin', 'lua', 'perl', 'ruby', 'rust', 'swift', 'powershell']) describe('real TanStack docs fixtures', () => { it('covers normalized language targets available in TanStack docs', () => { diff --git a/test/showcases/Get-StationReport.ps1 b/test/showcases/Get-StationReport.ps1 new file mode 100644 index 0000000..415d0ab --- /dev/null +++ b/test/showcases/Get-StationReport.ps1 @@ -0,0 +1,53 @@ +#Requires -Version 7.4 +<# Build typed records, then shape a pipeline for display. + Quotes and $variables in this comment stay quiet. #> +function Get-StationReport { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [ValidateSet('coast', 'ridge')] + [string[]] $Station + ) + + begin { + $script:Endpoint = $env:STATION_API ?? 'https://example.com' + $headers = @{ Accept = 'application/json' } + $budget = 64MB + } + process { + foreach ($name in $Station) { + $request = @{ + Uri = "$script:Endpoint/observations/$name" + Headers = $headers + ErrorAction = 'Stop' + } + try { + $reading = Invoke-RestMethod @request + [pscustomobject]@{ + Station = $name + Celsius = [double] $reading.celsius + Online = $true + } + } + catch { + Write-Warning -Message "Station $name unavailable: $_" + } + } + } + end { + $banner = @" +Forecast ready +Budget: $budget +"@ + $literal = @' +$HOME is literal; # this is text, not a comment. +'@ + Write-Verbose -Message ($banner + $literal) + } +} + +${report-title} = 'Today''s observatory' +Get-StationReport -Station coast, ridge | + Where-Object { $_.Online -and $_.Celsius -ge 0 } | + Sort-Object -Property Celsius -Descending | + Select-Object -First 5 -Property Station, Celsius diff --git a/test/swift.test.ts b/test/swift.test.ts index 4fff0f6..c536c68 100644 --- a/test/swift.test.ts +++ b/test/swift.test.ts @@ -369,6 +369,16 @@ let package = Package( expect(classes(code, '.testTarget(name:')).toEqual([undefined, 'function', undefined, 'property', undefined]) }) + it('recognizes macro declarations with short names', () => { + for (const name of ['m', 'm']) { + const code = `public macro ${name}() = #externalMacro(module: "M", type: "T")` + expect(classes(code, 'macro')).toEqual(['keyword']) + expect(classes(code, '#externalMacro')).toEqual(['meta']) + } + expect(classes('let macro = value', 'macro')).not.toContain('keyword') + expect(classes('value.macro()', 'macro')).toEqual(['function']) + }) + it('registers only the swift name', () => { expect(highlighter.normalizeLanguage('swift')).toBe('swift') expect(createHighlighter({ languages: [] }).normalizeLanguage('swift')).toBe('plaintext')