Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fix-private-class-names.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@vortexlabs/kirari': patch
---

Fix JavaScript and TypeScript private class names such as `#count` being split into a `#` text token and a separate name. They are now one token, highlighted as a property, or as a function for private methods.
6 changes: 3 additions & 3 deletions docs/comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,16 @@ Competitors measured on 2026-10-21; Kirari working tree measured on 2026-10-22:

| Setup | Gzip bytes |
| --- | ---: |
| Kirari working tree, `highlight()` wrapper | 4,153 |
| Kirari working tree, `highlight()` wrapper | 4,197 |
| Shiki 4.5.0, synchronous core, JavaScript regex engine, JavaScript grammar, GitHub Dark | 73,530 |
| Prism 1.30.0, core, clike dependency, JavaScript grammar | 5,278 |
| highlight.js 11.12.0, core, JavaScript grammar | 11,056 |

The script validates that each bundled function produces highlighted HTML and
preserves an escaped source sample. It does not measure speed or grammar accuracy.

Kirari's full JavaScript entry is 4,252 bytes gzip; its full set of 48 language
definitions with core is 23,171 bytes. These come from `bun run size`. The wrapper
Kirari's full JavaScript entry is 4,294 bytes gzip; its full set of 48 language
definitions with core is 23,273 bytes. These come from `bun run size`. The wrapper
above is smaller because unused exports can be removed. Kirari measurements use
the current source, including unreleased changes; competitors use the named npm
versions. Rerun both commands when the source or bundler changes.
Expand Down
6 changes: 3 additions & 3 deletions packages/kirari/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -333,9 +333,9 @@ Measured on 2026-10-22 using Bun 1.3.4:

| Entry | Minified bytes | Gzip bytes |
| --- | ---: | ---: |
| Core API | 8,527 | 3,735 |
| JavaScript default | 9,700 | 4,252 |
| All 48 languages | 69,243 | 23,171 |
| Core API | 8,607 | 3,774 |
| JavaScript default | 9,780 | 4,294 |
| All 48 languages | 69,380 | 23,273 |

## Development

Expand Down
19 changes: 12 additions & 7 deletions packages/kirari/src/tokenize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -155,13 +155,13 @@ export function* scan(source: string, language: Language, start = 0, previous?:
position = end
}

const match = (pattern: RegExp): string | undefined => {
pattern.lastIndex = position
const match = (pattern: RegExp, at = position): string | undefined => {
pattern.lastIndex = at
const result = pattern.exec(source)
// Unicode regexes can rewind into a surrogate pair in V8.
if (!result || result.index !== position) return undefined
if (!result || result.index !== at) return undefined
if (result[0].length === 0) {
throw new Error(`Kirari: empty match in ${language.name} at offset ${position}`)
throw new Error(`Kirari: empty match in ${language.name} at offset ${at}`)
}
return result[0]
}
Expand Down Expand Up @@ -216,12 +216,16 @@ export function* scan(source: string, language: Language, start = 0, previous?:
frames.push({ kind: 'template', opening: true })
continue
}
let name = identifier && match(identifier)
// `#name` is a private class member: one name token, and never a variable.
const privateName = script && char === '#'
const start = privateName ? position + 1 : position
let name = identifier && match(identifier, start)
if (script && identifier) {
const end = escapedIdentifierEnd(source, position, position + (name?.length ?? 0))
if (end > position) name = source.slice(position, end)
const end = escapedIdentifierEnd(source, start, start + (name?.length ?? 0))
if (end > start) name = source.slice(start, end)
}
if (name !== undefined) {
if (privateName) name = `#${name}`
// Classify the name; earlier branches win.
let type = 'variable'
const end = position + name.length
Expand Down Expand Up @@ -251,6 +255,7 @@ export function* scan(source: string, language: Language, start = 0, previous?:
if (found.function) type = 'function'
if (found.pending) context.functionPending = true
}
if (privateName && type === 'variable') type = 'property'
context.annotation = false
emit(type, end)
continue
Expand Down
8 changes: 8 additions & 0 deletions packages/kirari/test/corpus/fixtures.ts
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,14 @@ export const corpus: readonly CorpusFixture[] = [
{ text: 'view', type: 'variable' }, { text: 'Box', type: 'tag' },
],
},
{
id: 'javascript-private-members', language: 'javascript',
source: 'class Counter {\n #count = 0;\n #step() { return this.#count++ }\n has(other) { return #count in other }\n}',
assertions: [
{ text: '#count', type: 'property', occurrence: 0 }, { text: '#count', type: 'property', occurrence: 1 },
{ text: '#count', type: 'property', occurrence: 2 }, { text: '#step', type: 'function' },
],
},
{
id: 'typescript-class-and-operators', language: 'typescript',
source: 'class Meter {\n private reading: number = 1_024;\n sample(): unknown { return this.reading ?? undefined; }\n}',
Expand Down
11 changes: 11 additions & 0 deletions packages/kirari/test/script-accuracy.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,16 @@ for (const language of [javascript, typescript, jsx, tsx]) {
const validPrefix = 'parcel'.repeat(500)
expect(pairs(`${validPrefix}\\u0020`, language)).toContainEqual(['variable', validPrefix])
})

test(`${language.name} keeps private class names as one member token`, () => {
const names = (source: string) => pairs(source, language).filter(token => token[1]!.startsWith('#'))
expect(names('class C { #p = 1; m() { this.#p } }')).toEqual([['property', '#p'], ['property', '#p']])
expect(names('class C { static #s; get #a() {} #m() {} n() { this.#m(); return #s in C } }'))
.toEqual([['property', '#s'], ['function', '#a'], ['function', '#m'], ['property', '#m'], ['property', '#s']])
expect(names('class C { #handler = () => null }')).toEqual([['function', '#handler']])
expect(names('class C { #\\u0070 = 1; m() { this?.#\\u0070 } }')).toEqual([['property', '#\\u0070'], ['property', '#\\u0070']])
expect(pairs('class C { # = 1 }', language)).toContainEqual(['text', ' # '])
})
}

for (const language of [typescript, tsx]) {
Expand Down Expand Up @@ -126,6 +136,7 @@ test('script fixes preserve every unfinished UTF-16 prefix and streaming token p
'type\nParcel\n=\n{ label: string };\nconst cargo\n:\nParcel\n= item;\nfunction finish()\n:\nvoid {}',
'const result = active\n ? other\n : Parcel;\nconst n = void execute()',
'class Box { v: T;\n /* member */ w: string; x: number }\ninterface I { v: T; w: Box }',
'class Counter {\n #count = 0;\n #step() {}\n add() { this.#step(); return #count in this }\n}',
'const node = <Grid<Map<string, Item[]>>\n rows={items} />;\nnext()',
'<Grid<Map<string, Item>>>= text</Grid>',
'<Grid<A<B>= C>',
Expand Down
1 change: 1 addition & 0 deletions packages/kirari/test/stream-fuzz.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ const scriptFragments = [
const scriptSources = [
'class Box {\n v: T;\n w: string;\n target: Handler = () => null;\n}\n',
'interface I { v: T; /* member */\n w: Box; }\n',
'class Counter {\n #count = 0;\n #step() {}\n add() { this.#step(); return #count in this }\n}\n',
'name\n\t(\nname\n)',
'{name\n\t: true, "key"\r\n: null}',
'return /[a\\/]*/g\nname()',
Expand Down
12 changes: 12 additions & 0 deletions sites/docs/changelog/kirari/1.1.3.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,18 @@ version: 1.1.3
date: 2026-10-22
---

### Breaking changes

#### Public object types are type aliases

`Token`, `Rule`, `Language`, `RenderOptions`, `Decoration`, `TokenPatch`, and `TokenStream` are now type aliases instead of interfaces. Their members are unchanged, but code that extends them through declaration merging (module augmentation) no longer compiles.

**How to migrate**

1. Replace `declare module` augmentations of these types with your own type that intersects them, such as `type AppToken = Token & { line: number }`.
2. No change is needed if you only import and use the types.

### Bug fixes

- Markdown heading tokens no longer include the trailing line break, so heading styles and line-based renderers stop at the end of the line.
- JavaScript and TypeScript private class names such as `#count` are now one token, highlighted as properties at both the declaration and `this.#count`, and as functions for private methods.
2 changes: 1 addition & 1 deletion sites/docs/content/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Modern ESM runtimes with ES2022 features are required. The library has no DOM, N

## Bundle measurements

The JavaScript entry measures 4,252 bytes gzip; all 48 definitions together measure 23,171 bytes gzip. Measured on 2026-10-22 with Bun 1.3.4 minification and gzip level 9, excluding CSS. These are library bundle sizes, not website sizes or throughput benchmarks.
The JavaScript entry measures 4,294 bytes gzip; all 48 definitions together measure 23,273 bytes gzip. Measured on 2026-10-22 with Bun 1.3.4 minification and gzip level 9, excluding CSS. These are library bundle sizes, not website sizes or throughput benchmarks.

To reproduce, run `bun run size` in the Kirari repository.

Expand Down
2 changes: 1 addition & 1 deletion sites/docs/src/lib/site.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,6 @@ export const site = {
x: 'https://x.com/intent/follow?screen_name=ugi_dev',
version: kirariPackage.version,
// Tree-shaken highlight() setup; measured by packages/benchmarks/compare-size.ts.
javascriptHighlightGzipBytes: 4153,
javascriptHighlightGzipBytes: 4197,
install: 'npm install @vortexlabs/kirari',
} as const
Loading