Allow CJK adjacency in emphasis delimiter runs

[JoshuaKirby88]

Addresses #650.

Problem

CommonMark's punctuation-based flanking rules can leave ordinary CJK emphasis visible as literal * characters. Japanese and Chinese generally do not add spaces at these boundaries, and Korean particles attach directly to the text they follow.

Before and after

Japanese

Markdown:

これは**「重要なこと」**です。

Current CommonMark HTML:

<p>これは**「重要なこと」**です。</p>

Proposed HTML:

<p>これは<strong>「重要なこと」</strong>です。</p>

Chinese

Markdown:

这是**我想做的事。**所以继续。

Current CommonMark HTML:

<p>这是**我想做的事。**所以继续。</p>

Proposed HTML:

<p>这是<strong>我想做的事。</strong>所以继续。</p>

Korean

Markdown:

문서를 작성할 때 **마크다운(Markdown)**을 사용하세요.

Current CommonMark HTML:

<p>문서를 작성할 때 **마크다운(Markdown)**을 사용하세요.</p>

Proposed HTML:

<p>문서를 작성할 때 <strong>마크다운(Markdown)</strong>을 사용하세요.</p>

Proposed rule

This PR defines a CJK character as an assigned Unicode character that meets either of these conditions:

• its East_Asian_Width property is W, F, or H, excluding characters with Emoji_Presentation;
• its Script property is Hangul.

A delimiter run adjacent to such a character can satisfy the punctuation portion of the left-flanking or right-flanking definition.

The existing whitespace conditions still apply. The existing restrictions on underscore delimiters also remain unchanged.

Conformance examples

The PR adds nine examples covering Japanese, Chinese, Korean, strong and single emphasis, ASCII punctuation inside CJK text, and protected behavior involving ambiguous-width punctuation, underscores, whitespace, and emoji presentation.

Five examples demonstrate newly supported emphasis. Four protect behavior that must remain unchanged.

Related work

tats-u/markdown-cjk-friendly provides proposed CommonMark amendments and implementations addressing the same CJK emphasis problem. Its specification includes closely related CJK character classification and emphasis-flanking rules, with additional handling for variation sequences.

Thanks to @tats-u for reviewing this proposal and identifying gaps in the test coverage.

Proof implementation

The complete cmark proof is available as an exact comparison against upstream cmark. It includes the implementation, regression tests, and a Unicode 17 range generator that pins and verifies every input file by SHA-256.

The classifier uses binary search over 68 non-overlapping ranges, requiring at most seven range comparisons for each adjacent code point.

Reproduce the verification

The following starts from public repositories and does not require this branch to be checked out already:

workdir=$(mktemp -d)

git clone https://github.com/commonmark/commonmark-spec.git "$workdir/spec"
git -C "$workdir/spec" fetch origin pull/839/head:pr-839
git -C "$workdir/spec" switch pr-839

git clone https://github.com/JoshuaKirby88/cmark.git "$workdir/cmark"
git -C "$workdir/cmark" checkout 836a2757facef61aea2dad3cf3573123b3bf1793

python3 "$workdir/cmark/tools/generate_cjk_ranges.py" --check
make -C "$workdir/cmark" test

python3 "$workdir/spec/test/spec_tests.py" \
  --spec "$workdir/spec/spec.txt" \
  --program "$workdir/cmark/build/src/cmark --unsafe"

Expected results:

• the generated Unicode 17 table matches all 68 checked-in ranges;
• all nine native cmark test groups pass;
• all 664 examples in this proposed specification pass.

[Copilot]

Pull request overview

Updates the CommonMark specification to allow emphasis delimiter runs to open/close when adjacent to CJK characters, addressing cases where CJK punctuation/adjacency previously caused literal * to remain visible.

Changes:

• Defines a new “CJK character” based on Unicode East_Asian_Width / Script properties (excluding Emoji_Presentation).
• Extends left-/right-flanking delimiter run definitions to permit a CJK-adjacency alternative when punctuation would otherwise block emphasis.
• Adds conformance examples covering new supported cases and protected non-changes (underscores, whitespace, ambiguous-width punctuation, emoji presentation).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jgm]

@tats-u I would be grateful for your review of this.

[tats-u]

This is essentially, to put it nicely, an ultra-simplified version, or to put it bluntly, a poor man's copy of https://github.com/tats-u/markdown-cjk-friendly. It becomes equivalent to that if you remove the support for variation sequences before delimiter runs.

While it is commendable for handling many cases with minimal effort, it is regrettable that the PR description makes no mention of my repository. Also, although cases that cannot be covered by this are rare, Japanese or Chinese people who are knowledgeable about Unicode could easily list them.

The test cases lack coverage for at least the following:

• Delimiter runs sandwiched between non-CJK characters and characters with an East Asian Width of H (e.g., half-width katakana) or F (full-width parentheses, commas, exclamation marks, and question marks)
• Hangul in NFD representation (parts of Hangul whose East Asian Width is N)
• Supplementary characters (characters U+10000 or higher counted as two characters in UTF-16)
• CJK characters that are emoji characters but not default emoji presentation characters

[tats-u]

By the way, do CommonMark maintainers like you consider the change in 0.31.2, where emojis are now treated as punctuation (meaning, for example, a**😀**a is no longer recognized as bold), to be an expected change?

[JoshuaKirby88]

Thanks for reviewing and identifying missing tests.

Your implementation is relevant and I should've acknowledged it.

My aim was to address the behavior in CommonMark itself, since this still affects many widely used renderers like cmark, cmark-gfm, markdown-it, micromark, and Marked.
My view is that extensions can address it in apps that adopt them, but doesn't make it something authors can rely on across renderers.
So I think we should move toward making it part of the shared spec and default implementation, which makes your work relevant to this, rather than an alternative.

I'll add coverage for the cases you listed and look more closely at the differences in variation-sequence handling.

[jgm]

@tats-u would you like to prepare an alternative PR based on your work?

By the way, do CommonMark maintainers like you consider the change in 0.31.2, where emojis are now treated as punctuation (meaning, for example, a😀a is no longer recognized as bold), to be an expected change?

Just built 0.31.1, and it has same behavior. Are you sure it used to work differently?