Skip to content

Fix O(2^n) regex backtracking in consolidateBlockDiffs that freezes the extension host - #75

Open
wolframarnold wants to merge 2 commits into
phine-apps:mainfrom
wolframarnold:fix/consolidateblockdiffs-backtracking
Open

Fix O(2^n) regex backtracking in consolidateBlockDiffs that freezes the extension host#75
wolframarnold wants to merge 2 commits into
phine-apps:mainfrom
wolframarnold:fix/consolidateblockdiffs-backtracking

Conversation

@wolframarnold

Copy link
Copy Markdown
Contributor

Fixes #74.

Problem

consolidateBlockDiffs() in src/markdown/structuralDiff.ts builds a regex with a
(BLOCK(?:\s*BLOCK)*) shape that backtracks exponentially (O(2ⁿ) in the number
of block elements) when an <ins>/<del> region wraps a long run of sibling
block-level elements. On a real document edit this pegs one CPU core at 100% and
never returns, freezing the single-threaded VS Code extension host — and with it
every extension in that window. Full analysis, repro, and measurements are in #74.

Root cause

const blockElementPattern =
  `(?:<(?:${blockTags})[^>]*>[\\s\\S]*?<\\/(?:${blockTags})>|<(?:${selfClosingTags})[^>]*\\/?>)`;
const fullWrapRegex = new RegExp(
  `(<(ins|del)[^>]*>)\\s*(${blockElementPattern}(?:\\s*${blockElementPattern})*)\\s*(<\\/\\2>)`,
  "gi",
);

blockElementPattern is ambiguous — a lazy [\s\S]*? that closes on any block
tag — so a run of n blocks has ~2ⁿ partitions. Repeating it with
(?:\s*BLOCK)* and then requiring a trailing </\2> that usually doesn't align
makes the engine try them all. A synthetic 343-byte input already takes ~4 s
(runtime doubles per added block); ~40 blocks is effectively forever.

Fix

Match the region with a lazy inner (linear), then validate that the content is a
block run with a linear left-to-right scan (isBlockLevelRun). This removes the
nested repetition and the exponential blow-up while preserving behavior.

Verification

  • Isolated regex on the real captured 224 KB input: hang → 1 ms.
  • Full computeDiff() on the reproducing 129 KB document: hang → ~290 ms, valid
    diff HTML produced.
  • Spot checks: an <ins> wrapping a <table> + <h2> run is still marked
    diff-block; an <ins> around inline text is still left untouched.

Notes for review

  • The lazy inner matches to the first </ins>/</del>. This runs after
    balanceDiffTags, and the original \2 backreference had the same first-close
    behavior, so pairing should be equivalent — worth a test if diff tags can nest.
  • I could not run the repo's own suite in my environment (dev deps + the VS Code
    test harness weren't installed). Please run npm test; a regression fixture
    asserting consolidateBlockDiffs completes for, say, N=40 blocks would lock this
    in.
  • Considered an atomic group / possessive quantifier ((?>…) / *+) instead —
    V8 rejects both even on Node 24 (throws at construction), so that path would break
    the extension. The lazy-plus-validate form is portable.

🤖 Generated with Claude Code

…he extension host

The fullWrapRegex used a (BLOCK(?:\s*BLOCK)*) form whose block unit is ambiguous
(a lazy [\s\S]*? that closes on any block tag). When an <ins>/<del> region wraps a
long run of sibling block elements, the trailing </ins>/</del> usually fails to
align, so the engine explores ~2^n partitions before giving up. That pegs one core
and freezes the single-threaded extension host, taking down every extension in the
window. Reproduces on a real 129 KB document after an edit that inserts ~20+
contiguous block elements; a synthetic 343-byte input already takes ~4 s.

- Replace the nested-quantifier regex with a lazy
  (<(ins|del)\b[^>]*>)([\s\S]*?)(<\/\2>) match plus a linear isBlockLevelRun()
  validation in the callback.
- Behavior preserved: genuine block runs still get the diff-block class; inline
  content is still left untouched.
- Verified: isolated regex hang -> 1 ms; full computeDiff hang -> ~290 ms on the
  reproducing document.

Fixes phine-apps#74

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@phine-apps

Copy link
Copy Markdown
Owner

Hi @wolframarnold,

Thank you so much for another great PR and the thorough repro in #74! Eliminating the $O(2^n)$ backtracking is a huge win.

During review, I spotted one edge case with nested block elements (e.g. <blockquote><ul><li>...</li></ul></blockquote> or nested lists):
Because blockElementPattern uses a lazy close tag ([\s\S]*?<\/(?:blockTags)>), one.exec stops at the first inner close tag (e.g. </ul>), leaving the outer close tag unmatched and causing isBlockLevelRun to return false (missing diff-block).

Tracking tag depth (similar to findClosing in domUtils.ts) resolves this cleanly in $O(N)$ without backtracking.

How would you like to proceed?

  • Option A: You update the PR with depth tracking.
  • Option B: I push the follow-up commit & tests directly to your branch and merge.

Let me know your preference. Thanks again for tackling this!

Review follow-up for phine-apps#74/phine-apps#75. The previous isBlockLevelRun used a lazy
blockElementPattern whose close group matched any block tag, so it stopped at the
first inner </...> (e.g. </ul> inside <blockquote><ul>...</ul></blockquote>),
left the outer close tag unmatched, and returned false -- dropping the diff-block
class on nested block structures.

- Rewrite isBlockLevelRun to consume each paired block via findClosing (depth
  tracking, mirroring the domUtils helper); still O(N), no backtracking.
- Handle self-closing blocks (<hr>) explicitly; remove the now-unused
  blockElementPattern.
- Add unit tests: nested block/list get diff-block; inline content is untouched;
  a 40-block run with a non-block tail stays linear (the former O(2^n) case).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@wolframarnold

Copy link
Copy Markdown
Contributor Author

Thanks @phine-apps — good catch, that's exactly right. Went with Option A: isBlockLevelRun now consumes each paired block via findClosing (depth tracking) rather than the lazy-close pattern, so nested blocks like <blockquote><ul>...</ul></blockquote> are recognized as a single run and keep diff-block. Still O(N) — the old 2ⁿ shape (a long block run + non-block tail) stays linear. Added unit tests for the nested block/list cases, the inline-untouched case, and a linear-time guard. I couldn't run the full VS Code test harness locally, so a npm test on your side would be good to confirm nothing else regressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

consolidateBlockDiffs: catastrophic regex backtracking (O(2^n)) freezes the extension host on large multi-block edits

2 participants