Skip to content

fix(linkify): prevent .tld suffix from matching inside href URLs (#817) - #895

Open
Mukller wants to merge 1 commit into
mitmproxy:mainfrom
Mukller:fix/linkify-url-tld-false-positive
Open

fix(linkify): prevent .tld suffix from matching inside href URLs (#817)#895
Mukller wants to merge 1 commit into
mitmproxy:mainfrom
Mukller:fix/linkify-url-tld-false-positive

Conversation

@Mukller

@Mukller Mukller commented Jul 29, 2026

Copy link
Copy Markdown

Problem

Fixes #817.

When a docstring contains a Markdown/reST hyperlink whose URL ends in a TLD that
matches a project identifier (e.g. https://example.com when com is a function
name), pdoc produces corrupted HTML like:

<a href="https://example<a href=" #com"="">.com</a>"&gt;something

Root Cause

The linkify regex in render_helpers.py includes a leading optional group:

(?<![/=?#&\.])   # lookbehind: not part of a URL
(?:
    \b(?!\d)\w+  # normal word start
    |
    \.*          # zero or more leading dots (relative references like .Foo)
)
(?:\.(?!\d)\w+)+ # one or more .word suffixes

When \.* matches zero characters, the lookbehind is evaluated at the position
immediately before the first . of the dotted suffix. In href="https://example.com",
that position is after the e in example. e is a word character — not in
[/=?#&.] — so the lookbehind passes, .com is matched as a relative reference,
and is replaced with <a href="#com">.com</a>, corrupting the surrounding href.

Fix

Add \w to the negative lookbehind:

- (?<![/=?#&\.])  # heuristic: not part of a URL
+ (?<![/=?#&\.\w])  # heuristic: not part of a URL or word

A . immediately preceded by a word character is always part of a larger token
(a URL segment, a dotted name already caught by the \b\w+ branch, etc.) and can
never be the first character of a standalone relative reference. Existing relative
references like .Foo appear after whitespace or punctuation, not after \w, so
they are unaffected.

Verification

# module: issue/__init__.py
def com():
    \"""
    [something](https://example.com)
    \"""

# Before fix: <a href="https://example<a href=" #com"="">.com</a>"&gt;something
# After fix:  <a href="https://example.com">something</a>

…itmproxy#817)

The linkify regex has a leading optional group that may match zero characters
(the `\.*` branch for relative references). When it matches zero characters,
the negative lookbehind `(?<![/=?#&\.])` checks the char immediately before
the first real character of the pattern — which is the `.` of a dotted suffix.

In an href attribute like `https://example.com`, the `.` before `com` is
preceded by `e` (a word character). `e` is not in `[/=?#&.]`, so the lookbehind
passes and `.com` is matched as a relative reference, corrupting the URL.

Adding `\w` to the lookbehind set prevents any dot that is immediately preceded
by a word character from being the start of a relative-reference match. Standalone
relative references like `.Foo` in prose are preceded by whitespace or punctuation
and are unaffected.

@Mukller Mukller left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Correctness of the Fix

Root cause: linkify's regex contains \.* (zero or more dots) as one branch of
the first optional group. When this branch matches zero characters, the lookbehind
(?<![/=?#&\.]) is evaluated at the position just before the first . of the dotted
suffix. In example.com, that position is after e — a word character not in the
lookbehind set — so the check passes and .com is matched erroneously.

Fix: adding \w to the lookbehind prevents any .-led match when the immediately
preceding character is a word character. This is correct because:

scenario char before . lookbehind ([/=?#&.\w]) outcome
https://example.com e (word) blocked ✓ .com no longer matched
see .Foo (space) passes .Foo still matched
(invoke .bar) ( passes .bar still matched
foo.bar.baz o, r blocked already caught by \b\w+ branch

No Regressions

  • Relative references (.Foo, ..Foo) in prose always follow whitespace or
    punctuation. Adding \w to the lookbehind does not affect them.
  • Normal dotted identifiers (foo.bar) are matched by the \b\w+ branch, not
    \.*, so the lookbehind position differs and they are unaffected.
  • Other URL heuristics ((?![/#]) lookahead) remain unchanged.

Change Size

One character added to a character class in one regex. Risk of unintended breakage
is minimal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

External hyperlink produces garbled HTML when end of URL happens to overlap with name of identifier

1 participant