Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions changes.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
Change Log
==========

**Changes in next version**

* Added :meth:`Page.search_for_fuzzy` for edit-distance substring searches,
returning matched text, distance and quads without additional dependencies.


**Changes in version 2.1** (2026-11-02)

Expand Down
58 changes: 58 additions & 0 deletions docs/page.rst
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ In a nutshell, this is what you can do with PyMuPDF:
:meth:`Page.remove_rotation` PDF only: set page rotation to 0
:meth:`Page.replace_image` PDF only: replace an image
:meth:`Page.search_for` search for a string
:meth:`Page.search_for_fuzzy` search for approximate text occurrences
:meth:`Page.set_artbox` PDF only: modify `/ArtBox`
:meth:`Page.set_bleedbox` PDF only: modify `/BleedBox`
:meth:`Page.set_cropbox` PDF only: modify the :data:`cropbox` (visible page)
Expand Down Expand Up @@ -2158,6 +2159,63 @@ In a nutshell, this is what you can do with PyMuPDF:
|history_end|


.. method:: search_for_fuzzy(needle, *, max_edits, clip=None, textpage=None)

Search for approximate occurrences of *needle*, using Levenshtein edit
distance. No additional package is required. This is a separate Python
implementation; :meth:`search_for` remains the faster choice for exact text.

:arg str needle: Text to search for. Unicode case folding is applied and
runs of whitespace become one space. Leading and trailing query
whitespace is ignored. A hyphen followed by a line break between letters
is removed within a text block. Other punctuation, signs and digits are
preserved. Empty or whitespace-only queries return an empty list.
:arg int max_edits: Required maximum number of insertions, deletions and
substitutions. Must be non-negative and smaller than the normalized
query length. Transposing adjacent characters costs two edits.
:arg rect_like clip: Restrict text extraction to this rectangle.
:arg textpage: Reuse a :ref:`TextPage` belonging to this page. As with
:meth:`search_for`, *clip* is ignored when this is supplied. Text already
removed or dehyphenated in this TextPage cannot be restored.
:rtype: list[dict]
:returns: One dictionary per selected occurrence, in extraction order:

* ``text``: the matched original text, retaining case, punctuation and
extracted line breaks.
* ``distance``: the edit count after normalization, not a confidence score.
* ``quads``: a list of :ref:`Quad` objects in unrotated page coordinates,
one per matched span fragment. Multiple lines stay separate.

Both exact and approximate occurrences are returned. At each candidate end,
the earliest start with the lowest distance is retained. Overlaps are then
resolved by lowest distance, closest length to the normalized query, and
earliest position. Adjacent occurrences remain separate.
Matches start and end at original character boundaries, even when case
folding expands a character (for example, ``ß`` to ``ss``). This is
substring search; candidates are not expanded to whole words.

Text follows extraction order, without reconstructing columns or tables.
OCR is not run automatically, but an existing OCR TextPage can be supplied.
Exact query pieces first restrict the candidate regions. Searching with a
positive edit budget still takes O(NM) time in the worst case for N page
characters and M query characters, plus selection and geometry extraction.
Reusing a TextPage avoids extraction from the PDF, but character dictionaries
and the search representation are still constructed for each call.

Example of falling back to approximate search::

quads = page.search_for(quote, quads=True)
if quads:
page.add_highlight_annot(quads)
else:
for match in page.search_for_fuzzy(quote, max_edits=2):
print(match["text"], match["distance"])
page.add_highlight_annot(match["quads"])

Approximate matches are candidates for review. An edit can change a number,
sign or negation; a small distance does not establish equivalent meaning.


.. method:: set_mediabox(r)

PDF only: Change the physical page dimension by setting :data:`mediabox` in the page's object definition.
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -661,6 +661,7 @@ def build():
ret.append( (f'{g_root}/src/__main__.py', to_dir) )
ret.append( (f'{g_root}/src/pymupdf.py', to_dir) )
ret.append( (f'{g_root}/src/table.py', to_dir) )
ret.append( (f'{g_root}/src/_textsearch.py', to_dir) )
ret.append( (f'{g_root}/src/_table_refine.py', to_dir) )
ret.append( (f'{g_root}/src/_table_spans.py', to_dir) )
ret.append( (f'{g_root}/src/_table_union.py', to_dir) )
Expand Down
16 changes: 16 additions & 0 deletions src/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13670,6 +13670,19 @@ def search_for(
del tp
return rlist

def search_for_fuzzy(self, needle, *, max_edits, clip=None, textpage=None) -> list:
"""Find approximate text occurrences with their edit distance and quads.

`max_edits` bounds insertions, deletions and substitutions after Unicode
case-folding, whitespace folding and line-end dehyphenation. Each result
is a dict with `text`, `distance` and `quads`. See the API documentation
for overlap selection and text extraction limitations.
"""
from ._textsearch import search_for_fuzzy
return search_for_fuzzy(
self, needle, max_edits=max_edits, clip=clip, textpage=textpage
)

def set_artbox(self, rect):
"""Set the ArtBox."""
return self._set_pagebox("ArtBox", rect)
Expand Down Expand Up @@ -17233,6 +17246,7 @@ def _getNewBlockList(self, page_dict, raw):
JM_make_textpage_dict(self.this, page_dict, raw)

def _textpage_dict(self, raw=False):
# raw: 0 for span text, 1 for characters, 2 for characters with quads.
page_dict = {"width": self.rect.width, "height": self.rect.height}
self._getNewBlockList(page_dict, raw)
return page_dict
Expand Down Expand Up @@ -21485,6 +21499,8 @@ def __str__(self):
char_dict = dict()
char_dict[dictkey_origin] = JM_py_from_point( ch.m_internal.origin)
char_dict[dictkey_bbox] = JM_py_from_rect(r)
if raw == 2:
char_dict["quad"] = JM_py_from_quad(JM_char_quad(line, ch))
char_dict[dictkey_c] = chr(ch.m_internal.c)
char_dict['synthetic'] = bool(ch.m_internal.flags & mupdf.FZ_STEXT_SYNTHETIC)

Expand Down
235 changes: 235 additions & 0 deletions src/_textsearch.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,235 @@
"""
Copyright (C) 2026 Artifex Software, Inc.

This file is part of PyMuPDF.

PyMuPDF is free software: you can redistribute it and/or modify it under the
terms of the GNU Affero General Public License as published by the Free
Software Foundation, either version 3 of the License, or (at your option)
any later version.

PyMuPDF is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
details.

You should have received a copy of the GNU Affero General Public License
along with MuPDF. If not, see <https://www.gnu.org/licenses/agpl-3.0.en.html>

Alternative licensing terms are available from the licensor.
For commercial licensing, see <https://www.artifex.com/> or contact
Artifex Software, Inc., 39 Mesa Street, Suite 108A, San Francisco,
CA 94129, USA, for further information.

---------------------------------------------------------------------

Approximate substring search, with offsets into the extracted page text.
"""

from bisect import bisect_right
import re


def _normalize(text):
"""Case-fold and collapse whitespace, retaining original character ranges."""
parts, offsets = [], []
for match in re.finditer(r"\s+|\S+", text):
start, end = match.span()
token = match.group()
if token.isspace():
# Join only a single line break between letters, retaining the
# original hyphen and newline through the surrounding offsets.
if token == "\n" and start >= 2 and end < len(text):
if (text[start - 1] == "-" and text[start - 2].isalpha()
and text[end].isalpha()):
parts[-1] = parts[-1][:-1]
offsets.pop()
continue
parts.append(" ")
offsets.append((start, end))
else:
parts.append(token.casefold())
if token.isascii():
offsets.extend(zip(range(start, end), range(start + 1, end + 1)))
else:
for i, char in enumerate(token, start):
offsets.extend([(i, i + 1)] * len(char.casefold()))
return "".join(parts), offsets


def _windows(text, needle, max_edits):
"""A match with k edits must contain one of k+1 query pieces unchanged."""
size = len(needle)
windows = []
for part in range(max_edits + 1):
offset = part * size // (max_edits + 1)
piece = needle[offset:(part + 1) * size // (max_edits + 1)]
pos = text.find(piece)
while pos >= 0:
windows.append((max(0, pos - offset - max_edits),
min(len(text), pos - offset + size + max_edits)))
# Repetitive text or large budgets can make filtering cost more
# than a full scan. Bound the work and memory spent on seed hits.
if len(windows) * size >= len(text):
return [(0, len(text))]
pos = text.find(piece, pos + 1)
merged = []
for start, end in sorted(windows):
if merged and start <= merged[-1][1]:
merged[-1] = (merged[-1][0], max(merged[-1][1], end))
else:
merged.append((start, end))
return merged


def _candidates(text, needle, max_edits, boundaries=None):
"""Yield (distance, start, end), choosing the earliest optimal start per end."""
size = len(needle)
if max_edits == 0:
start = text.find(needle)
while start >= 0:
end = start + size
if boundaries is None or (boundaries[start] and boundaries[end]):
yield 0, start, end
start = text.find(needle, start + 1)
return

for first, last in _windows(text, needle, max_edits):
bounds = boundaries[first:last + 1] if boundaries is not None else None
for cost, start, end in _levenshtein(text[first:last], needle, max_edits, bounds):
yield cost, first + start, first + end


def _levenshtein(text, needle, max_edits, boundaries):
"""Semi-global Levenshtein, with two rows of costs and start offsets.

Skipping a text prefix is free; deleting a query character costs one.
"""
size = len(needle)
previous = list(range(size + 1))
active = list(range(max_edits + 1))
if boundaries is not None and not boundaries[0]:
previous = [max_edits + 1] * (size + 1)
active = []
starts = [0] * (size + 1)
dense = False
for end, char in enumerate(text, 1):
boundary = boundaries is None or boundaries[end]
current = [max_edits + 1] * (size + 1)
current_starts = [end] * (size + 1)
current[0] = 0 if boundary else max_edits + 1
# An over-budget state cannot lead to an in-budget alignment. Visit
# only active states, successors and deletions from a new start.
# Once states become dense, avoid bookkeeping for the rest of this
# window (important for repeated characters or large edit budgets).
if dense or len(active) > size // 3:
dense = True
columns = enumerate(needle, 1)
else:
columns = sorted(set(active) | {i + 1 for i in active if i < size}
| set(range(1, max_edits + 1) if boundary else ()))
columns = ((i, needle[i - 1]) for i in columns if i)
next_active = [0] if boundary else []
for i, wanted in columns:
cost = previous[i - 1] + (char != wanted)
start = starts[i - 1]
insertion = previous[i] + 1
if insertion < cost or (insertion == cost and starts[i] < start):
cost, start = insertion, starts[i]
deletion = current[i - 1] + 1
if deletion < cost or (deletion == cost and current_starts[i - 1] < start):
cost, start = deletion, current_starts[i - 1]
current[i], current_starts[i] = cost, start
if not dense and cost <= max_edits:
next_active.append(i)
if boundary and current[-1] <= max_edits and current_starts[-1] < end:
yield current[-1], current_starts[-1], end
previous, starts, active = current, current_starts, next_active


def _page_text(raw):
"""Return text and span ranges; RAWDICT already stores individual chars."""
text, spans = [], []
offset = 0
for block in raw["blocks"]:
if block["type"] != 0:
continue
if text:
text.append("\n")
offset += 1
for line in block["lines"]:
for span in line["spans"]:
# Each RAWDICT char holds one Unicode code point, including
# preserved ligatures. Store one range per span, not per char.
value = "".join(char["c"] for char in span["chars"])
text.append(value)
spans.append((offset, offset + len(value), line["dir"], span))
offset += len(value)
text.append("\n")
offset += 1
return "".join(text), spans


def search_for_fuzzy(page, needle, *, max_edits, clip=None, textpage=None):
from . import CheckParent, TEXT_MEDIABOX_CLIP, TEXT_PRESERVE_LIGATURES
from . import TEXT_PRESERVE_WHITESPACE, TOOLS, Quad, recover_span_quad

CheckParent(page)
if not isinstance(needle, str):
raise TypeError("needle must be a string")
if not isinstance(max_edits, int) or isinstance(max_edits, bool):
raise TypeError("max_edits must be an integer")
if max_edits < 0:
raise ValueError("max_edits must be non-negative")
needle = _normalize(needle)[0].strip()
if not needle:
return []
if max_edits >= len(needle):
raise ValueError("max_edits must be less than the normalized needle length")
if textpage is None:
# Preserve the actual line breaks, hyphens and glyphs for result text
# and quads. Normalization happens only in the search representation.
textpage = page.get_textpage(clip=clip, flags=(
TEXT_PRESERVE_WHITESPACE | TEXT_PRESERVE_LIGATURES | TEXT_MEDIABOX_CLIP
))
elif getattr(textpage, "parent", None) != page:
raise ValueError("not a textpage of this page")

# With corrections disabled, RAWDICT's font metrics cannot recover the
# native quads. Retain those only when needed, without changing global state.
raw = textpage._textpage_dict(raw=2 if TOOLS.unset_quad_corrections() else 1)
original, spans = _page_text(raw)
text, offsets = _normalize(original)
boundaries = [True] + [a[0] != b[0] for a, b in zip(offsets, offsets[1:])] + [True]
candidates = sorted(_candidates(text, needle, max_edits, boundaries),
key=lambda hit: (hit[0], abs(hit[2] - hit[1] - len(needle)),
hit[1], hit[2]))
# Resolve overlaps in original coordinates too: case-folding can expand
# one glyph (e.g. sharp s) into several searchable characters.
occupied = bytearray(len(original))
selected = []
for distance, start, end in candidates:
first, last = offsets[start][0], offsets[end - 1][1]
if any(occupied[first:last]):
continue
occupied[first:last] = b"\1" * (last - first)
selected.append((first, last, distance))

results = []
for first, last, distance in sorted(selected):
quads = []
index = bisect_right(spans, first, key=lambda item: item[1])
while index < len(spans):
start, end, direction, span = spans[index]
if start >= last:
break
chars = span["chars"][max(0, first - start):last - start]
if chars:
if "quad" in chars[0]:
left, right = chars[0]["quad"], chars[-1]["quad"]
quads.append(Quad(left[0], right[1], left[2], right[3]))
else:
quads.append(recover_span_quad(direction, span, chars))
index += 1
results.append(dict(text=original[first:last], distance=distance, quads=quads))
return results
8 changes: 8 additions & 0 deletions src/extra.i
Original file line number Diff line number Diff line change
Expand Up @@ -3215,6 +3215,14 @@ mupdf::FzRect JM_make_spanlist(

DICT_SETITEM_DROP(char_dict, dictkey_bbox, JM_py_from_rect(r));

// Internal raw=2 mode retains quads when font metrics cannot
// recover them, for example with quad corrections disabled.
if (raw == 2)
{
fz_quad quad = JM_char_quad(line.m_internal, ch.m_internal);
DICT_SETITEMSTR_DROP(char_dict, "quad", JM_py_from_quad(quad));
}

DICT_SETITEM_DROP(char_dict, dictkey_c, Py_BuildValue("C", ch.m_internal->c));
DICT_SETITEMSTR_DROP(char_dict, "synthetic", Py_BuildValue("O", (ch.m_internal->flags & FZ_STEXT_SYNTHETIC) ? Py_True : Py_False));
if (!char_list)
Expand Down
1 change: 1 addition & 0 deletions tests/test_pylint.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ def test_pylint():
'_table_refine.py',
'_table_spans.py',
'_table_union.py',
'_textsearch.py',
'_wxcolors.py',
'fitz___init__.py',
'fitz_table.py',
Expand Down
Loading
Loading