Skip to content

[RFC] ext/strmatch: Add prepared string matching and replacement - #25122

Open
ArtUkrainskiy wants to merge 1 commit into
php:masterfrom
ArtUkrainskiy:str-matcher-prototype
Open

ArtUkrainskiy wants to merge 1 commit into
php:masterfrom
ArtUkrainskiy:str-matcher-prototype

Conversation

@ArtUkrainskiy

@ArtUkrainskiy ArtUkrainskiy commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

This PR contains the current implementation in ext/strmatch. The RFC proposes integration into ext/standard; the final placement and naming remain open for discussion.

RFC: https://wiki.php.net/rfc/prepared_literal_string_matching

Discussion: https://news-web.php.net/php.internals/133028

I propose Str\Matcher and Str\Replacer for preparing a fixed literal dictionary once and using it across multiple texts. Matcher supports presence checks, unique matched keys, counting and enumerating overlapping occurrences; Str\MatchResult exposes byte offsets, lengths and keys. Replacer selects the earliest match, preferring the longest key at that position, and does not rescan inserted text.

Both classes capture an immutable dictionary and use Aho–Corasick with hybrid dense/sparse transitions. Matching is byte-oriented, binary-safe and case-sensitive. Reuse is within one request or a long-running process.

AMD Ryzen 9 5950X, Linux x86-64, GCC 15.2 / Clang 21.1. Times are medians; parentheses show median paired elapsed-time changes against the fastest eligible existing method in each measurement. Negative percentages mean faster; slowdowns of at least 2× are shown as ×N slower. Prepared includes the call and result release on an existing object; one-shot also includes construction and destruction. Existing methods use prebuilt arguments and warm PCRE caches. The displayed existing-method times are medians from the one-shot comparisons; prepared calls have separately measured references.

Operation / exact input Best existing method / time GCC prepared, µs GCC one-shot, µs Clang prepared, µs Clang one-shot, µs
matches(); 128 B; 128 keys; no matches preg_match()
GCC: 3.048 µs
Clang: 3.038 µs
0.186 (-93.88%) 52.553 (×17.2 slower) 0.164 (-94.63%) 51.907 (×17.1 slower)
matches(); 4 KiB; 128 keys; no matches preg_match()
GCC: 100.021 µs
Clang: 100.014 µs
3.817 (-96.18%) 58.107 (-42.02%) 3.428 (-96.57%) 56.359 (-43.72%)
matches(); 4 KiB; 128 keys; hit at byte 8 preg_match()
GCC: 0.414 µs
Clang: 0.408 µs
0.116 (-71.84%)† 52.787 (×127.4 slower) 0.110 (-72.94%) 52.310 (×128.1 slower)
matches(); 4 KiB; 128 keys; hit at byte 2042 preg_match()
GCC: 51.972 µs
Clang: 51.897 µs
1.986 (-96.17%) 54.966 (+5.90%) 1.694 (-96.73%) 54.147 (+4.43%)
matches(); 4 KiB; 128 keys; hit at byte 4076 str_contains() loop
GCC: 60.558 µs
Clang: 56.447 µs
3.814 (-93.66%) 57.599 (-4.94%) 3.364 (-94.03%) 55.063 (-2.40%)
matches(); 32 KiB; 1024 keys; no matches str_contains() loop
GCC: 7699.530 µs
Clang: 7317.499 µs
78.008 (-98.98%) 848.164 (-88.98%) 74.420 (-98.97%) 858.186 (-88.27%)
getMatchedKeywords(); 4 KiB; 128 keys; 1 occurrence preg_match_all()
GCC: 102.852 µs
Clang: 102.811 µs
4.317 (-95.79%) 58.478 (-43.01%) 3.499 (-96.60%) 56.524 (-45.20%)
getMatchedKeywords(); natural EN text, 12828 B; 128 keys; 23 occurrences, 12 unique keys strpos() loop
GCC: 307.035 µs
Clang: 289.993 µs
19.341 (-93.68%) 95.719 (-68.77%) 17.473 (-93.96%) 97.458 (-66.38%)
countMatches(); 4 KiB; 128 keys; 1 occurrence preg_match_all()
GCC: 101.504 µs
Clang: 101.701 µs
4.739 (-95.34%) 66.663 (-34.34%) 3.363 (-96.68%) 56.827 (-44.07%)
matchAll(); 64 KiB; 128 keys; 1 occurrence preg_match_all()
GCC: 1610.465 µs
Clang: 1614.484 µs
193.198 (-88.00%) 278.040 (-82.73%) 185.683 (-88.49%) 272.581 (-83.11%)
matchAll(); 64 KiB; the same 128 keys; 64 occurrences, 4 unique keys preg_match_all()
GCC: 1695.893 µs
Clang: 1702.396 µs
196.443 (-88.42%) 280.789 (-83.43%) 188.806 (-88.97%) 276.099 (-83.77%)
replace(); 1 KiB; 32 keys; 2 replacements preg_replace_callback()
GCC: 2.422 µs
Clang: 2.566 µs
1.436 (-40.71%) 14.887 (×6.1 slower) 1.348 (-47.55%) 15.270 (×6.0 slower)
replace(); 4 KiB; the same 32 keys; 0 replacements preg_replace_callback()
GCC: 8.482 µs
Clang: 8.173 µs
5.939 (-29.86%) 19.350 (×2.3 slower) 5.659 (-30.80%) 19.725 (×2.4 slower)
replace(); 4 KiB; the same 32 keys; 16 replacements preg_replace_callback()
GCC: 9.380 µs
Clang: 9.221 µs
6.247 (-33.42%) 19.790 (×2.1 slower) 6.120 (-33.61%) 20.173 (×2.2 slower)
replace(); 16 KiB; the same 32 keys; 16 replacements preg_replace_callback()
GCC: 62.595 µs
Clang: 61.774 µs
26.065 (-58.33%) 40.257 (-35.67%) 24.968 (-59.62%) 39.583 (-36.06%)
replace(); 4 KiB; 1 key of 8 B; 5 replacements strtr()
GCC: 0.400 µs
Clang: 0.390 µs
4.080 (×10.2 slower) 4.462 (×11.2 slower) 2.609 (×6.7 slower) 2.971 (×7.6 slower)
replace(); 1 MiB; 1000 keys of 8–24 B; 1000 replacements preg_replace_callback()
GCC: 13762.732 µs
Clang: 13679.984 µs
1550.727 (-88.73%) 2932.962 (-78.70%) 1290.924 (-90.57%) 2747.874 (-79.90%)
replace(); 1 MiB; 10000 keys of 16 B; 0 replacements strtr()
GCC: 7045.148 µs
Clang: 6573.642 µs
1466.895 (-79.15%) 32658.311 (×4.6 slower) 1185.057 (-81.97%) 32827.694 (×5.0 slower)
replace(); the same 1 MiB text; 10000 keys of 8–24 B; 0 replacements strtr()
GCC: 89600.329 µs
Clang: 89526.581 µs
1489.095 (-98.34%) 32682.993 (-63.52%) 1196.013 (-98.66%) 32943.512 (-63.24%)
replace(); 1 MiB; 1000 keys in {{…}} format; 100 replacements specialized preg_replace_callback()
GCC: 63.977 µs
Clang: 64.276 µs
1148.500 (×18.0 slower) 1471.567 (×23.0 slower) 1054.442 (×16.4 slower) 1372.608 (×21.4 slower)

Each row uses one fixed text and dictionary. Controlled Matcher keys are 12 B; the natural-text dictionary uses keys of 4–28 B. In the three positive matches() rows, the matching key is #65 in dictionary order. The 32-key Replacer rows share keys of 11–21 B and same-length replacement values.

For {{…}} keys, the specialized PCRE expression finds candidates and looks them up in the dictionary. These cases illustrate different trade-offs, not universal thresholds or production workload frequencies. † Observed time only: the measurement did not pass quality checks.

The extension builds with -Werror on GCC and Clang; 16 PHPT tests pass on both. Independent output checks pass under ASan/UBSan.

Full results, memory costs, exact inputs, correctness logs and reproduction.

Introduce independent final Str\Matcher and Str\Replacer classes for
reusing fixed keyword dictionaries across subjects.

Build hybrid Aho-Corasick automata with dense and packed sparse
transitions. Use byte-based matching and reversed matching for
leftmost-longest replacement, while retaining shared keyword strings.
Collect sparse replacement candidates and switch to a position-indexed
buffer when their density warrants it.

Check state, count and result limits, reject repeated initialization,
and keep prepared objects uncloneable, non-serializable and incomparable.

Include Unix and Windows build configuration, CI coverage and 16 PHPT
tests for API semantics, binary input, overlapping matches and lifecycle.
@ArtUkrainskiy
ArtUkrainskiy marked this pull request as ready for review November 6, 2026 12:59
@ArtUkrainskiy ArtUkrainskiy changed the title ext/strmatch: Add prepared string matching and replacement [RFC] ext/strmatch: Add prepared string matching and replacement Nov 6, 2026

@TimWolla TimWolla left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This absolutely should be part of ext/standard. Making it a separate extension is needless (maintenance) overhead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants