Skip to content

[RFC] ext/standard: Add prepared string matching and replacement - #25122

Open
ArtUkrainskiy wants to merge 1 commit into
php:masterfrom
ArtUkrainskiy:str-matcher-prototype
Open

ArtUkrainskiy wants to merge 1 commit into
php:masterfrom
ArtUkrainskiy:str-matcher-prototype

Conversation

@ArtUkrainskiy

@ArtUkrainskiy ArtUkrainskiy commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

RFC: https://wiki.php.net/rfc/prepared_literal_string_matching

Discussion: https://news-web.php.net/php.internals/133028

Benchmark report: https://github.com/ArtUkrainskiy/php-src-bench/tree/main/reports/multi-pattern-search

Str\Matcher and Str\Replacer take a dictionary of literal keys once and search or replace all of them in one pass over the text.

$matcher = new Str\Matcher(['viagra', 'casino', 'free money']);
$matcher->matches($comment);   // also getMatchedKeywords(), countMatches(), matchAll()
(new Str\Replacer(['str_contains' => '<a href="…">str_contains</a>']))->replace($page);

Change in time per call against the fastest existing method for the same input (str_contains()/strpos()/substr_count() loop, strtr() or PCRE). Random keys. Each range covers all five calls, match positions and both compilers. Negative = faster.

Keys 128 B 1 KiB 4 KiB 16 KiB 64 KiB
Prepared object
8 −33…+16% −42…+88% −34…+104% −28…+112% −26…+117%
16 −62…−26% −60…−6% −58…−2% −56…−2% −56…+1%
32 −76…−46% −79…−61% −73…−50% −72…−59% −70…−50%
128 −82…−61% −85…−72% −81…−73% −80…−73% −79…−72%
1024 −88…−80% −90…−78% −87…−79% −85…−79% −84…−79%
Build and one call
8 +94…+779% −3…+273% −18…+171% −22…+133% −25…+123%
16 +126…+859% −4…+173% −37…+59% −50…+15% −54…+5%
32 +159…+948% +5…+140% −53…+8% −71…−56% −73…−46%
128 +369…+982% −7…+108% −67…−29% −84…−75% −82…−77%
1024 +845…+5012% +55…+751% −54…+110% −81…−44% −81…−80%

AMD Ryzen 9 5950X, GCC 15.2 and Clang 21.1, calls go through a pool of different texts. Not in the table: matches() with a match in the first 64 bytes, where the existing methods stop early. There a prepared object gives −70…+49% with 32 keys or more.

On real data:

Application Prepared Build and one call Calls to repay the build, estimated
Moderation: 403 words, 199 B comments 8.0–13× faster 4.0–5.8× slower 14–32
Bot detection: 1,469 patterns, 118 B User-Agents 11–12× faster 27–31× slower 287–354
Auto-linking: 3,345 function names with _, 597 B manual pages 12–123× faster 2.6–9.5× slower 9–66
Auto-linking: 128 or 512 of those names, 4–16 KiB manual pages 4.0–7.3× faster 2.9–3.6× faster 1

On the same 4–16 KiB pages with all 3,345 names, one matchAll() is 60–62% faster, but one replace() is 32–40% slower than strtr(). On 1–2 KiB pages one call doesn't pay off.

On short texts the gain needs a reused object: a long-running worker, or a loop over many texts. Building a Matcher costs 21–58 µs per KiB of keys with GCC: 117 µs and 134 KiB for the moderation list, 1,640 µs and 809 KiB for the 3,345 function names. With 1 or 4 keys the existing methods are 1.9–22× faster.

matchAll() returns every occurrence, overlapping ones included, as Str\MatchResult objects with byte offsets. replace() follows strtr() array semantics: leftmost match, longest key, no rescanning of replaced text. Matching is byte-wise, binary-safe and case-sensitive. Both classes use an Aho–Corasick automaton with dense rows near the root and packed sparse rows deeper.

15 PHPT tests cover the API contract, binary input, overlapping matches, object lifecycle and 350 seeded cases against strpos() and strtr() oracles.

Not measured: other CPUs, cold caches, the JIT, userland Aho–Corasick libraries.

@ArtUkrainskiy
ArtUkrainskiy force-pushed the str-matcher-prototype branch 2 times, most recently from 287aaab to 7c7550b Compare November 5, 2026 10:41
@ArtUkrainskiy
ArtUkrainskiy marked this pull request as ready for review November 6, 2026 12:59
@ArtUkrainskiy ArtUkrainskiy changed the title ext/strmatch: Add prepared string matching and replacement [RFC] ext/strmatch: Add prepared string matching and replacement Nov 6, 2026

@TimWolla TimWolla left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This absolutely should be part of ext/standard. Making it a separate extension is needless (maintenance) overhead.

Add final Str\Matcher and Str\Replacer classes that prepare a fixed
dictionary of literal keys once and reuse it across subjects.

Str\Matcher reports whether any key occurs, the matched keys, the number
of occurrences and every occurrence with its byte offset. Str\Replacer
applies leftmost-longest replacement with strtr() array semantics.
Matcher keys must be strings.

Both build a hybrid Aho-Corasick automaton: dense rows near the root,
packed sparse transitions deeper, row offsets stored as state IDs.
Replacement scans a reversed automaton.

Objects compare equal when they hold the same key set or mapping, and
are not cloneable or serializable; neither is MatchResult.

Include 15 PHPT tests for API semantics, binary input, overlapping
matches, the replacement oracle and object lifecycle.
@ArtUkrainskiy ArtUkrainskiy changed the title [RFC] ext/strmatch: Add prepared string matching and replacement [RFC] ext/standard: Add prepared string matching and replacement Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants