[ebarkie]

string.find, match, gsub and gmatch try the matcher at every position of the subject. For most patterns that is wasted work: a pattern whose first item is a literal, a class (%d) or a set ([\128-\255]) can only match at a byte that item accepts, and most bytes of a text are not one. This change computes that set once per pattern and skips to the next candidate byte with a plain loop, or strings.IndexByte when the set is one byte. gsub copies the bytes it skips to its builder in one piece instead of one at a time.

The set is built only when the first item must take a byte: not after ^, not for ., not for an item followed by *, ? or -, not for $ alone, a position capture, %b, %f or a back-reference, and not when the matcher would fail before the item (33 captures open). A capture that opens on such an item ((%d+)) qualifies. The sets are cached by pattern in the executor, 64 at a time and cleared between entries, as building one costs 256 tests, more than a short subject's scan; patterns over 256 bytes are built each time and not kept, and a set item over 64 bytes is left to the matcher.

The step budget stays honest: the build is charged 256 times the item's length, and the bytes skipped are charged whether or not a candidate is found, so Limits.Steps bounds a scan as before. To keep the build free of side effects, the scan of a [set] for its ] moves from classEnd into setEnd, which neither charges nor raises; classEnd does both as before.

Motivation

Text processing scripts spend much of their time in find, gsub and gmatch with patterns that match sparsely: stripping control characters, finding the first byte over 127, splitting on a delimiter, replacing a literal. Profiling such scripts against a C Lua 5.4 build showed every pattern scan running 1.7 to 1.8 times slower than C Lua, while the matcher's cost per call was already at parity. The gap was entirely the per-position loop around the matcher. With this change those scans run several times faster than C Lua, and a script that parses and rewrites a 1 MiB message bundle with a mix of these calls goes from 62 ms to 45 ms (C Lua: 47).

Over 1 MiB of text, arm64:

Operation	before	after	C Lua 5.4.7
find("[\128-\255]"), no match	8.1 ms	0.3 ms	4.5 ms
gsub("[%z\1-\8\11-\31\127]", ""), no match	15.3 ms	0.3 ms	8.4 ms
gsub("[\128-\255]", fn), a quarter of the bytes match	15.3 ms	8.3 ms	9.2 ms
gsub("[\128-\255]", table)	12.1 ms	4.8 ms	6.7 ms
gsub("\n", "")	6.0 ms	0.5 ms	3.3 ms
gmatch("%S+"), dense words	10.7 ms	10.8 ms	7.2 ms
match("^(%d+)/(%d+)") per short word	unchanged	unchanged	at parity

Anchored patterns and dense matches are unchanged, as expected: the matcher's per-call cost is already at parity with C Lua.

Testing

• TestPatternFirstSet covers the first items that take a byte, those that may take none, anchors, captures, position captures, balances, frontiers, back-references, the gsub limit, gmatch's rules for a literal ^ and for empty matches, malformed patterns still raising, "too many captures" still raising when no candidate byte exists, the cache past 64 patterns, a pattern too long to cache, the step charge for skipped bytes (found or not) and for the build, and a gmatch iterator that the budget stopped mid-scan being called again by a closer.
• TestDifferential with GLUA_DIFF_N=3000 against C Lua 5.4.7: no pattern differences. On macOS the test also reports os.time with negative years and %g of nan, libc differences that predate this change; CI runs it on Linux.
• go vet, go test, -race, -d=checkptr, -tags lua_stackstress, bench/asmgate and genloop -check all pass.