Skip to content

Keep float remainders and float keys of whole numbers in the loop - #53

Merged
ebarkie merged 1 commit into
folbricht:mainfrom
ebarkie:float-arith
Oct 21, 2026
Merged

ebarkie merged 1 commit into
folbricht:mainfrom
ebarkie:float-arith

Conversation

@ebarkie

@ebarkie ebarkie commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Code written before Lua had integers takes bits with /, % and math.floor: bit buffers,
hashes, CRCs, PRNGs, base64, date arithmetic, byte packing. There is a lot of it, since Lua
5.1 was the common denominator for years, and it runs unchanged on 5.4, where that arithmetic
is in floats: a bit buffer is a float, bitbuf % p a float remainder, and bitbuf % size + 1
a float key. The loop did neither. A float remainder went to the driver and math.Mod, which
takes the quotient's bits one at a time, so a remainder of a 2^40 bit buffer cost tens of
iterations of frexp and ldexp, a cost that grows with the dividend; a float key went to the
driver to be normalised. A DEFLATE decoder written that way, 300 lines of pure Lua, ran 3
times slower than in C Lua, with half its time in math.Mod. The benchmark suite has none of
this pattern, which is why it went unnoticed.

Performance changes

  • Float remainders of whole numbers below 2^53 are computed as integers, inline in the
    loop's MOD and MODK cases, with the sign a zero keeps from the dividend, as fmod gives it.
    floatMod does the same for the driver and leaves math.Mod for values that are not
    whole or too large. Two integers with a divisor below 1 stay the driver's, since their
    result is an integer.
  • Float keys with whole values take the inline table paths of GETTABLE and SETTABLE as
    the integer they normalise to, in the array part and the hash part; 2^63 and above, NaN
    and fractions still go the long way.
  • The loop grows by about what PR 1's merged store cases freed. The gate's baseline moves by
    reloads on the exit paths of GETTABLE and of JMP's tick; the back edge itself touches no
    stack slot (checked in the amd64-v3 machine code), and GETTABLE's string path has one
    access fewer than before on both amd64 targets.

Times on an Apple M-series machine, best of five, for the DEFLATE decoder on 1 MiB of
compressed text:

Workload before after C Lua 5.4.9
The decoder as written, in 5.1 float arithmetic 141 ms 54 ms 45 ms
The same decoder rewritten with // and integer tables 42 ms 43 ms 27 ms
CRC-32 and CRC-16 over the same data (integer keys already) unchanged unchanged
Suite scripts (floatloop, mathcalls, nbody, sieve, fannkuch, spectralnorm) unchanged unchanged

The rewritten decoder is the number a script owner can reach by hand; the change is for the
code that will not be rewritten.

Tests

A test checks float remainders against the values C Lua gives, the signs of zero included
for integer and float dividends alike, in the register and the constant (MODK) forms, mixed
integer and float operands, values that are not whole or too large, and a float bit buffer
taken apart as a decoder does; another checks float keys with whole values against the array
and hash parts, -0.0, 2^53, 2^63, NaN, fractions and appends, on both loops. The
arithmetic corpus caught the one mistake on the way (integers with a negative divisor taking
the float path). vet, tests, race, checkptr, stack stress, asmgate and the generated-loop
check pass locally; a 20,000-program differential run against C Lua shows only the
pre-existing macOS libc differences (os.time before year 1, %a of denormals). Fuzzing was
not run.

Code written before Lua had integers takes bits with / and %, and in Lua
5.4 that arithmetic is in floats: a bit buffer is a float, bitbuf % p a
float remainder, and bitbuf % size + 1 a float key. The loop did neither.
A float remainder went to the driver and math.Mod, which takes the
quotient's bits one at a time, so a remainder of a 2^40 bit buffer cost
tens of iterations of frexp and ldexp; a float key went to the driver to be
normalised. A DEFLATE decoder written that way ran 3 times slower than in
C Lua, at 141 ms a MiB to 45.

A remainder of whole numbers below 2^53 is exact as integers: the loop
computes it so, with the sign a zero keeps from the dividend, as fmod gives
it; floatMod does the same for the driver, and leaves math.Mod for the
rest. A float key with a whole value takes the array and hash paths of
GETTABLE and SETTABLE as the integer it normalises to. The decoder goes to
54 ms a MiB; the suite, which has neither, is unchanged. Two integers with a
divisor below 1 stay the driver's, as their result is an integer.

The loop grows by what the merged store cases freed. The gate's baseline
moves by reloads on the exit paths of GETTABLE and of JMP's tick: the back
edge itself touches no stack slot.
@ebarkie
ebarkie merged commit 0127c76 into folbricht:main Oct 21, 2026
17 checks passed
folbricht added a commit that referenced this pull request Oct 21, 2026
Benchmarks: rerun on main with #50 to #53, glua at 1.13x C Lua
ebarkie pushed a commit to ebarkie/glua that referenced this pull request Oct 21, 2026
@ebarkie
ebarkie deleted the float-arith branch October 21, 2026 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant