Compiled code that fits the processor's caches: big pages up to 1.8 times faster, and fast compiled code on Windows and Intel machines
Floati compiles a function to machine code once it has run often enough to pay for the compile. On a large page that is thousands of functions, and past a point their machine code no longer fits the processor's instruction caches: every call then waits on memory, and compiled code ran slower than not compiling at all (1.7 times slower for 2,000 functions run 400 times). Floati now keeps a code-size budget. It estimates how much other code runs between two calls of a function (its reuse distance), and compiles it only while the code would still be in the cache when it comes back. Hot loops and their helpers still compile; code that runs once per pass stays in the fast interpreter. On an ARM64 Mac, 2,000 functions run 400 times went from 3,609 to 2,020 ms in a page, and 1,000 functions from 1,180 to 790 ms, with less memory. With the budget in place, Gust, Floati's fast compiler, is on by default on x86-64 too: on a Windows machine, a set of 50 hot functions went from 2,385 to 676 ms and 200 from 10,082 to 5,086 ms, and large sets are no slower than before. Its x86-64 code is also 40 percent smaller.
A code-size budget
A processor runs machine code fastest when it is already in its instruction cache, a few megabytes at most. A compiler that compiles every warm function fills that space on a big page, and the code it made evicts itself: each call reads from memory again. We measured it on generated pages of many small functions run in a shuffled order. With 200 functions compiling won 4 times over; with 2,000 it lost 1.7 times. The budget borrows from how caches themselves are modelled: a piece of code stays cached if the code that runs between two of its uses fits in the cache (its reuse distance). Floati measures that distance for every function as it runs, keeps a running count of compiled bytes, and compiles a function only while it and the code around it fit, about 4 MB on Apple silicon and 2 MB on x86-64, where the caches are smaller. Code that stops being used gives its bytes back. Other engines bound their compiled code too, mostly by flushing code once it has gone cold; Floati decides before compiling, from the reuse it measured.
Fast compiled code on Windows and Intel machines
Gust compiles a warm function in under a microsecond per operation. On x86-64 it was off by default because its code was twice the size of the ARM64 code and big pages ran slower with it. The code is now 40 percent smaller (registers held in real x86 registers, short branches, constants shared from a pool, slow paths moved out of line) and the budget keeps it inside the cache, so it is on by default on Windows and Intel machines. Compiled frames on Windows are registered with the system's unwinder, so crash reports and debuggers can walk through them.
What changed 5 changes
Try it in your browser
Each check runs live in the browser you are reading with. Nothing is sent anywhere.