Modern Intel (Ice Lake and newer) and AMD (Zen 4 and newer) processors have an instruction that does 8-bit integer maths faster. Caterpillar has a kernel that uses it, behind CATERPILLAR_CPU_VNNI=1.

It ships disabled, and the reason is worth stating plainly: on our test hardware it measured within run-to-run noise of the standard path. Not faster, not slower — indistinguishable. Shipping it on by default would be claiming a win we can't demonstrate.

It may well help on other chips, particularly ones with more memory bandwidth. If you try it:

CATERPILLAR_CPU_VNNI=1

Please post your CPU and both numbers, run back-to-back. Alternating measurements matter here — machines get hotter as you test them, and that alone can swamp the difference.