Longer answer, because it surprises people: using every logical thread is usually slower, not faster. On a 4-core/8-thread laptop, 4 threads measured about 9% faster than 8, with steadier timings.

The reason is that generating text is limited by how fast the machine can read the model from memory, not by how much arithmetic it can do. Extra threads add contention without adding memory bandwidth.

If you want to experiment, CATERPILLAR_CPU_THREADS overrides it. If you try it, please post both numbers and your CPU — one result on its own is hard to learn from, since thermal state moves these numbers around a lot.