The short version: generating a word normally means reading the whole model from memory once. If the next few words are predictable, Caterpillar drafts a guess and checks several words in a single pass instead of one pass each.

Two things worth knowing:

1. It is lossless. You get exactly the same words you would have got anyway — it's verified against the real model, not an approximation. If a guess is wrong it's thrown away.
2. It steps aside when it isn't helping. On normal prose, guesses rarely land, so it stops trying rather than making you pay for failed guesses.

So the honest summary is: sometimes much faster, never slower in a way you'd notice, never different output.

Anyone seen a big speed-up on a particular kind of prompt?