What the latest Native and Caterpillar results actually show

[lizardadmin]

These analyzed captures put Lizard Native and Caterpillar in the foreground beside stock llama.cpp and optional Ollama. They also show the honest result: no provider wins every metric, and Caterpillar trails the other two lanes in these selected general-case runs.

Interactive gallery: https://lizard-llm.qendryx.com/benchmarks.html

Wall time, throughput and decode are different metrics

llama.cpp wins this run's wall time and end-to-end throughput. Lizard Native leads native decode at 15.198 tok/s versus Caterpillar's 9.201. The dashboard does not blend those numbers.

No provider wins every dimension

The heatmap and radar show the trade: llama.cpp leads speed and memory efficiency here, while the native lanes carry their own decode and execution-lane evidence.

The decision guide names the real winners

For this hardware, llama.cpp Q4 is the balanced choice at 16.31 tok/s and 3.925 seconds. Lizard Native Q4 is called out separately for best native decode at 15.198 tok/s. Ollama was not included and remains unmeasured.

A larger model changes the gap

For Gemma 4 E4B, llama.cpp records 4.37 tok/s, Lizard Native 4.08, and Caterpillar 1.41. The Ollama card says not included and unmeasured—not zero.

Provider identity stays attached to every metric

Runtime, model load, warm-up, peak memory and efficiency stay under the provider that produced them. This is the evidence needed to evaluate Lizard as a local inference provider.

---

Question for you: What hardware and model should we run next to test where Caterpillar closes the gap—or where Native's decode path matters most?