A benchmark dashboard can mislead without containing a false number.

Treat an excluded provider as zero. Mix warm-up into decode. Turn one metric winner into a universal recommendation. Each shortcut changes the story.

Lizard keeps provider identity, wall time, end-to-end throughput, native decode, load, warm-up and memory separate. Missing providers stay unmeasured.

The goal is not to remove judgment. It is to keep the evidence needed for judgment.

Engineering fact: Lizard's dashboard keeps excluded providers unmeasured and separates wall time, end-to-end throughput, decode, warm-up, and memory.

Read the relevant Lizard page

#Benchmarking #DataIntegrity #LLMInference #Reproducibility #LocalAI

<!-- lizard-marketing-slot:day-09-pm -->