<![CDATA[From GGUF selection to local inference: the benchmark setup]]>A benchmark is only meaningful when the exact model file and runtime state are known. These five screens show the path from repository metadata to a locally activated model. Planning estimates are shown before the run; measured evidence is recorded later.

Interactive gallery: https://lizard-llm.qendryx.com/benchmarks.html

Inspect repository and compatibility metadata

Llama 3.2 GGUF model details

Lizard shows the model family, author, supported quantizations and whether the file can run on the native engine before anything is downloaded.

Choose the exact quantized file

GGUF quantization variants

The selected Q4_K_M file has explicit download size, estimated RAM and planning speed. Those estimates are not relabeled as measurements later.

Download identity remains visible

Selected GGUF download progress

The repository and exact GGUF filename remain on screen while the file downloads, so the later provider rows can be traced back to their intended weights.

Cold is different from warm

Cold local chat model state

The chat identifies that the model is cold and offers an explicit wake action. Benchmark load and inference timings preserve the same distinction.

Activate the model for local inference

Locally activated Llama 3.2 model

The activated model becomes available to the CLI, chat and local API. The screen names Lizard Native and Caterpillar as the two on-device engines.

Question for you: Which model and quantization did Lizard recommend for your machine, and did the measured result match the planning estimate?

]]>https://community.lizard-llm.qendryx.com/topic/27/from-gguf-selection-to-local-inference-the-benchmark-setupRSS for NodeMon, 24 Aug 2026 03:19:27 GMTWed, 22 Jul 2026 15:57:46 GMT60