Inference Engineering · Day 30 · Morning [image: Screenshot%202026-07-22%20221126.png] Performance leadership is evidence with boundaries, not a permanent badge. A benchmark that omits hardware, model weights, quantization, context, concurrency, transport, and correctness makes it too easy to generalize a result past the machine that produced it. Lizard’s benchmark tooling keeps those fields together so a fast run stays scoped to the exact setup that produced it. That matters in practice. The same model can look different when the context window changes, when concurrent requests arrive, or when the transport path changes from one local route to another. If correctness is not recorded alongside speed, you can end up optimizing a failure mode and calling it progress. The useful habit is simple: publish the evidence you would want back when the result is challenged. Record the machine, the quant, the load shape, and the correctness gate before you compare one run to another. That does not slow the work down; it keeps the work honest. How do you usually fence a benchmark so a local win does not get repeated as a universal claim? Engineering fact: Lizard's benchmark tooling records hardware, model, quantization, context, concurrency, transport, and correctness so a local win remains scoped instead of becoming a universal claim. Lizard The AI Runtime You'll Own—Not Rent. Receive two professional Windows AI runtimes with lifetime updates. Run AI at native speed, keep every conversation private, and stay independent with intelligent hardware optimization and no cloud dependency. Read the relevant Lizard page #localai #benchmarking #inference #llmops #performanceengineering <!-- lizard-marketing-slot:day-30-am -->

Inference Engineering · Day 29 · Morning [image: Screenshot%202026-07-22%20221126.png] Reproducibility begins with a complete command. Lizard's HTTP benchmark command exposes the provider, model identity, exact GGUF path, concurrency lanes, and token budget. The llama.cpp comparison uses the same artifact unless the operator explicitly skips it. The command becomes part of the evidence, not an undocumented setup detail. Could someone reproduce your latest performance claim from the command you published? Engineering fact: Lizard documents the provider, model, GGUF path, concurrency lanes, token budget, and optional llama.cpp comparison in one HTTP benchmark command. Read the relevant Lizard page #LizardLLM #LLMInference #Benchmarking #LocalAI #PerformanceEngineering <!-- lizard-marketing-slot:day-29-am -->

Inference Engineering · Day 28 · Evening [image: Screenshot%202026-07-22%20221126.png] Zero is a measurement; missing is a state. When resource or performance telemetry is unavailable, Lizard preserves that absence instead of writing zero. A zero could incorrectly imply the GPU was measured and idle, the provider generated no tokens, or the path failed. Honest missing data protects later comparisons from a false baseline. Where in your dashboards could an empty value be silently turning into zero? Engineering fact: Lizard preserves missing resource or performance fields as unavailable rather than converting them to zero, which would falsely imply measured inactivity or failure. Read the relevant Lizard page #LizardLLM #LLMInference #Benchmarking #LocalAI #PerformanceEngineering <!-- lizard-marketing-slot:day-28-pm -->

Inference Engineering · Day 26 · Evening [image: Screenshot%202026-07-22%20221126.png] Throughput variation is evidence, not formatting noise. Lizard's repeated HTTP report keeps mean, median, min, max, standard deviation, percentiles, and coefficient of variation. Resource samples such as peak working set and average GPU utilization remain attached when available. A wide spread can point to contention, thermal behavior, memory pressure, or an unstable serving path before another optimization is attempted. Do you trust a single fastest run or the shape of the whole series? Engineering fact: Repeated Lizard HTTP reports aggregate throughput distribution together with working-set and GPU-utilization summaries when those resource samples are available. Read the relevant Lizard page #LizardLLM #LLMInference #Benchmarking #LocalAI #PerformanceEngineering <!-- lizard-marketing-slot:day-26-pm -->

Inference Engineering · Day 26 · Morning [image: Screenshot%202026-07-22%20221126.png] Hardware names alone do not describe an inference configuration. The same GPU can behave differently with another backend, memory pressure, CPU, or hybrid split. Lizard records the adapter, vendor, available memory, backend, and configuration class with benchmark evidence. This makes a result reproducible enough to compare and narrow enough not to become a universal promise. Which hardware detail is most often missing from the benchmark results you see? Engineering fact: Lizard records CPU, GPU vendor and adapter, available memory, backend, and configuration class so a benchmark result can be tied to the machine that produced it. Read the relevant Lizard page #LizardLLM #LLMInference #Benchmarking #LocalAI #PerformanceEngineering <!-- lizard-marketing-slot:day-26-am -->

Inference Engineering · Day 22 · Evening [image: Screenshot%202026-07-22%20221126.png] The main trap in inference benchmarks is a bad unit label. If someone reads “batch size” and assumes tokens, sequence length, or a compression setting, the chart stops meaning what you think it means. In Lizard’s HTTP benchmark, B=16 is deliberately narrow: sixteen concurrent client requests against one warm server. That keeps the concurrency axis honest. Quantization sweeps are tracked separately, so a run that changes model compression is not being mixed into a run that changes request pressure. That distinction matters in practice. If you are tuning queueing, health checks, session reuse, or server warmup, you want the concurrency curve to move for one reason at a time. Otherwise a nice-looking result can hide whether the server handled more requests, a different model representation, or just a different benchmark mode. I have found that the simplest check is to write the unit in the same sentence as the chart axis: concurrent requests, warm server, separate compression sweep. It removes a lot of debate later when the numbers get compared across runs, tools, or teams. When your team says “batch,” what is the first thing you clarify from experience? Engineering fact: In Lizard's HTTP benchmark, B=16 means sixteen requests in flight on one server; quantization sweeps are a separate benchmark dimension. Lizard The AI Runtime You'll Own—Not Rent. Receive two professional Windows AI runtimes with lifetime updates. Run AI at native speed, keep every conversation private, and stay independent with intelligent hardware optimization and no cloud dependency. Read the relevant Lizard page #llmbenchmark #inference #httpruntime #quantization #systemsengineering <!-- lizard-marketing-slot:day-22-pm -->

Inference Engineering · Day 22 · Morning [image: Screenshot%202026-07-22%20221126.png] Direct and HTTP benchmarks answer different questions, and the label matters more than people admit. A direct run is useful when you want to isolate the runtime path itself: the model load, the worker protocol, and the engine work without the rest of the request path in the way. That makes it a good tool for diagnosing a regression inside the runtime or comparing two engine changes under the same conditions. An HTTP run adds the part a user actually feels: request handling, server behavior, and the served API path. If the question is, “How does this model behave when clients talk to it through the product?”, HTTP is the measurement that matches the claim. Lizard exposes both modes separately, and the UI points to HTTP when the goal is end-user serving performance. That distinction saves teams from a common mistake: reporting a number that is technically true but answers the wrong question. The fastest path inside the engine is not always the best proxy for a real deployment. If I am trying to decide whether a server change is worth shipping, I want the served measurement. If I am trying to understand where time is going, I want the direct one first. What benchmark mode do you reach for when you need to separate engine cost from serving cost? Engineering fact: Lizard exposes native direct and native HTTP benchmark modes separately; the UI recommends HTTP when the goal is end-user serving performance. Lizard The AI Runtime You'll Own—Not Rent. Receive two professional Windows AI runtimes with lifetime updates. Run AI at native speed, keep every conversation private, and stay independent with intelligent hardware optimization and no cloud dependency. Read the relevant Lizard page #benchmarking #httptest #llmruntime #localai #performanceengineering <!-- lizard-marketing-slot:day-22-am -->

Inference Engineering · Day 14 · Evening [image: Screenshot%202026-07-22%20221126.png] Local benchmark caches need an evidence hierarchy. A fresh single run is useful for a new model or machine, but it should not silently replace a completed repeated series. Lizard marks repeated HTTP evidence as stable and gives it precedence when lane records are merged. This keeps adaptive serving responsive to evidence without making it fragile to one busy minute. How many repetitions do you require before a benchmark changes a production default? Engineering fact: A completed repeated native HTTP series is stored as stable lane evidence, and a later single run cannot overwrite that repeated result. Read the relevant Lizard page #LizardLLM #LLMInference #Benchmarking #LocalAI #PerformanceEngineering <!-- lizard-marketing-slot:day-14-pm -->

Inference Engineering · Day 15 · Morning [image: Screenshot%202026-07-22%20221126.png] Performance evidence is only useful when you can still tell what it came from. If a concurrency profile was measured on one provider, one model ID, one quantization, and one context bucket, that profile should stay tied to those four dimensions. Change any of them and you can change memory pressure, prefill cost, and the point where serving starts to fall over. That is why native HTTP tuning profiles are keyed by runtime provider, model ID, quantization, and context bucket before they are allowed to affect serving concurrency. That sounds strict, but the alternative is familiar to anyone who has debugged a benchmark cache: a result collected on a smaller context or a different quant can look “good enough” and still push the wrong live configuration into contention. A profile that was safe for one run may be too optimistic once the KV cache grows, or too conservative once a lighter quant cuts memory use enough to open room for more parallel requests. The practical lesson is to make the identity of the evidence part of the control plane, not just the log line. Keep the benchmark row, the runtime provider, the exact model, the quant, and the context bucket together, then let concurrency decisions inherit from that record instead of from a loose memory of “that fast run we saw earlier.” It is a small guardrail, but it closes a class of mistakes that are expensive to find after deployment. How do you key benchmark evidence when the same model family shows different behavior across context lengths? Engineering fact: Native HTTP tuning profiles are keyed by runtime provider, model ID, quantization, and context bucket before they influence serving concurrency. Lizard The AI Runtime You'll Own—Not Rent. Receive two professional Windows AI runtimes with lifetime updates. Run AI at native speed, keep every conversation private, and stay independent with intelligent hardware optimization and no cloud dependency. Read the relevant Lizard page #modelperformance #quantization #servingconcurrency #benchmarking #localllm <!-- lizard-marketing-slot:day-15-am -->

Inference Engineering · Day 12 · Evening [image: Screenshot%202026-07-22%20221126.png] A runtime can become faster by doing less work—or the wrong work. That is why the benchmark artifact keeps canonical output evidence next to throughput. The performance result is not considered admissible simply because the timer stopped sooner. Correctness is part of the performance contract, not a cleanup task after optimization. What correctness gate sits beside your fastest inference number? Engineering fact: The same-weight HTTP benchmark records canonical output evidence so a faster result is not accepted when the compared runtimes produce materially different token output. Read the relevant Lizard page #LizardLLM #LLMInference #Benchmarking #LocalAI #PerformanceEngineering <!-- lizard-marketing-slot:day-12-pm -->

Inference Engineering · Day 12 · Morning [image: Screenshot%202026-07-22%20221126.png] A model label tells you almost nothing about provenance. Two runners can point at the same name and still resolve different GGUF files, different revisions, or different conversion paths. That is enough to change the meaning of any timing result. The useful check is not “did the label match?” but “did both runtimes resolve the same physical file?” That is why Lizard’s HTTP comparison records the real GGUF path for both Lizard and llama.cpp, then treats the comparison as admissible only when both sides land on the same file on disk. It turns a benchmark from a trust exercise into a file-identity check. In practice, that means you can separate engine behavior from model drift. If the path differs, the result is a comparison across two artifacts, not a clean engine-vs-engine result. If the path matches, you at least know the weights are the same before you argue about throughput, memory, or decode behavior. I’ve found this matters most when model names get reused across quantizations and local copies. The filename looks stable, but the object behind it is not. What would you require before you repeat a same-model benchmark claim in your own workflow? Engineering fact: The HTTP comparison records the real GGUF path for both Lizard and llama.cpp and admits a performance claim only when both resolve to the same physical file. Lizard The AI Runtime You'll Own—Not Rent. Receive two professional Windows AI runtimes with lifetime updates. Run AI at native speed, keep every conversation private, and stay independent with intelligent hardware optimization and no cloud dependency. Read the relevant Lizard page #llama_cpp #gguf #benchmarking #modelprovenance #localai <!-- lizard-marketing-slot:day-12-am -->

Inference Engineering · Day 11 · Evening [image: Screenshot%202026-07-22%20221126.png] I think one thing that's easy to misunderstand in LLM benchmarks is tokens/sec. Our latest benchmark showed 40.7 tok/s for Lizard versus 21.8 tok/s for llama.cpp, using the exact same Llama 3.2 3B Q4_K_M GGUF model. The important part is that this is aggregate throughput with 16 concurrent requests (B=16) over multiple real HTTP runs. It's not the speed of a single chat response. Per-user latency and aggregate throughput answer different questions, and I think we should be much clearer about which one we're talking about when comparing inference engines. Whenever I see a tokens/sec number now, my first question is: Is that per response, or total throughput? Engineering fact: On one verified Llama 3.2 3B Q4_K_M run series, Lizard measured a 40.736 tok/s median at B=16 versus 21.755 tok/s for llama.cpp using the same physical GGUF; the result is aggregate HTTP throughput on that test system. Read the relevant Lizard page #LizardLLM #LLMInference #Benchmarking #LocalAI #PerformanceEngineering <!-- lizard-marketing-slot:day-11-pm -->

Inference Engineering · Day 11 · Morning [image: Screenshot%202026-07-22%20221126.png] A fast worker loop is not automatically a fast product. If the benchmark only times an internal execution loop, it can hide the parts users feel immediately: request parsing, scheduling, serialization, and the wait for the full response. Lizard's native readiness benchmark avoids that trap by sending concurrent requests through the OpenAI-compatible HTTP endpoint and measuring full-response throughput on the same path a client actually uses. That matters when you are comparing runtimes, not just kernels. A tight inner loop can look excellent while the request path still pays for queueing, transport, and response assembly. Once you measure the full HTTP path, the numbers are harder to hand-wave away, but they are also more useful for capacity planning and for spotting where latency is really coming from. Internal measurements still have a place. They help isolate decode cost, scheduler overhead, and serialization work. But the user-facing claim should come from the user-facing path. Otherwise you end up optimizing a component and calling it a product result. What is the first thing you add when you want a benchmark to reflect real user wait time instead of a convenient inner loop? Engineering fact: Lizard's native readiness benchmark sends concurrent requests through the OpenAI-compatible HTTP endpoint and measures full-response throughput instead of quoting only an internal worker loop. Lizard The AI Runtime You'll Own—Not Rent. Receive two professional Windows AI runtimes with lifetime updates. Run AI at native speed, keep every conversation private, and stay independent with intelligent hardware optimization and no cloud dependency. Read the relevant Lizard page #inference #benchmarking #httppath #latency #localai <!-- lizard-marketing-slot:day-11-am -->

Inference Engineering · Day 10 · Evening [image: Screenshot%202026-07-22%20222551.png] Ten days of inference engineering ends with the most important benchmark input: your question. Our published results belong to the hardware, model, quantization, prompt and token budget shown. They identify real bottlenecks; they do not create a universal provider ranking. What should we test next? Please include your CPU/GPU, available memory, model, quantization and the decision you need the benchmark to answer. Engineering fact: Published Lizard results are model-, quantization-, prompt-, hardware-, and run-specific; the next useful test should be selected with that context. Read the relevant Lizard page #Benchmarking #LocalAI #InferenceEngineering #LizardNative #CaterpillarEngine <!-- lizard-marketing-slot:day-10-pm -->

Inference Engineering · Day 9 · Evening [image: Screenshot%202026-07-22%20221220.png] A benchmark dashboard can mislead without containing a false number. Treat an excluded provider as zero. Mix warm-up into decode. Turn one metric winner into a universal recommendation. Each shortcut changes the story. Lizard keeps provider identity, wall time, end-to-end throughput, native decode, load, warm-up and memory separate. Missing providers stay unmeasured. The goal is not to remove judgment. It is to keep the evidence needed for judgment. Engineering fact: Lizard's dashboard keeps excluded providers unmeasured and separates wall time, end-to-end throughput, decode, warm-up, and memory. Read the relevant Lizard page #Benchmarking #DataIntegrity #LLMInference #Reproducibility #LocalAI <!-- lizard-marketing-slot:day-09-pm -->

Inference Engineering · Day 9 · Morning [image: Screenshot%202026-07-22%20213915.png] A signed GPU result must prove that the GPU path actually ran. When a remote benchmark model is larger than a discrete card's memory, a quiet CPU fallback would produce a number—but not the number requested. lizard-native can instead stream weights through the Direct3D 12 decode lane. It is slower than full residency, and the result says so, but the measurement remains a genuine DirectX path with adapter and backend evidence. A slower honest result is more useful than a faster mislabeled one. Engineering fact: For remote benchmarks on a discrete GPU smaller than the model, lizard-native can stream weights through the D3D12 lane rather than silently reporting a CPU fallback as GPU. Read the relevant Lizard page #D3D12 #GPUInference #Benchmarking #LizardNative #LocalAI <!-- lizard-marketing-slot:day-09-am -->

Inference Engineering · Day 8 · Morning [image: Screenshot%202026-07-22%20213816.png] A missing baseline should produce a failed preflight—not a blank treated as zero. In this run, the requested Ollama model was not installed. Lizard recorded the exact command, failure reason and exit status, then refused to manufacture an Ollama measurement. The later Native, Caterpillar and llama.cpp run is valid for those providers. Ollama remains unmeasured. Reliable benchmarking begins by preserving failure evidence. Engineering fact: When the requested Ollama model is absent, Lizard records the failed preflight and produces no synthetic Ollama measurement. Read the relevant Lizard page #Benchmarking #Reproducibility #LocalAI #LLMInference #LizardLLM <!-- lizard-marketing-slot:day-08-am -->

If the machine with the interesting GPU is not the one you sit at, you can now benchmark it from the one you are at. [image: remote-pairing.png] No inbound port Both computers open an outbound HTTPS/WSS connection to the relay. Neither machine accepts an inbound connection, so there is nothing to forward on your router and no inbound firewall rule to add. The relay queues messages between the two and never holds a model. Pairing On the computer doing the asking, press Add benchmark computer for a one-time code. Enter that code on the other PC. It sends its signed-in account email and public-key fingerprint through the relay. The host sees the email and the full key fingerprint, and approves or rejects it. A machine you have not approved cannot be asked to do anything. Request a DirectX probe, or a benchmark of the model selected on that client. The client confirms before anything runs. Read the result. What comes back is evidence, not a headline [image: remote-runs.png] Each completed run returns: model id and SHA-256, tokens per second and total ms, the prefill/decode split, peak RAM and VRAM, GPU adapter and driver version, and whether a fallback path was used. That last field is the one that matters. If the GPU path was not actually used, the run says so instead of quietly reporting a CPU number as a GPU result. What never crosses the network Model weights stay on the machine that runs them — never uploaded, never relayed. Prompts and answers stay local too; a remote run measures speed, not text. Availability Remote benchmarking is part of Pro. The five-day trial covers local benchmarks, the dashboard and both native engines; buying credits once adds remote runs and keeps them. Full details: https://lizard-llm.qendryx.com/remote.html Question: which two machines are you comparing, and how far apart did they land?

These analyzed captures put Lizard Native and Caterpillar in the foreground beside stock llama.cpp and optional Ollama. They also show the honest result: no provider wins every metric, and Caterpillar trails the other two lanes in these selected general-case runs. Interactive gallery: https://lizard-llm.qendryx.com/benchmarks.html Wall time, throughput and decode are different metrics [image: Screenshot%202026-07-22%20221126.png] llama.cpp wins this run's wall time and end-to-end throughput. Lizard Native leads native decode at 15.198 tok/s versus Caterpillar's 9.201. The dashboard does not blend those numbers. No provider wins every dimension [image: Screenshot%202026-07-22%20221157.png] The heatmap and radar show the trade: llama.cpp leads speed and memory efficiency here, while the native lanes carry their own decode and execution-lane evidence. The decision guide names the real winners [image: Screenshot%202026-07-22%20221220.png] For this hardware, llama.cpp Q4 is the balanced choice at 16.31 tok/s and 3.925 seconds. Lizard Native Q4 is called out separately for best native decode at 15.198 tok/s. Ollama was not included and remains unmeasured. A larger model changes the gap [image: Screenshot%202026-07-22%20222539.png] For Gemma 4 E4B, llama.cpp records 4.37 tok/s, Lizard Native 4.08, and Caterpillar 1.41. The Ollama card says not included and unmeasured—not zero. Provider identity stays attached to every metric [image: Screenshot%202026-07-22%20222551.png] Runtime, model load, warm-up, peak memory and efficiency stay under the provider that produced them. This is the evidence needed to evaluate Lizard as a local inference provider. Question for you: What hardware and model should we run next to test where Caterpillar closes the gap—or where Native's decode path matters most?

This sequence follows a provider comparison from a failed preflight to a successful run. It includes the exact command, warm-up state, partial progress and final rows, because an industrial benchmark should be reproducible and should not hide failure evidence. Interactive gallery: https://lizard-llm.qendryx.com/benchmarks.html A missing Ollama model stops before measurement [image: Screenshot%202026-07-22%20213816.png] The attempted triple-provider run exits because no matching Ollama model is installed. Lizard records the reason and exit code instead of inventing an Ollama result. The successful provider run begins [image: Screenshot%202026-07-22%20213915.png] The rerun shows the exact providers, quantization, prompt budget and warm-server reuse while Lizard Native starts its first row. Running and completed rows stay distinct [image: Screenshot%202026-07-22%20213952.png] Mid-run, Lizard Native Q4 measures 12.8 tok/s and Caterpillar Q4 8.23 while llama.cpp is still warming. A running lane is not presented as finished. The finished head-to-head [image: Screenshot%202026-07-22%20214012.png] For this Llama 3.2 Q4 run, stock llama.cpp reaches 16.31 tok/s, Lizard Native 12.8, and Caterpillar 8.23. Setup and warm-up fields explain the wall-time difference. Repeatable recipes instead of hidden presets [image: Screenshot%202026-07-22%20214040.png] Single-pass, full-sweep and triple-stack recipes are visible and editable. The command center makes the intended comparison explicit before it consumes a run. Question for you: Should the next public run include an installed Ollama baseline, and if so which exact Ollama model tag should we use?