Direct and HTTP benchmarks answer different questions, and the label matters more than people admit.

A direct run is useful when you want to isolate the runtime path itself: the model load, the worker protocol, and the engine work without the rest of the request path in the way. That makes it a good tool for diagnosing a regression inside the runtime or comparing two engine changes under the same conditions.

An HTTP run adds the part a user actually feels: request handling, server behavior, and the served API path. If the question is, “How does this model behave when clients talk to it through the product?”, HTTP is the measurement that matches the claim. Lizard exposes both modes separately, and the UI points to HTTP when the goal is end-user serving performance.

That distinction saves teams from a common mistake: reporting a number that is technically true but answers the wrong question. The fastest path inside the engine is not always the best proxy for a real deployment.

If I am trying to decide whether a server change is worth shipping, I want the served measurement. If I am trying to understand where time is going, I want the direct one first.

What benchmark mode do you reach for when you need to separate engine cost from serving cost?

Engineering fact: Lizard exposes native direct and native HTTP benchmark modes separately; the UI recommends HTTP when the goal is end-user serving performance.

Lizard The AI Runtime You'll Own—Not Rent.

Receive two professional Windows AI runtimes with lifetime updates. Run AI at native speed, keep every conversation private, and stay independent with intelligent hardware optimization and no cloud dependency.

Read the relevant Lizard page

#benchmarking #httptest #llmruntime #localai #performanceengineering

<!-- lizard-marketing-slot:day-22-am -->