If Lizard is an inference provider, the fair question is which parts of the stack it actually owns. Here is the answer, top to bottom. [image: lizard-native-stack.png] The API layer speaks the OpenAI Chat Completions format. This is deliberate: it means the client libraries you already use work by changing a base URL, and it means you are not locked in — anything you build against Lizard runs against a hosted provider too, and vice versa. Scheduling and batching is where a lot of local runtimes lose time. The naive approach submits work to the GPU many times per token, and each submission costs you. lizard-native compiles the decoder graph into coalesced spans ahead of time and gets that down to roughly one submission per generated token. The KV cache is resident and its precision is yours to pick — f16, q8_0, q4_0. This is the setting nobody exposes and everybody should: it is a direct trade between context length and memory. A hosted provider makes that choice for you and you never see it. The kernels are where the model's weights get multiplied. These are quantization-aware — Q4_K, Q5_K, Q6_K — with the elementwise work fused into the matrix multiply so a token costs fewer passes over memory. The device layer is Direct3D 12, and the important word is resident: the weights are uploaded once and stay in GPU memory. Loading is the expensive part; after the first time, the shaders are cached on disk too, so a load is seconds rather than a minute. Why this matters for a developer: every one of those layers is a decision that determines your tokens per second, and on a hosted API all of them are made somewhere you cannot see or change. Locally, they are yours — the KV cache precision and the quantization level are literally settings in the app. The practical ceiling is memory bandwidth, not arithmetic. Generating a token means reading the whole model out of memory, so on most machines the model's size divided by your memory bandwidth is the speed limit. That is also why a smaller quantization is often faster, not just smaller.

When you call an LLM API, it is easy to assume the company you are calling is the one running the model. Often it isn't, and the difference matters more than it looks. [image: tokens-origin.png] An API router is a proxy. It owns no compute. Your request arrives, gets forwarded to whichever upstream provider it has a deal with, and the response comes back through it. Useful — one key, many models, automatic failover when an endpoint degrades. But every request takes an extra network hop, so median time-to-first-token is always worse than calling that same provider directly. Routers improve tail latency by rerouting around bad endpoints; they cannot improve the median. Nothing they do touches the decisions that actually determine speed. An inference provider owns the hardware. The same company controls the endpoint and the GPUs your tokens are computed on. They choose the batching strategy, the KV cache layout, the kernels, the quantization. Those choices are what your latency and throughput actually come from. There is a second, quieter difference. A router can only bind itself. If you sign a zero-retention agreement with a router, your request still lands at an upstream provider whose policies you never reviewed. The agreement doesn't travel with your data. Lizard is an inference provider where the provider is your own machine. The endpoint is 127.0.0.1:7817, the GPU is the one in your computer, and there is no upstream at all — no hop to add latency, no third party to have a retention policy, no agreement chain to audit. When you turn off your Wi-Fi, it keeps working. That is the honest test of the claim, and you can run it yourself. The trade is real and worth stating: you get one machine's worth of compute, not a fleet. A hosted provider will out-run your laptop on a 70B model. What you get instead is that nothing leaves the building. How do you tell which kind you are using? Read the docs. "Our clusters", "our GPUs" means a provider. "Access 200+ models from leading providers" means a router. In a DPA, any mention of sub-processors or third-party infrastructure partners is a router tell — a direct provider has no such chain.