Caterpillar benchmark CLI and HTTP command reference

[lizardadmin]

Caterpillar benchmark CLI and HTTP guide

The provider flag is explicit and verified. The health response and artifact
must both report runtimeProvider: caterpillar; the harness fails on a
provider mismatch.

npm run bench:native:http -- `
  --provider caterpillar `
  --model llama-3.2-3b-instruct `
  --gguf "$env:USERPROFILE\.lizard-llm\models\llama-3.2-3b-instruct.gguf" `
  --batch "16,8,4,1" `
  --tokens 64 `
  --label caterpillar-llama32-same-weights-http

Use --skip-llama for Caterpillar-only tuning. Verify the runtime provider
before interpreting any number.

Adaptive concurrency

The native policy uses B=16 as the aggregate-throughput target and B=8 as
the normal secondary lane. B=4 and B=1 are used only when both primary lanes
are unavailable. Automatic result selection chooses the faster successful
B=16/B=8 measurement; a failed or token-unmeasured lane is never promoted.

What a valid llama.cpp ratio requires

The candidate and control must load the same physical GGUF file. Keep the
model, quantization, prompt, output token budget, concurrency, warm-up and
machine conditions fixed. A Llama 3.2 candidate cannot be compared with a Gemma
control. The HTTP harness refuses a mismatched canonical path.

The artifact records provider identity, candidate/control paths, completion and
failure counts, aggregate completion tokens per second, full-response latency,
lane selection and provenance. The current non-streaming harness does not claim
TTFT or TPOT.

Read the complete command reference

The public guide includes copy buttons, parameters, output interpretation,
throughput and latency runs, serving, offline and online checks, repeated-prefix
workloads, sequence-length and stress runs, production checks, and current
multi-GPU/tensor-parallel/data-parallel limitations:

https://lizard-llm.qendryx.com/benchmark-cli.html

Benchmark methodology and screenshots: https://lizard-llm.qendryx.com/benchmarks.html
Technical architecture: https://lizard-llm.qendryx.com/technical-overview.html