The same cache can be a product feature and a benchmark contaminant.

A conversation naturally repeats its previous prefix, so retaining prompt state can avoid reprocessing the whole transcript. A benchmark that repeats an identical prompt for measurement should not accidentally turn that reuse into a fake prefill win.

Cache policy should follow workload semantics, not simply remain globally enabled.

Which caches in your stack need different rules for production and benchmarking?

Engineering fact: Lizard Chat can reuse prior prompt state for the same conversation while benchmark paths avoid chat-only cache behavior that would make repeated prompts look artificially cheap.

Read the relevant Lizard page

#LizardNative #LizardLLM #LocalAI #GGUF #InferenceEngineering

<!-- lizard-marketing-slot:day-21-pm -->