<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Running a model that's bigger than your graphics card]]></title><description><![CDATA[<p dir="auto">You don't need a card big enough to hold the whole model.</p>
<p dir="auto">When a model doesn't fit, Lizard has two honest options and it will tell you which one it took:</p>
<ol>
<li><strong>Stream the weights through the GPU.</strong> The card still does the work, but it reads parts of the model as it goes. Slower than fully resident, but it's still a real GPU result.</li>
<li><strong>Hand it to Caterpillar</strong>, which is built for CPU and shared graphics.</li>
</ol>
<p dir="auto">What it deliberately will <strong>not</strong> do is quietly pin an oversized model into shared memory and let it swap — that produces a machine that feels broken for no visible reason. If your model is too big for comfort, Lizard now says so and picks a supported path instead.</p>
<p dir="auto">Rule of thumb: a Q4 model needs roughly 60% of the download size in free graphics memory to stay fully resident.</p>
<p dir="auto">Anyone running a large model on a small card? What did you settle on?</p>
]]></description><link>https://community.lizard-llm.qendryx.com/topic/5/running-a-model-that-s-bigger-than-your-graphics-card</link><generator>RSS for Node</generator><lastBuildDate>Mon, 24 Aug 2026 01:36:17 GMT</lastBuildDate><atom:link href="https://community.lizard-llm.qendryx.com/topic/5.rss" rel="self" type="application/rss+xml"/><pubDate>Thu, 16 Jul 2026 10:25:00 GMT</pubDate><ttl>60</ttl></channel></rss>