Using Lizard from your own code (Python, Node, curl)
-
Lizard serves the same API as OpenAI, on your machine. If you already have code written against a cloud AI service, you usually change one line.
Start the server first:
lizard-llm serve --model activePython
from openai import OpenAI client = OpenAI(base_url="http://127.0.0.1:8013/v1", api_key="lizard-local") res = client.chat.completions.create( model="active", messages=[{"role": "user", "content": "Hello"}], ) print(res.choices[0].message.content)Node.js
import OpenAI from "openai"; const client = new OpenAI({ baseURL: "http://127.0.0.1:8013/v1", apiKey: "lizard-local" });curl
curl http://127.0.0.1:8013/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model":"active","messages":[{"role":"user","content":"Hi"}]}'The key is ignored — nothing leaves your computer — but most clients insist on a value, so give it anything.
Anything that reads
OPENAI_BASE_URL(LangChain, LlamaIndex, lots of editor plugins) can point at Lizard the same way:OPENAI_BASE_URL=http://127.0.0.1:8013/v1 OPENAI_API_KEY=lizard-localFull docs: https://lizard-llm.qendryx.com/docs.html
What have you wired it into?
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login