ahtx /
kindi
A custom LLM inference engine for low-cost GPUs: NVIDIA cards from a 1 GB Quadro K420 to an RTX 3080, the AMD Radeon 780M and the CPU. Runs the Bonsai 2B to 27B models and Nemotron with an OpenAI-compatible server.
Updated 28 min ago
Why now
Version v0.3.0 came out on October 29.
+50 stars gained this week
Stars gained each day, last 30 days
Oct 4Oct 19Today
This week Before
In short
Aimed at anyone running local models on older or cheap hardware: it needs an NVIDIA or AMD GPU, or just a CPU, and it serves an OpenAI-compatible API that chat clients and agents can call.
Summary written by AI from the repository’s description and README.
ahtx / kindi
View on GitHub(opens in new tab)