wici-ai /
inferssd
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Updated 2 min ago
Maintain this repo? Add this badge to your README.
Why now
Version v0.1.1 came out on October 31.
+227 stars gained this week
Stars gained each day, last 30 days
Oct 4Oct 19Today
This week Before
In short
Needs a Mac with Apple Silicon (Metal) or a Linux machine with an NVIDIA GPU (CUDA), and is built from source. It serves a browser chat and an OpenAI-compatible API for the DeepSeek and Muse models.
Summary written by AI from the repository’s description and README.
wici-ai / inferssd
View on GitHub(opens in new tab)