jundot /
omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Updated 44 min ago
Why now
Version v0.7.3 came out on October 31.
+232 stars gained this week
Stars gained each day, last 30 days
Usual pace · 29 / day
Oct 4Oct 19Today
This week Before Usual pace
In short
It needs a Mac with Apple Silicon running macOS 15 or later, and Python 3.11 to 3.13 if built from source. Use it through the menu bar app or a terminal omlx command, with an OpenAI-compatible API.
Summary written by AI from the repository’s description and README.
jundot / omlx
View on GitHub(opens in new tab)