hardenedlinux /
SonicBoom
A compiler-first PyTorch alternative. MLIR-based Boom IR, Guile scripting, CUDA and CPU. On consumer GPUs it already beats llama.cpp’s LLM decode with the same kernels; the compiler makes the difference. GPLv3.
Updated 42 min ago
+1 stars gained this week
Stars gained each day, last 30 days
Usual pace · 1 / day
Oct 4Oct 19Today
This week Before Usual pace
In short
It is a C++ and CUDA runtime with a stable C ABI, so you load libsonicboom.so from any language without Python. Local LLM inference on consumer NVIDIA GPUs is its main target.
Summary written by AI from the repository’s description and README.
hardenedlinux / SonicBoom
View on GitHub(opens in new tab)