1115714829 /
llama.cpp-v100
llama.cpp optimized for NVIDIA Tesla V100 (SM70): Qwen3.8-Flash-Next with MTP (prefill 3402 tok/s at 200K context on 6x V100-16GB; also runs on 2x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 220 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving, built-in web UI.
Updated 53 min ago
+9 stars gained this week
Stars gained each day, last 30 days
Usual pace · 2 / day
Oct 4Oct 19Today
This week Before Usual pace
In short
It is aimed at V100 owners running Qwen3.8 models locally. It needs 2 to 6 V100 GPUs, about 63 GiB of host RAM for the 2-card setup, and is built from source or run via the Docker image.
Summary written by AI from the repository’s description and README.
1115714829 / llama.cpp-v100
View on GitHub(opens in new tab)