OrbitInfra /
OrbitKV
High Performance KV Cache for LLM Inference
Updated 34 min ago
Stars gained each day, last 30 days
Usual pace · 3 / day
Oct 4Oct 19Today
This week Before Usual pace
In short
OrbitKV extends the KV cache of vLLM and SGLang beyond GPU memory by storing reusable prefixes in DRAM or SSD. It is for teams serving repeated documents, shared system prompts, or long conversations.
Summary written by AI from the repository’s description and README.
OrbitInfra / OrbitKV
View on GitHub(opens in new tab)