#llm-serving
LLM serving on GitHub, November 2–8, 2026
The week of November 2, 159 repos made the LLM serving ranking, with 36,420 stars gained: 33% more than the week before. synchopate/llama-modchip climbed to #2, up from #21, gaining 411 stars. The top 10 had 8 repos that were not in it the week before, among them yetone/magpie at #3, in its first week in the topic. Continuum-AI-Corp/OrcaRouter-Lite joined the topic's ranking that week at #15 with 1,535 stars, and has more than doubled them since.
The ranking as the week’s last hourly run left it, on November 9 at 00:00 UTC. It does not change.
- repos ranked
- 159
- stars gained +33% vs the week before
- 36,420
- new in the top 10
- 8 of 10
- ranked in #llm-serving for the first time
- 32
The top 10
Ordered by stars gained over the seven days relative to each repo’s size, which is why a repo can rank above one that gained more.
- 1Prefrontal-AI/latent-sparse-attention Code for Latent Sparse Attention: Towards Subquadratic Ultra-Long Context Attentionstars that week +441
- 2synchopate/llama-modchip Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).stars that week +411
- 3yetone/magpie Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.stars that week +4.4k
- 4FareedKhan-dev/kimi-k3-in-c A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.stars that week +2.1k
- 5Niko1221/Strata Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.stars that week +4.1k
- 6ShawnZhuge/aiotto 面向 OpenAI Codex 的桌面伴侣 —— 账号额度、中转路由、会话与用量,一个应用搞定。🦦stars that week +206
- 7jesdga95/basalt Local inference engine optimized for NVIDIA Blackwellstars that week +351
- 8tashfeenahmed/freellmapi 10.1 billion tokens per month. 87 free LLM providers. 1,313 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.stars that week +3.4k
6 repos of this ranking were removed since and are not shown: deleted from GitHub or taken out by Upstar’s review. Their ranks stay empty.
Ranks 11 to 30
- 11RustingStudio/RustingStratanot ranked the week before +134 stars that week
- 12Magnvs7/Promethevsnot ranked the week before +125 stars that week
- 13Alice-Marx/dsh-model-routerup from #14 the week before +304 stars that week
- 14undrgrnd-labs/hunchnot ranked the week before +203 stars that week
- 15Continuum-AI-Corp/OrcaRouter-Litenot ranked the week before +398 stars that week
- 16FlashML-org/FreeTokenup from #37 the week before +1.2k stars that week
- 17Generated-Computers/Localsup from #36 the week before +200 stars that week
- 18ThinkWatchProject/ThinkWatch-Liteup from #25 the week before +231 stars that week
- 19StartLuxLabs/StartLux-Prismdown from #4 the week before +136 stars that week
- 20cheaper-inference/OmniRoutedown from #16 the week before +2.1k stars that week
- 21christopherowen/spark-ds41fup from #52 the week before +74 stars that week
- 22Zihao-Wu06/distillopsnot ranked the week before +72 stars that week
- 23AIAgentAndy/DeepAAnot ranked the week before +70 stars that week
- 24simplexai-labs/PolyStratadown from #11 the week before +177 stars that week
- 25respanai/respan-gatewaydown from #15 the week before +141 stars that week
- 26Wallawalla47/Infernixup from #54 the week before +89 stars that week
- 27mw00/project-mayadown from #10 the week before +303 stars that week
- 29JakeATX/llamAmpereup from #88 the week before +88 stars that week
- 30dayuwang401/spectral-orthogonal-explorationdown from #26 the week before +60 stars that week
Show the other 126 repos
- 31deepseek-ai/DeepInferdown from #5 the week before +785 stars that week
- 32nace-ai/kapitsaup from #34 the week before +116 stars that week
- 3301men/ybkkasidown from #30 the week before +47 stars that week
- 34affifuddin-hazam/ghost-infernot ranked the week before +46 stars that week
- 36prabanta-dev/janasup from #38 the week before +105 stars that week
- 38openlake-project/openlakenot ranked the week before +218 stars that week
- 39drawthingsai/local-code-communitydown from #17 the week before +42 stars that week
- 40bitcloud-ai/Bittunedown from #23 the week before +51 stars that week
- 41skydiscover-ai/jitenginenot ranked the week before +39 stars that week
- 42davanstrien/saturatenot ranked the week before +37 stars that week
- 43atdev-company/atdev-servenot ranked the week before +37 stars that week
- 44LAGcomcom/Lamp-Guidenot ranked the week before +36 stars that week
- 46patchy631/time-to-first-tokennot ranked the week before +105 stars that week
- 47Lowaltitudeflight/local-llm-tuning-guidenot ranked the week before +33 stars that week
- 48yanstd888/yanstd888.github.ionot ranked the week before +33 stars that week
- 49ahtx/kindidown from #33 the week before +34 stars that week
- 50incoai/splashdown from #39 the week before +138 stars that week
- 510xShug0/audio.cppup from #59 the week before +192 stars that week
- 52WizisCool/oh-my-cpadown from #47 the week before +50 stars that week
- 53musicurgy/delta-inference-enginedown from #31 the week before +29 stars that week
- 54syv-ai/HyperQwendown from #32 the week before +144 stars that week
- 55ashhart/TensorFoldup from #61 the week before +113 stars that week
- 56davide-troiani/moeshiftnot ranked the week before +28 stars that week
- 57JaydenTeoh/Soft-MTPnot ranked the week before +28 stars that week
- 58ekzhang/resindown from #1 the week before +53 stars that week
- 59higress-group/HiRouteup from #95 the week before +53 stars that week
- 60mozilla-ai/otarinot ranked the week before +74 stars that week
- 61Incept5-AI/inferdnot ranked the week before +26 stars that week
- 62drumih/turbo-fieldfareup from #118 the week before +218 stars that week
- 63lyogavin/airllmdown from #41 the week before +492 stars that week
- 64benmaster82/Pithosnot ranked the week before +24 stars that week
- 65skyzh/tiny-llmdown from #12 the week before +178 stars that week
- 66rednote-machine-learning/RedKnotdown from #24 the week before +131 stars that week
- 67Imironman123/llm-interview-notesnot ranked the week before +23 stars that week
- 68ZhuLinsen/LiteMoEnot ranked the week before +23 stars that week
- 69Indras-Mirror/Xolotldown from #40 the week before +48 stars that week
- 70peonist-ai/halogen-flash-serverdown from #68 the week before +78 stars that week
- 71agentgateway/agentgatewaydown from #55 the week before +173 stars that week
- 72shuishuipingan/qoder2api-hubdown from #29 the week before +49 stars that week
- 73zexadev/gemini-web2api-godown from #65 the week before +59 stars that week
- 74NVIDIA/hyperflare-inferencedown from #62 the week before +58 stars that week
- 75ddalcu/mlx-servedown from #42 the week before +101 stars that week
- 76OmniJev/Flidenot ranked the week before +21 stars that week
- 77pegainfer-project/kernnot ranked the week before +25 stars that week
- 78ggml-org/llama.cppdown from #70 the week before +742 stars that week
- 79giienew/9router-prodown from #20 the week before +20 stars that week
- 80ai-gateways/awesome-ai-gatewaysnot ranked the week before +20 stars that week
- 81QuantumNous/new-apidown from #71 the week before +447 stars that week
- 82decolua/9routerdown from #64 the week before +352 stars that week
- 83eliasbiondo/antiriondown from #3 the week before +40 stars that week
- 84panxunying/ai-coding-welfareup from #93 the week before +67 stars that week
- 85bojieli/ai-infra-bookdown from #58 the week before +161 stars that week
- 86amitshekhariitbhu/ai-engineering-coursedown from #72 the week before +45 stars that week
- 87vllm-project/vllm-omniup from #90 the week before +150 stars that week
- 88BerriAI/litellmdown from #76 the week before +425 stars that week
- 89maximhq/bifrostdown from #78 the week before +163 stars that week
- 90vllm-project/vllmdown from #79 the week before +492 stars that week
- 91MemTensor/MemT-1.0down from #43 the week before +35 stars that week
- 92Orchestra-Research/AI-Research-SKILLsdown from #77 the week before +184 stars that week
- 93eagle-hq/EAGLE-4down from #48 the week before +51 stars that week
- 94rkinas/basalnot ranked the week before +26 stars that week
- 95datawhalechina/zero-to-sglangdown from #84 the week before +62 stars that week
- 96StartLuxLabs/StartLux-Decisiondown from #82 the week before +61 stars that week
- 97lemonade-sdk/lemonadeup from #127 the week before +113 stars that week
- 98lidge-jun/opencodexdown from #75 the week before +194 stars that week
- 99jundot/omlxdown from #86 the week before +197 stars that week
- 100gitleaks/gitleaksdown from #87 the week before +222 stars that week
- 101recursive-oss/recdown from #56 the week before +20 stars that week
- 102trymirai/uzudown from #8 the week before +69 stars that week
- 103T-Crypt/berthdown from #35 the week before +20 stars that week
- 104sgl-project/sglang-omnidown from #102 the week before +48 stars that week
- 105nokia-applied-research/AnyJevdown from #83 the week before +46 stars that week
- 106AtomicBot-ai/Atomic-Chatdown from #85 the week before +51 stars that week
- 107magnitudedev/magnitudedown from #46 the week before +100 stars that week
- 108verl-project/verl-omnidown from #81 the week before +43 stars that week
- 109pallavi-shekhar/ai-engineering-interview-questions-company-wisedown from #66 the week before +50 stars that week
- 1101CatAI/1Cat-vLLMdown from #92 the week before +41 stars that week
- 111lbjlaq/Antigravity-Managerdown from #94 the week before +194 stars that week
- 112SemiAnalysisAI/InferenceXup from #122 the week before +47 stars that week
- 113Calcium-Ion/AstrLinkdown from #13 the week before +22 stars that week
- 114w8123/EnterpriseAgentFrameworkdown from #73 the week before +30 stars that week
- 115NightMean/OlliteRTdown from #109 the week before +23 stars that week
- 116XHToken/Spark-X2.6down from #100 the week before +27 stars that week
- 117ARahim3/kaggle-tpu-labdown from #106 the week before +24 stars that week
- 118lightseekorg/tokenspeeddown from #97 the week before +46 stars that week
- 119alibaba/MNNup from #135 the week before +110 stars that week
- 120vllm-project/vllm-metaldown from #117 the week before +34 stars that week
- 121gpustack/gpustackup from #123 the week before +59 stars that week
- 122warpfront/hipfiredown from #98 the week before +21 stars that week
- 123seakee/CPA-Manager-Plusdown from #110 the week before +46 stars that week
- 124StayLameBro/backburnerdown from #114 the week before +24 stars that week
- 125kvcache-ai/Mooncakedown from #112 the week before +58 stars that week
- 126vllm-project/semantic-routerdown from #91 the week before +55 stars that week
- 127antirez/ds4down from #116 the week before +107 stars that week
- 128antirez/h3.cdown from #121 the week before +37 stars that week
- 129fosrl/pangolinup from #134 the week before +99 stars that week
- 130musistudio/claude-code-routerdown from #129 the week before +122 stars that week
- 131flashinfer-ai/flashinferdown from #128 the week before +50 stars that week
- 132algorithmicsuperintelligence/openevolveup from #139 the week before +53 stars that week
- 133exo-explore/exodown from #131 the week before +124 stars that week
- 134LMCache/LMCachedown from #113 the week before +56 stars that week
- 135IBM/mcp-context-forgedown from #124 the week before +33 stars that week
- 136amitshekhariitbhu/ai-engineering-interview-questionsdown from #120 the week before +28 stars that week
- 137NVIDIA/TensorRT-LLMup from #141 the week before +57 stars that week
- 138youssofal/MTPLXdown from #126 the week before +24 stars that week
- 139vllm-project/vllm-ascenddown from #133 the week before +25 stars that week
- 140Portkey-AI/gatewayup from #150 the week before +52 stars that week
- 141ai-dynamo/dynamodown from #138 the week before +41 stars that week
- 142casdoor/casdoordown from #130 the week before +54 stars that week
- 143future-agi/future-agidown from #115 the week before +20 stars that week
- 144raullenchai/Rapid-MLXdown from #107 the week before +27 stars that week
- 145sqliteai/warpnot ranked the week before +21 stars that week
- 146EricLBuehler/mistral.rsnot ranked the week before +31 stars that week
- 147superlinked/sienot ranked the week before +20 stars that week
- 148InsForge/InsForgedown from #125 the week before +38 stars that week
- 149looplj/axonhubdown from #145 the week before +24 stars that week
- 150openvinotoolkit/openvinodown from #140 the week before +33 stars that week
- 151skypilot-org/skypilotdown from #148 the week before +32 stars that week
- 152kserve/kservedown from #147 the week before +24 stars that week
- 153deepseek-ai/FlashMLAdown from #142 the week before +30 stars that week
- 154cactus-compute/enginenot ranked the week before +20 stars that week
- 155budtmo/docker-androiddown from #137 the week before +32 stars that week
- 156tbphp/gpt-loaddown from #144 the week before +21 stars that week
- 157apache/apisixdown from #151 the week before +24 stars that week
- 158mlc-ai/web-llmdown from #149 the week before +23 stars that week
- 159kvcache-ai/ktransformersdown from #152 the week before +23 stars that week