Upstar classe toutes les heures les projets GitHub qui montent le plus vite, dans 69 thèmes. Voir les thèmes

#llm-serving · IA & ML

Inférence & passerelles LLM

Moteurs d’inférence, serveurs de modèles et passerelles qui répartissent les requêtes entre les API de modèles.

Cette semaine (27 oct. – 2 nov.), en tête : Strata (+4 653), antirion (+297) et StartLux-Prism (+146). Les 150 projets classés ont gagné 27 498 étoiles, 48 % de moins que la semaine précédente.

Mis à jour le 2 nov., 15:42 UTC · classé toutes les heures à partir des étoiles quotidiennes de GitHub

Mis à jour il y a 31 min

Sujet GitHub ↗ (nouvel onglet)

Classement de la semaine

150 sur 150 affichés

Comment Upstar classe

  1. 1
    Niko1221/Strata Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input. C++ C++ · 38,2 k au total
    Gagnées en 7 j +4,7 k Score 25,4 Total 38,2 k
  2. 2
    eliasbiondo/antirion 🕷️ The fastest open-source AI Gateway and LLM Gateway — one binary with built-in console to route, secure, budget and observe every LLM call across OpenAI, Anthropic, Gemini, Bedrock and more. Rust Rust · 622 au total Repéré à 95 étoiles
    Gagnées en 7 j +297 Score 16,5 Total 622
  3. 3
    StartLuxLabs/StartLux-Prism Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers. Python Python · 238 au total
    Gagnées en 7 j +146 Score 14,6 Total 238
  4. 4
    jesdga95/basalt Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell C++ C++ · 448 au total
    Gagnées en 7 j +218 Score 14,4 Total 448
  5. 5
    ekzhang/resin 🧱 Experimental library for building inference engines Python Python · 369 au total En bonne santé
    Gagnées en 7 j +187 Score 13,9 Total 369
  6. 6
    deepseek-ai/DeepInfer DeepInfer: High-performance, large-scale inference framework for DeepSeek models Python Python · 25 k au total
    Gagnées en 7 j +2,1 k Score 13,5 Total 25 k
  7. 7
    NerverDevs/nervecenter new-api / sub2api 中转站账号管理中心：余额、API Key、用量、倍率、模型比价 | https://nvcenter.cc TypeScript TypeScript · 199 au total
    Gagnées en 7 j +135 Score 13,5 Total 199
  8. 8
    trymirai/uzu A high-performance inference engine for AI models Rust Rust · 3,2 k au total En bonne santé
    Gagnées en 7 j +601 Score 11,8 Total 3,2 k
  9. 9
    Alice-Marx/dsh-model-router Model Router and GALGame plugin for DeepSeek Harness JavaScript JavaScript · 637 au total En bonne santé
    Gagnées en 7 j +234 Score 11,7 Total 637
  10. 10
    skyzh/tiny-llm learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen Python Python · 5,5 k au total En bonne santé
    Gagnées en 7 j +731 Score 10,6 Total 5,5 k
  11. 11
    VecSzn/PolyStrata Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost. C++ C++ · 762 au total Repéré à 196 étoiles
    Gagnées en 7 j +227 Score 9,8 Total 762
  12. 12
    mw00/project-maya Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools. Cuda Cuda · 2,4 k au total Repéré à 762 étoiles
    Gagnées en 7 j +432 Score 9,7 Total 2,4 k
  13. 13
    drawthingsai/local-code-community Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built into the app. C C · 90 au total
    Gagnées en 7 j +90 Score 9,0 Total 90
  14. 14
    FlashML-org/FreeToken FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently. Python Python · 15,8 k au total En bonne santé
    Gagnées en 7 j +1,1 k Score 9,0 Total 15,8 k
  15. 15
    cheaper-inference/OmniRoute Never stop coding. Free MIT AI gateway: one endpoint, 375 providers (150+ free), 1,400+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A… TypeScript TypeScript · 83,9 k au total En bonne santé
    Gagnées en 7 j +2,5 k Score 8,7 Total 83,9 k
  16. 16
    respanai/respan-gateway Open-source LLM gateway in Rust. One endpoint for OpenAI, Anthropic, Gemini and every other vendor. Any SDK, any model, any tool. Rust Rust · 489 au total
    Gagnées en 7 j +156 Score 8,5 Total 489
  17. 17
    kekzl/imp Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one… C++ C++ · 124 au total En bonne santé
    Gagnées en 7 j +77 Score 7,7 Total 124
  18. 18
    routerhubSI/uncensored-ai-api Uncensored AI API and models — one key for Claude, GPT, Gemini, Grok, DeepSeek, GLM and Qwen, with uncensored and abliterated variants where available. OpenAI- and Anthropic-compatible. Pay per token. 509 au total
    Gagnées en 7 j +144 Score 7,5 Total 509
  19. 19
    synchopate/nemotron-30b-cpu-256k Running a 30B MoE LLM on CPU at 30-50+ tok/s with a 256K Context Window via Hybrid Mamba-2 SSM, L3 NTA Slicing, AVX-512 VNNI Tile Repacking & 2MB HugePages Python Python · 70 au total Nouveau
    Gagnées en 7 j +70 Score 7,0 Total 70
  20. 20
    giienew/9router-pro 9Router Pro by Gienetic — Pro edition of 9Router with Prompt Injection (system prompt or chat script per global / API key / model), per-key token quotas, accent themes and an optimized dashboard. JavaScript JavaScript · 83 au total
    Gagnées en 7 j +69 Score 6,9 Total 83
  21. 21
    Calcium-Ion/AstrLink A local AI gateway for AI agents, unifying your subscriptions and API providers with smart routing and on-device privacy protection. Go Go · 524 au total
    Gagnées en 7 j +132 Score 6,7 Total 524
  22. 22
    ThinkWatchProject/ThinkWatch-Lite Local gateway for Claude Code, Codex and other AI clients on macOS, Windows and Linux: switch upstreams without touching clients, keep API keys from relays, cut off malicious tool calls, trace every request. TypeScript TypeScript · 534 au total
    Gagnées en 7 j +133 Score 6,6 Total 534
  23. 23
    bitcloud-ai/Bittune Local-first engineering agent that deploys, benchmarks and tunes vLLM/SGLang inference services on your own GPUs, with auditable run evidence. TypeScript TypeScript · 161 au total Signaux mitigés
    Gagnées en 7 j +64 Score 6,4 Total 161
  24. 24
    rednote-machine-learning/RedKnot Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention Python Python · 3,1 k au total Signaux mitigés
    Gagnées en 7 j +327 Score 6,3 Total 3,1 k
  25. 25
    mnfst/awesome-free-llm-apis List of Permanent Free LLM API (API Keys) JavaScript JavaScript · 12 k au total Signaux mitigés
    Gagnées en 7 j +640 Score 6,0 Total 12 k
  26. 26
    dayuwang401/spectral-orthogonal-exploration Official code for "Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration" (SOE, ACL 2026 Oral). A training-free, test-time scaling method that fixes reasoning collapse in LLM mathematical reasoning via orthogonal probing. Python Python · 134 au total
    Gagnées en 7 j +59 Score 5,9 Total 134
  27. 27
    Wallawalla47/Infernix C++/CUDA single-GPU inference engine for Qwen3.8 (incl. Qwen3.8-Flash-Next with offloaded experts) on the RTX 5090; grew from NInfer C++ C++ · 218 au total
    Gagnées en 7 j +70 Score 5,8 Total 218
  28. 28
    cablate/model-relay Per-window model switcher, subagent router, and automatic 429 failover for Claude Code — switch any window with /provider, delegate subagents across models, and keep Remote Control & statuslines working. JavaScript JavaScript · 103 au total
    Gagnées en 7 j +57 Score 5,7 Total 103
  29. 29
    musicurgy/delta-inference-engine Delta Inference Engine: DeepSeek-V4.1-Flash on one NVIDIA GPU, natively on Windows (OpenAI-compatible server) C++ C++ · 53 au total
    Gagnées en 7 j +53 Score 5,3 Total 53
  30. 30
    shuishuipingan/qoder2api-hub Qoder国际国内多账号反代网关，Qoder 国际国内双区多账号 OpenAI 兼容网关中枢 — COSY 签名推理、稳定设备指纹防风控、OAuth 设备授权免客户端登录、每日签到与 Pro 福利、后台定时调度器、Web 监控看板，支持 Codex / Claude Code 与标准 OpenAI 客户端 Python Python · 501 au total
    Gagnées en 7 j +105 Score 5,3 Total 501
  31. 31
    01men/ybkkasi One control plane for every AI asset — MCP servers, skills, agents, AI apps. Self-hosted ledger with lifecycle state machine, 85 permission points, department × agent × skill cost attribution, and audit trail. MIT, zero runtime deps (Node 22 + node:sqlite). 企业 AI 资产治理。 JavaScript JavaScript · 58 au total
    Gagnées en 7 j +52 Score 5,2 Total 58
  32. 32
    nace-ai/kapitsa Blazing fast Apple Silicon optimized inference engine built for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai C++ C++ · 598 au total
    Gagnées en 7 j +114 Score 5,2 Total 598
  33. 33
    syv-ai/HyperQwen Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks. Python Python · 2,5 k au total En bonne santé
    Gagnées en 7 j +241 Score 5,1 Total 2,5 k
  34. 34
    Generated-Computers/Locals Run many AI agents at once on your Mac, up to 3x faster. Python Python · 448 au total
    Gagnées en 7 j +95 Score 5,1 Total 448
  35. 35
    prabanta-dev/janas 37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU. C C · 574 au total
    Gagnées en 7 j +105 Score 4,8 Total 574
  36. 36
    ddalcu/mlx-serve Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation. Zig Zig · 2,3 k au total En bonne santé
    Gagnées en 7 j +218 Score 4,8 Total 2,3 k
  37. 37
    ahtx/kindi A custom LLM inference engine for low-cost GPUs: NVIDIA cards from a 1 GB Quadro K420 to an RTX 3080, the AMD Radeon 780M and the CPU. Runs the Bonsai 2B to 27B models and Nemotron with an OpenAI-compatible server. Python Python · 114 au total
    Gagnées en 7 j +47 Score 4,7 Total 114
  38. 38
    T-Crypt/berth Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off. Go Go · 282 au total
    Gagnées en 7 j +66 Score 4,5 Total 282
  39. 39
    Indras-Mirror/Xolotl Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up to ~27 tok/s, 512K context, October 1017 builds included. Plus GLM-5.3-Flash and Qwen3.8. MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic… C++ C++ · 441 au total
    Gagnées en 7 j +82 Score 4,3 Total 441
  40. 40
    incoai/splash A local inference engine for Apple silicon, built around the model. C++ C++ · 2 k au total
    Gagnées en 7 j +180 Score 4,2 Total 2 k
  41. 41
    lyogavin/airllm AirLLM 2.8T LLM inference with single 4GB GPU Jupyter Notebook Jupyter Notebook · 41,6 k au total En bonne santé
    Gagnées en 7 j +829 Score 4,1 Total 41,6 k
  42. 42
    martina9516/uncensored-ai-models Uncensored AI models from RouterHub — the website for one-key API access to Claude, GPT, Gemini, Grok, DeepSeek, GLM and Qwen, with uncensored and abliterated variants where available. OpenAI- and Anthropic-compatible. Pay per token. 108 au total Nouveau
    Gagnées en 7 j +41 Score 4,1 Total 108
  43. 43
    MemTensor/MemT-1.0 Official repository for MemT-1.0 Python Python · 501 au total
    Gagnées en 7 j +83 Score 4,1 Total 501
  44. 44
    carloslfu/slotstream Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex. Swift Swift · 642 au total En bonne santé
    Gagnées en 7 j +93 Score 4,0 Total 642
  45. 45
    chalk-ai/chalk-router-oss chalk-router: an extremely fast & focused LLM router implementation. Rust Rust · 39 au total Nouveau
    Gagnées en 7 j +39 Score 3,9 Total 39
  46. 46
    magnitudedev/magnitude Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU. Rust Rust · 7,7 k au total En bonne santé
    Gagnées en 7 j +333 Score 3,9 Total 7,7 k
  47. 47
    eagle-hq/EAGLE-4 Official Inference Code of EAGLE-4 Python Python · 1,1 k au total
    Gagnées en 7 j +112 Score 3,5 Total 1,1 k
  48. 48
    WizisCool/oh-my-cpa Self-hosted control plane & usage observability console for CLIProxyAPI (CPA) — AI providers, client keys, streaming telemetry, and model cost snapshots. TypeScript TypeScript · 297 au total
    Gagnées en 7 j +49 Score 3,1 Total 297
  49. 49
    LAGcomcom/zen-gate 把 OpenCode Zen 免费模型装进你所有的 AI Agent —— Windows 托盘网关，OpenAI/Anthropic 兼容，自动适配 10+ Agent，限流自动切换 Go Go · 284 au total
    Gagnées en 7 j +47 Score 3,1 Total 284
  50. 50
    christopherowen/spark-ds41f Reproducible Docker/vLLM deployment, tuning, and benchmarks for DeepSeek V4.1 Flash on a switchless three-node or four-node DGX Spark fabric. Python Python · 78 au total
    Gagnées en 7 j +30 Score 3,0 Total 78
  51. 51
    3145544645/gouji 构屎 GOUJI — 纯 C++/CUDA 的三值 mini LLM。2 bit/权重，零 Python；联想记忆工具路由，零训练参数；143 个缺陷与 29 个变异测试记录在案。 C++ C++ · 99 au total
    Gagnées en 7 j +30 Score 3,0 Total 99
  52. 52
    nsbradley88/talus-public Rust LLM inference engine tuned for AMD Strix Halo (gfx1151): a fork of Strata with ggml/llama.cpp-derived HIP kernels. Five models on one 128 GB machine. Rust Rust · 30 au total Nouveau
    Gagnées en 7 j +30 Score 3,0 Total 30
  53. 53
    agentgateway/agentgateway Next Generation Agentic Proxy for AI Agents and MCP servers Rust Rust · 5,9 k au total En bonne santé
    Gagnées en 7 j +222 Score 3,0 Total 5,9 k
  54. 54
    bpx03/QEngine Source-available C++/CUDA inference engine for Qwen3.8-27B INT4 on 3x RTX 3090 Cuda Cuda · 36 au total
    Gagnées en 7 j +29 Score 2,9 Total 36
  55. 55
    NVIDIA/spark-local-inference Rust Rust · 743 au total
    Gagnées en 7 j +73 Score 2,8 Total 743
  56. 56
    bojieli/ai-infra-book 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验 Python Python · 7,2 k au total Signaux mitigés
    Gagnées en 7 j +235 Score 2,8 Total 7,2 k
  57. 57
    meshapplied/goethe Serve your models fast on your own infra in 3 steps. Rust Rust · 43 au total
    Gagnées en 7 j +28 Score 2,8 Total 43
  58. 58
    Agentic-System-Lab/agent-loop-visualizer This project is for learning: it shows how an agent loop works underneath, in a way more intuitive and visual than reading about it. Behind an agent's chat box runs a loop: step after step, the model proposes an action and the harness carries it out, until it stops. Python Python · 51 au total
    Gagnées en 7 j +28 Score 2,8 Total 51
  59. 59
    0xShug0/audio.cpp An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency. C++ C++ · 3,9 k au total En bonne santé
    Gagnées en 7 j +169 Score 2,8 Total 3,9 k
  60. 60
    mozilla-ai/otari Open-source, OpenAI-compatible LLM gateway you run yourself. One endpoint for 40+ providers, with virtual keys, budgets, and usage tracking. Python Python · 812 au total En bonne santé
    Gagnées en 7 j +74 Score 2,7 Total 812
  61. 61
    ashhart/TensorFold LLM Inference Engine for Metal, CUDA and Vulkan. Zig Zig · 1,6 k au total En bonne santé
    Gagnées en 7 j +103 Score 2,7 Total 1,6 k
  62. 62
    yreidev/workbuddy2api-panel 把腾讯 WorkBuddy 账号变成 OpenAI 兼容 API 的多账号网关。请求头照官方桌面端逐项对齐，主备入口 + HTTPDNS 自动切换，国内 / 国际双域合池，企业号可用 Claude，签到与成长任务全自动，React 管理面板支持一键更新。原版 workbuddy2api-panel 的持续同步增强版，Docker / 单文件部署。 Go Go · 62 au total
    Gagnées en 7 j +26 Score 2,6 Total 62
  63. 63
    afdEP/afdEP-comm Expert-parallel communication for Attention–FFN Disaggregation (AFD), built on DeepEP V2 Python Python · 30 au total
    Gagnées en 7 j +26 Score 2,6 Total 30
  64. 64
    peonist-ai/halogen-flash-server Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port. Shell Shell · 1,2 k au total Signaux mitigés
    Gagnées en 7 j +86 Score 2,6 Total 1,2 k
  65. 65
    zexadev/gemini-web2api-go Gemini web → OpenAI-compatible + native Gemini API, self-hosted, no API key. Reverse-proxies gemini.google.com into /v1/chat/completions + /v1beta generateContent: single Go binary, Chrome TLS fingerprint, cookie pool, auto-renewal, proxy pool, admin dashboard, SQLite/MySQL/PG, image & video… Go Go · 763 au total En bonne santé
    Gagnées en 7 j +65 Score 2,5 Total 763
  66. 66
    decolua/9router Unlimited FREE AI coding. Connect Claude Code, Codex, Cursor, Cline, Copilot, Antigravity to FREE Claude/GPT/Gemini via 40+ providers. Auto-fallback, RTK -40% tokens, never hit limits. JavaScript JavaScript · 31,8 k au total En bonne santé
    Gagnées en 7 j +427 Score 2,4 Total 31,8 k
  67. 67
    recursive-oss/rec Our lightning-fast inference engine. Python Python · 261 au total
    Gagnées en 7 j +35 Score 2,3 Total 261
  68. 68
    Leonccaa/ReVolta Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/disk KV offload, and MTP or DFlash2 speculative decoding. Python Python · 36 au total
    Gagnées en 7 j +23 Score 2,3 Total 36
  69. 69
    amitshekhariitbhu/ai-engineering-course AI Engineering Course - A free and complete AI Engineering Course to learn AI Engineering step by step - from Machine Learning, Neural Networks, and Transformers to LLMs, Fine-Tuning, RAG, AI Agents, LLM Inference, Evaluation, AI Safety, and AI System Design. Markdown Markdown · 624 au total
    Gagnées en 7 j +54 Score 2,3 Total 624
  70. 70
    QuantumNous/new-api Open-source AI gateway. Self-hosted LLM gateway with OpenAI-, Claude- and Gemini-compatible APIs, model routing, access control, usage tracking and cost accounting. Go Go · 51,2 k au total En bonne santé
    Gagnées en 7 j +500 Score 2,2 Total 51,2 k
  71. 71
    ggml-org/llama.cpp LLM inference in C/C++ C++ C++ · 133 k au total En bonne santé
    Gagnées en 7 j +770 Score 2,1 Total 133 k
  72. 72
    w8123/EnterpriseAgentFramework ReachAI企业级智能体开发平台：快速、安全完成已有业务系统智能化改造，让 AI 在 OA、ERP、CRM 等原系统中查数据、填表单、办业务。ReachAI: Quickly and securely bring AI to existing enterprise systems, enabling AI to query data, fill out forms, and execute business tasks directly within OA, ERP, CRM, and other business applications. Java Java · 1 k au total En bonne santé
    Gagnées en 7 j +64 Score 2,1 Total 1 k
  73. 73
    CSUNX233/qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B：NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。 C++ C++ · 82 au total
    Gagnées en 7 j +20 Score 2,0 Total 82
  74. 74
    pallavi-shekhar/ai-engineering-interview-questions-company-wise Your Cheat Sheet For AI Engineering Interviews at Top AI Companies - Questions and Answers. Markdown Markdown · 2 k au total
    Gagnées en 7 j +86 Score 2,0 Total 2 k
  75. 75
    BerriAI/litellm The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] Python Python · 62,5 k au total En bonne santé
    Gagnées en 7 j +486 Score 2,0 Total 62,5 k
  76. 76
    Orchestra-Research/AI-Research-SKILLs Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research. TeX TeX · 14,2 k au total Peu d’activité
    Gagnées en 7 j +230 Score 1,9 Total 14,2 k
  77. 77
    lidge-jun/opencodex Universal provider proxy for OpenAI Codex & Claude Code — use any LLM (Claude, Gemini, Grok, DeepSeek, Ollama…) with Codex CLI, App, SDK, and Claude Code TypeScript TypeScript · 18,2 k au total En bonne santé
    Gagnées en 7 j +258 Score 1,9 Total 18,2 k
  78. 78
    JakeATX/llamAmpere llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx C++ C++ · 214 au total
    Gagnées en 7 j +26 Score 1,9 Total 214
  79. 79
    maximhq/bifrost Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of… Go Go · 9,3 k au total En bonne santé
    Gagnées en 7 j +177 Score 1,9 Total 9,3 k
  80. 80
    vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs Python Python · 96,1 k au total En bonne santé
    Gagnées en 7 j +555 Score 1,8 Total 96,1 k
  81. 81
    nokia-applied-research/AnyJev Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request) Python Python · 1,6 k au total
    Gagnées en 7 j +68 Score 1,8 Total 1,6 k
  82. 82
    verl-project/verl-omni Multimodal RL training framework for diffusion & omni models Python Python · 1,4 k au total En bonne santé
    Gagnées en 7 j +62 Score 1,7 Total 1,4 k
  83. 83
    justxor/deepseek-local-guide-ru 🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, агенты, fine-tuning, лучшие фишки Shell Shell · 169 au total
    Gagnées en 7 j +20 Score 1,6 Total 169
  84. 84
    datawhalechina/zero-to-sglang Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese. Python Python · 1,7 k au total En bonne santé
    Gagnées en 7 j +62 Score 1,5 Total 1,7 k
  85. 85
    StartLuxLabs/StartLux-Decision Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX. Python Python · 1,6 k au total Repéré à 644 étoiles
    Gagnées en 7 j +61 Score 1,5 Total 1,6 k
  86. 86
    panxunying/ai-coding-welfare AI Coding 福利站导航：免费白嫖 Claude Code / Codex 的中转站与公益站合集，含额度、模型、价格自动更新与一键配置脚本 JavaScript JavaScript · 1,2 k au total En bonne santé
    Gagnées en 7 j +52 Score 1,5 Total 1,2 k
  87. 87
    AtomicBot-ai/Atomic-Chat Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V TypeScript TypeScript · 1,9 k au total En bonne santé
    Gagnées en 7 j +65 Score 1,5 Total 1,9 k
  88. 88
    jundot/omlx LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar Python Python · 23,3 k au total En bonne santé
    Gagnées en 7 j +229 Score 1,5 Total 23,3 k
  89. 89
    sergqwer/strata-nvfp4 Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8… C++ C++ · 215 au total
    Gagnées en 7 j +21 Score 1,5 Total 215
  90. 90
    gitleaks/gitleaks Find secrets with Gitleaks 🔑 Go Go · 30,6 k au total Peu d’activité
    Gagnées en 7 j +250 Score 1,4 Total 30,6 k
  91. 91
    vllm-project/vllm-omni A framework for efficient model inference with omni-modality models Python Python · 7,5 k au total En bonne santé
    Gagnées en 7 j +121 Score 1,4 Total 7,5 k
  92. 92
    vllm-project/semantic-router An open, programmable decision layer for models and compute. Go Go · 6,4 k au total En bonne santé
    Gagnées en 7 j +106 Score 1,3 Total 6,4 k
  93. 93
    tetherto/qvac Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS. TypeScript TypeScript · 744 au total En bonne santé
    Gagnées en 7 j +35 Score 1,3 Total 744
  94. 94
    1CatAI/1Cat-vLLM V100 / SM70-focused vLLM engineering fork for modern LLM inference. Python Python · 1,4 k au total En bonne santé
    Gagnées en 7 j +49 Score 1,3 Total 1,4 k
  95. 95
    higress-group/HiRoute Make Codex and Claude Code subscriptions go further: a local model router that gives routine agent work to economical models and keeps your strongest model for the hard parts. Rust Rust · 371 au total
    Gagnées en 7 j +23 Score 1,2 Total 371
  96. 96
    lbjlaq/Antigravity-Manager Professional Antigravity Account Manager & Switcher. One-click seamless account switching for Antigravity Tools. Built with Tauri v2 + React (Rust).专业的 Antigravity 账号管理与切换工具。为 Antigravity 提供一键无缝账号切换功能。 Rust Rust · 32,8 k au total En bonne santé
    Gagnées en 7 j +219 Score 1,2 Total 32,8 k
  97. 97
    sgl-project/sglang-omni High Performance Serving Runtime For Interactive Super Intelligence System Python Python · 1,7 k au total En bonne santé
    Gagnées en 7 j +46 Score 1,1 Total 1,7 k
  98. 98
    warpfront/hipfire RDNA-native LLM inference engine in Rust. Rust Rust · 786 au total En bonne santé
    Gagnées en 7 j +31 Score 1,1 Total 786
  99. 99
    zhongkaifu/TensorSharp A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability C# C# · 651 au total En bonne santé
    Gagnées en 7 j +26 Score 1,0 Total 651
  100. 100
    ollaya-dev/ollaya Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app… Rust Rust · 1,4 k au total
    Gagnées en 7 j +37 Score 1,0 Total 1,4 k
  101. 101
    lightseekorg/tokenspeed TokenSpeed is a speed-of-light LLM inference engine. Python Python · 2,8 k au total En bonne santé
    Gagnées en 7 j +52 Score 1,0 Total 2,8 k
  102. 102
    FareedKhan-dev/kimi-k3-in-c A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU. C C · 9,3 k au total En bonne santé
    Gagnées en 7 j +94 Score 1,0 Total 9,3 k
  103. 103
    mostlygeek/llama-swap Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it. Go Go · 6,2 k au total En bonne santé
    Gagnées en 7 j +76 Score 1,0 Total 6,2 k
  104. 104
    flyteorg/flyte Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts) Go Go · 7,8 k au total En bonne santé
    Gagnées en 7 j +82 Score 0,9 Total 7,8 k
  105. 105
    ARahim3/kaggle-tpu-lab Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi. Python Python · 757 au total Peu d’activité
    Gagnées en 7 j +25 Score 0,9 Total 757
  106. 106
    feder-cr/jev jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU. C++ C++ · 948 au total Signaux mitigés
    Gagnées en 7 j +28 Score 0,9 Total 948
  107. 107
    raullenchai/Rapid-MLX Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights. Python Python · 4,1 k au total En bonne santé
    Gagnées en 7 j +56 Score 0,9 Total 4,1 k
  108. 108
    StayLameBro/backburner Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable Python Python · 1,2 k au total
    Gagnées en 7 j +29 Score 0,9 Total 1,2 k
  109. 109
    XHToken/Spark-X2.6 Spark-x2.6 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models 877 au total Signaux mitigés
    Gagnées en 7 j +25 Score 0,9 Total 877
  110. 110
    kvcache-ai/Mooncake Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. C++ C++ · 7 k au total En bonne santé
    Gagnées en 7 j +70 Score 0,8 Total 7 k
  111. 111
    NightMean/OlliteRT Turn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source Kotlin Kotlin · 599 au total Signaux mitigés
    Gagnées en 7 j +20 Score 0,8 Total 599
  112. 112
    LMCache/LMCache LMCache: Supercharge Your LLM with the Fastest KV Cache Layer Python Python · 12,3 k au total En bonne santé
    Gagnées en 7 j +91 Score 0,8 Total 12,3 k
  113. 113
    seakee/CPA-Manager-Plus A self-hosted CPA / CLIProxyAPI management panel and AI gateway observability dashboard for requests, usage, cost, quota, failures, and account health. Go Go · 4 k au total En bonne santé
    Gagnées en 7 j +51 Score 0,8 Total 4 k
  114. 114
    antirez/ds4 DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm C C · 24,2 k au total Signaux mitigés
    Gagnées en 7 j +122 Score 0,8 Total 24,2 k
  115. 115
    SemiAnalysisAI/InferenceX Open Source AI Accelerator Research Platform Standard / 开源推理研究平台 Python Python · 2 k au total En bonne santé
    Gagnées en 7 j +34 Score 0,8 Total 2 k
  116. 116
    drumih/turbo-fieldfare Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook Swift Swift · 7,1 k au total En bonne santé
    Gagnées en 7 j +64 Score 0,8 Total 7,1 k
  117. 117
    vllm-project/vllm-metal Community maintained hardware plugin for vLLM on Apple Silicon Python Python · 1,9 k au total En bonne santé
    Gagnées en 7 j +33 Score 0,8 Total 1,9 k
  118. 118
    gpustack/gpustack A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances. Python Python · 5,9 k au total En bonne santé
    Gagnées en 7 j +57 Score 0,7 Total 5,9 k
  119. 119
    lemonade-sdk/lemonade Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk C++ C++ · 6 k au total En bonne santé
    Gagnées en 7 j +56 Score 0,7 Total 6 k
  120. 120
    antirez/h3.c MiniMax H3 inference engine for Mac computers C C · 3 k au total Peu d’activité
    Gagnées en 7 j +39 Score 0,7 Total 3 k
  121. 121
    future-agi/future-agi Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0. Python Python · 2,2 k au total En bonne santé
    Gagnées en 7 j +33 Score 0,7 Total 2,2 k
  122. 122
    amitshekhariitbhu/ai-engineering-interview-questions Your Cheat Sheet for AI Engineering Interview – Questions and Answers. Markdown Markdown · 3,6 k au total Signaux mitigés
    Gagnées en 7 j +40 Score 0,7 Total 3,6 k
  123. 123
    ENTERPILOT/GoModel AI gateway / AI control plane / AI proxy written in Go. Unified OpenAI-compatible and Anthropic-compatible API for OpenAI, Anthropic, Gemini, Groq, xAI, Ollama, vLLM and more. A LiteLLM alternative with observability, guardrails, streaming, cost tracking, intelligent routing, sticky sessions… Go Go · 1,3 k au total En bonne santé
    Gagnées en 7 j +23 Score 0,6 Total 1,3 k
  124. 124
    flashinfer-ai/flashinfer FlashInfer: Kernel Library for LLM Serving Cuda Cuda · 6,8 k au total En bonne santé
    Gagnées en 7 j +53 Score 0,6 Total 6,8 k
  125. 125
    youssofal/MTPLX The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server. Python Python · 2,7 k au total En bonne santé
    Gagnées en 7 j +33 Score 0,6 Total 2,7 k
  126. 126
    IBM/mcp-context-forge An AI Gateway, registry, and proxy that sits in front of any MCP, A2A, or REST/gRPC APIs, exposing a unified endpoint with centralized discovery, guardrails and management. Optimizes Agent & Tool calling, and supports plugins. Python Python · 4,7 k au total En bonne santé
    Gagnées en 7 j +42 Score 0,6 Total 4,7 k
  127. 127
    musistudio/claude-code-router One local control plane for every AI agent: route across models, fuse new capabilities, orchestrate tools, and stay fully in control. TypeScript TypeScript · 38 k au total En bonne santé
    Gagnées en 7 j +116 Score 0,6 Total 38 k
  128. 128
    casdoor/casdoor An open-source Agent-first Identity and Access Management (IAM) /LLM MCP & agent gateway and auth server with web UI supporting OpenClaw, MCP, OAuth, OIDC, SAML, CAS, LDAP, SCIM, WebAuthn, TOTP, MFA, Face ID, Google Workspace, Azure AD Go Go · 14,7 k au total En bonne santé
    Gagnées en 7 j +65 Score 0,5 Total 14,7 k
  129. 129
    exo-explore/exo Run frontier AI locally. Python Python · 48,2 k au total Signaux mitigés
    Gagnées en 7 j +114 Score 0,5 Total 48,2 k
  130. 130
    ai-dynamo/dynamo A Datacenter Scale Distributed Inference Serving Framework Rust Rust · 8,4 k au total En bonne santé
    Gagnées en 7 j +43 Score 0,5 Total 8,4 k
  131. 131
    fosrl/pangolin Modern networking and security platform providing secure access and connectivity to apps, infrastructure, and AI workloads. Connect and protect your users. TypeScript TypeScript · 23,3 k au total En bonne santé
    Gagnées en 7 j +69 Score 0,5 Total 23,3 k
  132. 132
    InsForge/InsForge The all-in-one, open-source backend platform for agentic coding. InsForge gives your coding agent database, auth, storage, compute, hosting, and AI gateway to ship full-stack apps end-to-end. TypeScript TypeScript · 13,3 k au total En bonne santé
    Gagnées en 7 j +52 Score 0,5 Total 13,3 k
  133. 133
    alibaba/MNN MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI. C++ C++ · 16,3 k au total En bonne santé
    Gagnées en 7 j +57 Score 0,4 Total 16,3 k
  134. 134
    vllm-project/vllm-ascend Community maintained hardware plugin for vLLM on Huawei Ascend Python Python · 3 k au total En bonne santé
    Gagnées en 7 j +23 Score 0,4 Total 3 k
  135. 135
    algorithmicsuperintelligence/openevolve Open-source implementation of AlphaEvolve Python Python · 7,7 k au total En bonne santé
    Gagnées en 7 j +36 Score 0,4 Total 7,7 k
  136. 136
    octelium/octelium A next-gen FOSS self-hosted unified zero trust secure access platform that can operate as a remote access VPN, a ZTNA platform, API/AI/MCP gateway, a PaaS, an ngrok-alternative and a homelab infrastructure. Go Go · 4,1 k au total En bonne santé
    Gagnées en 7 j +25 Score 0,4 Total 4,1 k
  137. 137
    NVIDIA/TensorRT-LLM TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference… Python Python · 15 k au total En bonne santé
    Gagnées en 7 j +47 Score 0,4 Total 15 k
  138. 138
    nextlevelbuilder/goclaw GoClaw - GoClaw is OpenClaw rebuilt in Go — with multi-tenant isolation, 5-layer security, and native concurrency. Deploy AI agent teams at scale without compromising on safety. Go Go · 3,7 k au total En bonne santé
    Gagnées en 7 j +23 Score 0,4 Total 3,7 k
  139. 139
    budtmo/docker-android Android in docker solution with noVNC supported, video recording, mcp server and AI-agent Python Python · 16,1 k au total En bonne santé
    Gagnées en 7 j +48 Score 0,4 Total 16,1 k
  140. 140
    openvinotoolkit/openvino OpenVINO™ is an open source toolkit for optimizing and deploying AI inference C++ C++ · 11,1 k au total En bonne santé
    Gagnées en 7 j +39 Score 0,4 Total 11,1 k
  141. 141
    deepseek-ai/FlashMLA FlashMLA: Efficient Multi-head Latent Attention Kernels C++ C++ · 13,2 k au total Peu d’activité
    Gagnées en 7 j +42 Score 0,4 Total 13,2 k
  142. 142
    looplj/axonhub ⚡️ Open-source AI Gateway — Use any SDK to call 100+ LLMs. Built-in failover, load balancing, cost control & end-to-end tracing. Go Go · 5,4 k au total En bonne santé
    Gagnées en 7 j +25 Score 0,3 Total 5,4 k
  143. 143
    tbphp/gpt-load Self-hosted AI gateway for multi-channel, multi-credential setups — API keys and subscription accounts, scheduling, failover, request logs and usage. 自托管 AI 网关：多渠道多凭据统一接入，含密钥与订阅账号、调度容错、日志与用量。 Go Go · 7,2 k au total En bonne santé
    Gagnées en 7 j +28 Score 0,3 Total 7,2 k
  144. 144
    skypilot-org/skypilot The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster. Python Python · 10,8 k au total En bonne santé
    Gagnées en 7 j +33 Score 0,3 Total 10,8 k
  145. 145
    weave-os/router Model router for agentic systems. Routes every prompt to the right model in <50ms. Cut costs 40-70% with just an endpoint change. Go Go · 5,6 k au total En bonne santé
    Gagnées en 7 j +23 Score 0,3 Total 5,6 k
  146. 146
    mlc-ai/web-llm High-performance In-browser LLM Inference Engine TypeScript TypeScript · 19,3 k au total Signaux mitigés
    Gagnées en 7 j +41 Score 0,3 Total 19,3 k
  147. 147
    kserve/kserve Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes Go Go · 6,2 k au total En bonne santé
    Gagnées en 7 j +22 Score 0,3 Total 6,2 k
  148. 148
    apache/apisix The Cloud-Native API Gateway and AI Gateway Lua Lua · 17,3 k au total En bonne santé
    Gagnées en 7 j +36 Score 0,3 Total 17,3 k
  149. 149
    Portkey-AI/gateway A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API. TypeScript TypeScript · 13,3 k au total Peu d’activité
    Gagnées en 7 j +29 Score 0,3 Total 13,3 k
  150. 150
    kvcache-ai/ktransformers A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations Python Python · 19,7 k au total En bonne santé
    Gagnées en 7 j +24 Score 0,2 Total 19,7 k

1 dépôt à la croissance inhabituelle n’est pas affiché. Afficher le classement complet

Émergents en Inférence & passerelles LLM Semaines passées : 26 oct.19 oct.