Upstar classe toutes les heures les projets GitHub qui montent le plus vite, dans 69 thèmes. Voir les thèmes

#llm-serving · IA & ML

Inférence & passerelles LLM

Moteurs d’inférence, serveurs de modèles et passerelles qui répartissent les requêtes entre les API de modèles.

Ce mois-ci (4 oct. – 2 nov.), en tête : DeepInfer (+25 007), Strata (+29 490) et project-maya (+2 400). Les 333 projets classés ont gagné 153 333 étoiles.

Mis à jour le 2 nov., 17:37 UTC · classé toutes les heures à partir des étoiles quotidiennes de GitHub

Mis à jour il y a 26 min

Sujet GitHub ↗ (nouvel onglet)

Classement du mois

130 sur 335 affichés

Comment Upstar classe

Classement complet, avec les dépôts masqués par défaut. Revenir au classement

  1. 1
    deepseek-ai/DeepInfer DeepInfer: High-performance, large-scale inference framework for DeepSeek models Python Python · 25 k au total
    Gagnées en 30 j +25 k Score 2 500,7 Total 25 k
  2. 2
    Niko1221/Strata Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input. C++ C++ · 38,3 k au total
    Gagnées en 30 j +29,5 k Score 314,8 Total 38,3 k
  3. 3
    mw00/project-maya Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools. Cuda Cuda · 2,4 k au total Repéré à 762 étoiles
    Gagnées en 30 j +2,4 k Score 240,0 Total 2,4 k
  4. 4
    StartLuxLabs/StartLux-Decision Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX. Python Python · 1,6 k au total Repéré à 644 étoiles
    Gagnées en 30 j +1,5 k Score 118,1 Total 1,6 k
  5. 5
    eagle-hq/EAGLE-4 Official Inference Code of EAGLE-4 Python Python · 1,1 k au total
    Gagnées en 30 j +1,1 k Score 111,5 Total 1,1 k
  6. 6
    VecSzn/PolyStrata Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost. C++ C++ · 763 au total Repéré à 196 étoiles
    Gagnées en 30 j +763 Score 76,3 Total 763
  7. 7
    NVIDIA/spark-local-inference Rust Rust · 750 au total
    Gagnées en 30 j +750 Score 75,0 Total 750
  8. 8
    Alice-Marx/dsh-model-router Model Router and GALGame plugin for DeepSeek Harness JavaScript JavaScript · 637 au total En bonne santé
    Gagnées en 30 j +637 Score 63,7 Total 637
  9. 9
    eliasbiondo/antirion 🕷️ The fastest open-source AI Gateway and LLM Gateway — one binary with built-in console to route, secure, budget and observe every LLM call across OpenAI, Anthropic, Gemini, Bedrock and more. Rust Rust · 629 au total Repéré à 95 étoiles
    Gagnées en 30 j +629 Score 62,9 Total 629
  10. 10
    nace-ai/kapitsa Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai C++ C++ · 598 au total
    Gagnées en 30 j +598 Score 59,8 Total 598
  11. 11
    StayLameBro/backburner Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable Python Python · 1,2 k au total
    Gagnées en 30 j +905 Score 57,6 Total 1,2 k
  12. 12
    prabanta-dev/janas 37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU. C C · 578 au total
    Gagnées en 30 j +549 Score 54,9 Total 578
  13. 13
    routerhubSI/uncensored-ai-api Uncensored AI API and models — one key for Claude, GPT, Gemini, Grok, DeepSeek, GLM and Qwen, with uncensored and abliterated variants where available. OpenAI- and Anthropic-compatible. Pay per token. 508 au total
    Gagnées en 30 j +508 Score 50,8 Total 508
  14. 14
    MemTensor/MemT-1.0 Official repository for MemT-1.0 Python Python · 502 au total
    Gagnées en 30 j +502 Score 50,2 Total 502
  15. 15
    respanai/respan-gateway Open-source LLM gateway in Rust. One endpoint for OpenAI, Anthropic, Gemini and every other vendor. Any SDK, any model, any tool. Rust Rust · 489 au total
    Gagnées en 30 j +489 Score 48,9 Total 489
  16. 16
    ThinkWatchProject/ThinkWatch-Lite Local gateway for Claude Code, Codex and other AI clients on macOS, Windows and Linux: switch upstreams without touching clients, keep API keys from relays, cut off malicious tool calls, trace every request. TypeScript TypeScript · 534 au total
    Gagnées en 30 j +478 Score 47,8 Total 534
  17. 17
    jesdga95/basalt Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell C++ C++ · 461 au total
    Gagnées en 30 j +461 Score 46,1 Total 461
  18. 18
    securetron-gh/aifw AI Agent Identity and Access Management and Inference Firewall Python Python · 454 au total Peu d’activité
    Gagnées en 30 j +452 Score 45,2 Total 454
  19. 19
    Generated-Computers/Locals Run many AI agents at once on your Mac, up to 3x faster. Python Python · 448 au total
    Gagnées en 30 j +448 Score 44,8 Total 448
  20. 20
    shuishuipingan/qoder2api-hub Qoder国际国内多账号反代网关，Qoder 国际国内双区多账号 OpenAI 兼容网关中枢 — COSY 签名推理、稳定设备指纹防风控、OAuth 设备授权免客户端登录、每日签到与 Pro 福利、后台定时调度器、Web 监控看板，支持 Codex / Claude Code 与标准 OpenAI 客户端 Python Python · 501 au total
    Gagnées en 30 j +447 Score 44,7 Total 501
  21. 21
    Indras-Mirror/Xolotl Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up to ~27 tok/s, 512K context, October 1017 builds included. Plus GLM-5.3-Flash and Qwen3.8. MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic… C++ C++ · 442 au total
    Gagnées en 30 j +442 Score 44,2 Total 442
  22. 22
    cheaper-inference/OmniRoute Never stop coding. Free MIT AI gateway: one endpoint, 375 providers (150+ free), 1,400+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A… TypeScript TypeScript · 83,9 k au total En bonne santé
    Gagnées en 30 j +11,7 k Score 43,5 Total 83,9 k
  23. 23
    ekzhang/resin 🧱 Experimental library for building inference engines Python Python · 369 au total En bonne santé
    Gagnées en 30 j +369 Score 36,9 Total 369
  24. 24
    Calcium-Ion/AstrLink A local AI gateway for AI agents, unifying your subscriptions and API providers with smart routing and on-device privacy protection. Go Go · 524 au total
    Gagnées en 30 j +396 Score 35,0 Total 524
  25. 25
    higress-group/HiRoute Make Codex and Claude Code subscriptions go further: a local model router that gives routine agent work to economical models and keeps your strongest model for the hard parts. Rust Rust · 371 au total
    Gagnées en 30 j +348 Score 34,8 Total 371
  26. 26
    lyogavin/airllm AirLLM 2.8T LLM inference with single 4GB GPU Jupyter Notebook Jupyter Notebook · 41,6 k au total En bonne santé
    Gagnées en 30 j +6,4 k Score 34,1 Total 41,6 k
  27. 27
    trymirai/uzu A high-performance inference engine for AI models Rust Rust · 3,2 k au total En bonne santé
    Gagnées en 30 j +1,4 k Score 32,1 Total 3,2 k
  28. 28
    mnfst/awesome-free-llm-apis List of Permanent Free LLM API (API Keys) JavaScript JavaScript · 12 k au total Signaux mitigés
    Gagnées en 30 j +2,9 k Score 30,3 Total 12 k
  29. 29
    xiaodou-official/xiaodou-open-platform 小豆支付开放平台：个人也能申请的支付宝+微信聚合收款接口——RSA2 签名 + webhook 验签、持牌机构清算、次日到账；兼容易支付 / 彩虹支付类插件协议零代码迁移（submit.php / mapi.php / api.php，网关 / 商家 ID / 密钥三格配置）；已兼容 Sub2API、New API（源码级核对）；支持多到账账号收款分配与分账；含 14 篇文档、四语言签名示例与本地 demo。Payment API for individuals & digital-goods merchants: Alipay + WeChat Pay, Epay-compatible… 327 au total
    Gagnées en 30 j +293 Score 29,3 Total 327
  30. 30
    WizisCool/oh-my-cpa Self-hosted control plane & usage observability console for CLIProxyAPI (CPA) — AI providers, client keys, streaming telemetry, and model cost snapshots. TypeScript TypeScript · 297 au total
    Gagnées en 30 j +287 Score 28,7 Total 297
  31. 31
    LAGcomcom/zen-gate 把 OpenCode Zen 免费模型装进你所有的 AI Agent —— Windows 托盘网关，OpenAI/Anthropic 兼容，自动适配 10+ Agent，限流自动切换 Go Go · 284 au total
    Gagnées en 30 j +284 Score 28,4 Total 284
  32. 32
    T-Crypt/berth Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off. Go Go · 282 au total
    Gagnées en 30 j +282 Score 28,2 Total 282
  33. 33
    recursive-oss/rec Our lightning-fast inference engine. Python Python · 261 au total
    Gagnées en 30 j +261 Score 26,1 Total 261
  34. 34
    incoai/splash A local inference engine for Apple silicon, built around the model. C++ C++ · 2 k au total
    Gagnées en 30 j +872 Score 25,7 Total 2 k
  35. 35
    StartLuxLabs/StartLux-Prism Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers. Python Python · 241 au total
    Gagnées en 30 j +240 Score 24,0 Total 241
  36. 36
    Wallawalla47/Infernix C++/CUDA single-GPU inference engine for Qwen3.8 (incl. Qwen3.8-Flash-Next with offloaded experts) on the RTX 5090; grew from NInfer C++ C++ · 234 au total
    Gagnées en 30 j +234 Score 23,4 Total 234
  37. 37
    7API/uncensored-ai-api Uncensored AI API and models — one key for Claude, GPT, Gemini, Grok, DeepSeek, GLM and Kimi, with uncensored and abliterated variants where available. OpenAI- and Anthropic-compatible. Pay per token. 215 au total
    Gagnées en 30 j +215 Score 21,5 Total 215
  38. 38
    ashhart/TensorFold LLM Inference Engine for Metal, CUDA and Vulkan. Zig Zig · 1,6 k au total En bonne santé
    Gagnées en 30 j +641 Score 20,7 Total 1,6 k
  39. 39
    sergqwer/strata-nvfp4 Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8… C++ C++ · 215 au total
    Gagnées en 30 j +203 Score 20,3 Total 215
  40. 40
    NerverDevs/nervecenter new-api / sub2api 中转站账号管理中心：余额、API Key、用量、倍率、模型比价 | https://nvcenter.cc TypeScript TypeScript · 199 au total
    Gagnées en 30 j +199 Score 19,9 Total 199
  41. 41
    bojieli/ai-infra-book 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验 Python Python · 7,2 k au total Signaux mitigés
    Gagnées en 30 j +1,4 k Score 18,8 Total 7,2 k
  42. 42
    magnitudedev/magnitude Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU. Rust Rust · 7,7 k au total En bonne santé
    Gagnées en 30 j +1,4 k Score 17,3 Total 7,7 k
  43. 43
    nokia-applied-research/AnyJev Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request) Python Python · 1,6 k au total
    Gagnées en 30 j +543 Score 17,0 Total 1,6 k
  44. 44
    modelsphere/modelsphere ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized. Shell Shell · 171 au total
    Gagnées en 30 j +168 Score 16,8 Total 171
  45. 45
    justxor/deepseek-local-guide-ru 🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, агенты, fine-tuning, лучшие фишки Shell Shell · 169 au total
    Gagnées en 30 j +168 Score 16,8 Total 169
  46. 46
    nicorobinreal/llmspeed Feel how fast a local LLM runs before you buy the hardware TypeScript TypeScript · 167 au total Peu d’activité
    Gagnées en 30 j +160 Score 16,0 Total 167
  47. 47
    bitcloud-ai/Bittune Local-first engineering agent that deploys, benchmarks and tunes vLLM/SGLang inference services on your own GPUs, with auditable run evidence. TypeScript TypeScript · 163 au total Signaux mitigés
    Gagnées en 30 j +158 Score 15,8 Total 163
  48. 48
    FlashML-org/FreeToken FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently. Python Python · 15,9 k au total En bonne santé
    Gagnées en 30 j +1,9 k Score 15,7 Total 15,9 k
  49. 49
    syv-ai/HyperQwen Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks. Python Python · 2,5 k au total En bonne santé
    Gagnées en 30 j +664 Score 15,6 Total 2,5 k
  50. 50
    ddalcu/mlx-serve Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation. Zig Zig · 2,3 k au total En bonne santé
    Gagnées en 30 j +601 Score 14,5 Total 2,3 k
  51. 51
    lightseekorg/tokenspeed TokenSpeed is a speed-of-light LLM inference engine. Python Python · 2,8 k au total En bonne santé
    Gagnées en 30 j +662 Score 14,2 Total 2,8 k
  52. 52
    mozilla-ai/otari Open-source, OpenAI-compatible LLM gateway you run yourself. One endpoint for 40+ providers, with virtual keys, budgets, and usage tracking. Python Python · 815 au total En bonne santé
    Gagnées en 30 j +313 Score 14,0 Total 815
  53. 53
    rednote-machine-learning/RedKnot Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention Python Python · 3,1 k au total Signaux mitigés
    Gagnées en 30 j +667 Score 13,6 Total 3,1 k
  54. 54
    zexadev/gemini-web2api-go Gemini web → OpenAI-compatible + native Gemini API, self-hosted, no API key. Reverse-proxies gemini.google.com into /v1/chat/completions + /v1beta generateContent: single Go binary, Chrome TLS fingerprint, cookie pool, auto-renewal, proxy pool, admin dashboard, SQLite/MySQL/PG, image & video… Go Go · 763 au total En bonne santé
    Gagnées en 30 j +293 Score 13,5 Total 763
  55. 55
    dayuwang401/spectral-orthogonal-exploration Official code for "Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration" (SOE, ACL 2026 Oral). A training-free, test-time scaling method that fixes reasoning collapse in LLM mathematical reasoning via orthogonal probing. Python Python · 135 au total
    Gagnées en 30 j +135 Score 13,5 Total 135
  56. 56
    ovg-project/kvcached Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond Python Python · 2 k au total En bonne santé
    Gagnées en 30 j +516 Score 13,3 Total 2 k
  57. 57
    peonist-ai/halogen-flash-server Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port. Shell Shell · 1,2 k au total Signaux mitigés
    Gagnées en 30 j +373 Score 13,0 Total 1,2 k
  58. 58
    amurgola/LumaBrowser Local AI desktop app: a Chromium browser with a GPU-aware llama.cpp runtime, OpenAI-compatible API, AI chat and agents, Stable Diffusion image/video, TTS and music, plus an MCP server so Claude Code, Codex and Cline can drive your browser. Windows, macOS, Linux. JavaScript JavaScript · 130 au total
    Gagnées en 30 j +130 Score 13,0 Total 130
  59. 59
    0xShug0/audio.cpp An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency. C++ C++ · 3,9 k au total En bonne santé
    Gagnées en 30 j +727 Score 12,8 Total 3,9 k
  60. 60
    panxunying/ai-coding-welfare AI Coding 福利站导航：免费白嫖 Claude Code / Codex 的中转站与公益站合集，含额度、模型、价格自动更新与一键配置脚本 JavaScript JavaScript · 1,2 k au total En bonne santé
    Gagnées en 30 j +364 Score 12,4 Total 1,2 k
  61. 61
    carloslfu/slotstream Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex. Swift Swift · 642 au total En bonne santé
    Gagnées en 30 j +236 Score 11,7 Total 642
  62. 62
    ahtx/kindi A custom LLM inference engine for low-cost GPUs: NVIDIA cards from a 1 GB Quadro K420 to an RTX 3080, the AMD Radeon 780M and the CPU. Runs the Bonsai 2B to 27B models and Nemotron with an OpenAI-compatible server. Python Python · 114 au total
    Gagnées en 30 j +114 Score 11,4 Total 114
  63. 63
    skyzh/tiny-llm learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen Python Python · 5,5 k au total En bonne santé
    Gagnées en 30 j +776 Score 11,3 Total 5,5 k
  64. 64
    lidge-jun/opencodex Universal provider proxy for OpenAI Codex & Claude Code — use any LLM (Claude, Gemini, Grok, DeepSeek, Ollama…) with Codex CLI, App, SDK, and Claude Code TypeScript TypeScript · 18,2 k au total En bonne santé
    Gagnées en 30 j +1,5 k Score 11,3 Total 18,2 k
  65. 65
    martina9516/uncensored-ai-models Uncensored AI models from RouterHub — the website for one-key API access to Claude, GPT, Gemini, Grok, DeepSeek, GLM and Qwen, with uncensored and abliterated variants where available. OpenAI- and Anthropic-compatible. Pay per token. 109 au total Nouveau
    Gagnées en 30 j +109 Score 10,9 Total 109
  66. 66
    XHToken/Spark-X2.6 Spark-x2.6 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models 877 au total Signaux mitigés
    Gagnées en 30 j +267 Score 10,8 Total 877
  67. 67
    BerriAI/litellm The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] Python Python · 62,6 k au total En bonne santé
    Gagnées en 30 j +2,6 k Score 10,8 Total 62,6 k
  68. 68
    groundingpi/GroundAnything A 4B diffusion grounding model for fast, precise visual evidence extraction. Unifies diverse grounding tasks through parallel decoding, with optional self-speculative decoding delivering 4.51x autoregressive throughput on COCO. Python Python · 109 au total
    Gagnées en 30 j +106 Score 10,6 Total 109
  69. Yzw202011/OmniSpace 本地优先的全栈 AI 创作工作站：AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块；vLLM / llama.cpp / Transformers 多引擎本地推理，显存智能调度、舱壁隔离自愈、仅供学习使用，欢迎交流指正。（最新进度（除插件系统）受到协议限制，暂时无法提交到仓库） Croissance inhabituelle, confiance moyenne Python Python · 531 au total Peu d’activité
    Gagnées en 30 j +194 Score 10,6 Total 531
  70. 69
    decolua/9router Unlimited FREE AI coding. Connect Claude Code, Codex, Cursor, Cline, Copilot, Antigravity to FREE Claude/GPT/Gemini via 40+ providers. Auto-fallback, RTK -40% tokens, never hit limits. JavaScript JavaScript · 31,8 k au total En bonne santé
    Gagnées en 30 j +1,8 k Score 10,4 Total 31,8 k
  71. 70
    vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs Python Python · 96,1 k au total En bonne santé
    Gagnées en 30 j +3,2 k Score 10,3 Total 96,1 k
  72. 71
    cablate/model-relay Per-window model switcher, subagent router, and automatic 429 failover for Claude Code — switch any window with /provider, delegate subagents across models, and keep Remote Control & statuslines working. JavaScript JavaScript · 103 au total
    Gagnées en 30 j +103 Score 10,3 Total 103
  73. 72
    apowerb/apowerb The open-source agentic framework to build, orchestrate, and operate production AI agents. Python Python · 149 au total En bonne santé
    Gagnées en 30 j +102 Score 10,2 Total 149
  74. 73
    agentgateway/agentgateway Next Generation Agentic Proxy for AI Agents and MCP servers Rust Rust · 5,9 k au total En bonne santé
    Gagnées en 30 j +724 Score 10,1 Total 5,9 k
  75. 74
    sgl-project/sglang-omni High Performance Serving Runtime For Interactive Super Intelligence System Python Python · 1,7 k au total En bonne santé
    Gagnées en 30 j +362 Score 10,0 Total 1,7 k
  76. 75
    3145544645/gouji 构屎 GOUJI — 纯 C++/CUDA 的三值 mini LLM。2 bit/权重，零 Python；联想记忆工具路由，零训练参数；143 个缺陷与 29 个变异测试记录在案。 C++ C++ · 99 au total
    Gagnées en 30 j +99 Score 9,9 Total 99
  77. 76
    QuantumNous/new-api Open-source AI gateway. Self-hosted LLM gateway with OpenAI-, Claude- and Gemini-compatible APIs, model routing, access control, usage tracking and cost accounting. Go Go · 51,2 k au total En bonne santé
    Gagnées en 30 j +2,1 k Score 9,7 Total 51,2 k
  78. 77
    pegainfer-project/kern A model-agnostic GPU runtime for shipping models as verified programs. Rust Rust · 148 au total En bonne santé
    Gagnées en 30 j +96 Score 9,6 Total 148
  79. 78
    drawthingsai/local-code-community Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built into the app. C C · 96 au total
    Gagnées en 30 j +96 Score 9,6 Total 96
  80. 79
    ggml-org/llama.cpp LLM inference in C/C++ C++ C++ · 133 k au total En bonne santé
    Gagnées en 30 j +3,4 k Score 9,6 Total 133 k
  81. 80
    pallavi-shekhar/ai-engineering-interview-questions-company-wise Your Cheat Sheet For AI Engineering Interviews at Top AI Companies - Questions and Answers. Markdown Markdown · 2 k au total
    Gagnées en 30 j +382 Score 9,5 Total 2 k
  82. 81
    shawnshekari/strixite LLM inference engine written from scratch in HIP for AMD Strix Halo (gfx1151) and Qwen3.8-Flash-Next - ~50 tokens/s out to 492k context C++ C++ · 93 au total
    Gagnées en 30 j +93 Score 9,3 Total 93
  83. 82
    ARahim3/kaggle-tpu-lab Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi. Python Python · 757 au total Peu d’activité
    Gagnées en 30 j +209 Score 8,9 Total 757
  84. 83
    Orchestra-Research/AI-Research-SKILLs Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research. TeX TeX · 14,2 k au total Peu d’activité
    Gagnées en 30 j +1 k Score 8,8 Total 14,2 k
  85. 84
    datawhalechina/zero-to-sglang Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese. Python Python · 1,7 k au total En bonne santé
    Gagnées en 30 j +322 Score 8,7 Total 1,7 k
  86. 85
    maximhq/bifrost Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of… Go Go · 9,3 k au total En bonne santé
    Gagnées en 30 j +777 Score 8,4 Total 9,3 k
  87. 86
    Sega757/Adaptive-AtomRAG Sub-millisecond query routing & cost-aware execution for GraphRAG architectures. Adaptive-AtomRAG is a highly optimized conditional routing gateway designed to resolve the latency and cost bottlenecks of heavy Atom-Entity Graph (AEG) RAG systems. Python Python · 85 au total
    Gagnées en 30 j +84 Score 8,4 Total 85
  88. 87
    ollaya-dev/ollaya Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app… Rust Rust · 1,4 k au total
    Gagnées en 30 j +282 Score 8,3 Total 1,4 k
  89. 88
    giienew/9router-pro 9Router Pro by Gienetic — Pro edition of 9Router with Prompt Injection (system prompt or chat script per global / API key / model), per-key token quotas, accent themes and an optimized dashboard. JavaScript JavaScript · 83 au total
    Gagnées en 30 j +83 Score 8,3 Total 83
  90. 89
    CSUNX233/qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B：NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。 C++ C++ · 82 au total
    Gagnées en 30 j +82 Score 8,2 Total 82
  91. 90
    verl-project/verl-omni Multimodal RL training framework for diffusion & omni models Python Python · 1,4 k au total En bonne santé
    Gagnées en 30 j +276 Score 8,1 Total 1,4 k
  92. 91
    feder-cr/jev jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU. C++ C++ · 948 au total Signaux mitigés
    Gagnées en 30 j +217 Score 8,0 Total 948
  93. 92
    kekzl/imp Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one… C++ C++ · 124 au total En bonne santé
    Gagnées en 30 j +80 Score 8,0 Total 124
  94. 93
    professorpalmer/bonsai-ada-surgery Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts. Python Python · 98 au total
    Gagnées en 30 j +78 Score 7,8 Total 98
  95. 94
    TheLastOxygen/mafdet You type. It already knows. Local LLMs that read and think while you type. Python Python · 78 au total
    Gagnées en 30 j +78 Score 7,8 Total 78
  96. 95
    w8123/EnterpriseAgentFramework ReachAI企业级智能体开发平台：快速、安全完成已有业务系统智能化改造，让 AI 在 OA、ERP、CRM 等原系统中查数据、填表单、办业务。ReachAI: Quickly and securely bring AI to existing enterprise systems, enabling AI to query data, fill out forms, and execute business tasks directly within OA, ERP, CRM, and other business applications. Java Java · 1 k au total En bonne santé
    Gagnées en 30 j +211 Score 7,4 Total 1 k
  97. 96
    christopherowen/spark-ds41f Reproducible Docker/vLLM deployment, tuning, and benchmarks for DeepSeek V4.1 Flash on a switchless three-node or four-node DGX Spark fabric. Python Python · 78 au total
    Gagnées en 30 j +73 Score 7,3 Total 78
  98. 97
    Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark DeepSeek-V4.1-Flash-1017 EXL3 2.9bpw on 2x NVIDIA DGX Spark (GB10) with TensorFold: 2.1-2.3x the vLLM kit's decode speed. Python Python · 73 au total
    Gagnées en 30 j +73 Score 7,3 Total 73
  99. 98
    qiniu/xhub A self-hosted AI gateway for enterprises. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] TypeScript TypeScript · 71 au total
    Gagnées en 30 j +71 Score 7,1 Total 71
  100. 99
    synchopate/nemotron-30b-cpu-256k Running a 30B MoE LLM on CPU at 30-50+ tok/s with a 256K Context Window via Hybrid Mamba-2 SSM, L3 NTA Slicing, AVX-512 VNNI Tile Repacking & 2MB HugePages Python Python · 71 au total Nouveau
    Gagnées en 30 j +71 Score 7,1 Total 71
  101. 100
    antirez/ds4 DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm C C · 24,2 k au total Signaux mitigés
    Gagnées en 30 j +1,1 k Score 7,0 Total 24,2 k
  102. 101
    1CatAI/1Cat-vLLM V100 / SM70-focused vLLM engineering fork for modern LLM inference. Python Python · 1,4 k au total En bonne santé
    Gagnées en 30 j +241 Score 6,9 Total 1,4 k
  103. 102
    SlyBase/vllm-sly-radiance Working on the fastest vllm path for AMD Radeon AI PRO R9700 Python Python · 102 au total
    Gagnées en 30 j +65 Score 6,5 Total 102
  104. 103
    OrbitInfra/OrbitKV High Performance KV Cache for LLM Inference Rust Rust · 320 au total En bonne santé
    Gagnées en 30 j +97 Score 6,5 Total 320
  105. 104
    AtomicBot-ai/Atomic-Chat Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V TypeScript TypeScript · 1,9 k au total En bonne santé
    Gagnées en 30 j +263 Score 6,5 Total 1,9 k
  106. 105
    JakeATX/llamAmpere llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx C++ C++ · 214 au total
    Gagnées en 30 j +75 Score 6,4 Total 214
  107. 106
    smg-project/smg Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant… Rust Rust · 699 au total En bonne santé
    Gagnées en 30 j +148 Score 6,3 Total 699
  108. 107
    0xSero/DeepSeek-V4.1-Flash-Two-Sparks DeepSeek-V4.1-Flash on two DGX Sparks: EXL3 routed experts, 262k context, 2M-token KV, vision, tools, MTP, bundled with Pi Shell Shell · 95 au total
    Gagnées en 30 j +62 Score 6,2 Total 95
  109. 108
    amitshekhariitbhu/ai-engineering-course AI Engineering Course - A free and complete AI Engineering Course to learn AI Engineering step by step - from Machine Learning, Neural Networks, and Transformers to LLMs, Fine-Tuning, RAG, AI Agents, LLM Inference, Evaluation, AI Safety, and AI System Design. Markdown Markdown · 624 au total
    Gagnées en 30 j +136 Score 6,2 Total 624
  110. 109
    yreidev/workbuddy2api-panel 把腾讯 WorkBuddy 账号变成 OpenAI 兼容 API 的多账号网关。请求头照官方桌面端逐项对齐，主备入口 + HTTPDNS 自动切换，国内 / 国际双域合池，企业号可用 Claude，签到与成长任务全自动，React 管理面板支持一键更新。原版 workbuddy2api-panel 的持续同步增强版，Docker / 单文件部署。 Go Go · 62 au total
    Gagnées en 30 j +61 Score 6,1 Total 62
  111. 110
    surya-koritala/Glyd Lossless AI compression: 33% less GPU memory, bit for bit. Rust Rust · 321 au total
    Gagnées en 30 j +92 Score 6,1 Total 321
  112. 111
    Nexilume-AI/nexus-cloud Nexus — execution infrastructure for AI agents. Host agents in the cloud or behind NAT; let users attach and authorize their own computers and Android phones. Deploy once, work across devices. Python SDK · MCP · Shared run context · Browser, terminal and files · Built-in chat UI · Model routing. Python Python · 59 au total
    Gagnées en 30 j +59 Score 5,9 Total 59
  113. 112
    01men/ybkkasi One control plane for every AI asset — MCP servers, skills, agents, AI apps. Self-hosted ledger with lifecycle state machine, 85 permission points, department × agent × skill cost attribution, and audit trail. MIT, zero runtime deps (Node 22 + node:sqlite). 企业 AI 资产治理。 JavaScript JavaScript · 59 au total
    Gagnées en 30 j +59 Score 5,9 Total 59
  114. 113
    lbjlaq/Antigravity-Manager Professional Antigravity Account Manager & Switcher. One-click seamless account switching for Antigravity Tools. Built with Tauri v2 + React (Rust).专业的 Antigravity 账号管理与切换工具。为 Antigravity 提供一键无缝账号切换功能。 Rust Rust · 32,8 k au total En bonne santé
    Gagnées en 30 j +1 k Score 5,8 Total 32,8 k
  115. 114
    amitshekhariitbhu/llm-inference-engineering Learn LLM Inference Engineering step by step - from KV cache, PagedAttention, and continuous batching to vLLM, SGLang, and GPUs. Markdown Markdown · 394 au total Signaux mitigés
    Gagnées en 30 j +100 Score 5,8 Total 394
  116. 115
    gitleaks/gitleaks Find secrets with Gitleaks 🔑 Go Go · 30,6 k au total Peu d’activité
    Gagnées en 30 j +996 Score 5,8 Total 30,6 k
  117. 116
    jundot/omlx LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar Python Python · 23,3 k au total En bonne santé
    Gagnées en 30 j +865 Score 5,8 Total 23,3 k
  118. 117
    amitshekhariitbhu/ai-engineering-interview-questions Your Cheat Sheet for AI Engineering Interview – Questions and Answers. Markdown Markdown · 3,6 k au total Signaux mitigés
    Gagnées en 30 j +328 Score 5,7 Total 3,6 k
  119. 118
    al-jshen/flashmap Flow Map Speculative Decoding Python Python · 56 au total
    Gagnées en 30 j +56 Score 5,6 Total 56
  120. 119
    zhongkaifu/TensorSharp A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability C# C# · 651 au total En bonne santé
    Gagnées en 30 j +128 Score 5,6 Total 651
  121. 120
    warpfront/hipfire RDNA-native LLM inference engine in Rust. Rust Rust · 786 au total En bonne santé
    Gagnées en 30 j +142 Score 5,6 Total 786
  122. 121
    vllm-project/vllm-omni A framework for efficient model inference with omni-modality models Python Python · 7,5 k au total En bonne santé
    Gagnées en 30 j +465 Score 5,5 Total 7,5 k
  123. 122
    maximpri/mlxtop A top for local LLMs: memory, paging, GPU and tokens/s for oMLX, Ollama, llama.cpp, vLLM and more Rust Rust · 143 au total
    Gagnées en 30 j +55 Score 5,5 Total 143
  124. 123
    PBJ-2/clauderipple Use GPT, DeepSeek, GLM, Gemini, Kimi and 400+ models in Claude Desktop's own model picker and in Claude Code, while staying signed in to your Claude subscription (1P, no gateway mode). Auto reasoning effort per prompt, subagents routed by difficulty, cache and limit warnings. Claude inside Codex… TypeScript TypeScript · 83 au total
    Gagnées en 30 j +55 Score 5,5 Total 83
  125. 124
    jpezzulli/sglang-rtxpro6000 Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL. Python Python · 188 au total En bonne santé
    Gagnées en 30 j +61 Score 5,4 Total 188
  126. 125
    apimartjgp76/llm-gateway-guide AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up. Python Python · 54 au total
    Gagnées en 30 j +54 Score 5,4 Total 54
  127. 126
    musicurgy/delta-inference-engine Delta Inference Engine: DeepSeek-V4.1-Flash on one NVIDIA GPU, natively on Windows (OpenAI-compatible server) C++ C++ · 53 au total
    Gagnées en 30 j +53 Score 5,3 Total 53
  128. 127
    juliankang4/spark-scope Read-only dashboard and rack panel for NVIDIA DGX Spark-class nodes, other NVIDIA GPU Linux machines and Apple Silicon Macs running vLLM, SGLang, TensorFold, llama.cpp, bloomery, NVIDIA Flare, Strata or oMLX JavaScript JavaScript · 70 au total
    Gagnées en 30 j +52 Score 5,2 Total 70
  129. 128
    apimartqfbi49xip/vllm-migration-guide AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up. Python Python · 52 au total
    Gagnées en 30 j +52 Score 5,2 Total 52
  130. 129
    Ianzhyh/workbuddy-to-dsh 把本机 WorkBuddy 桌面端已登录的模型（DeepSeek / GLM / Kimi / MiniMax 等）变成本地的 OpenAI 与 Anthropic 兼容接口 —— Claude Code / opencode / Cursor / Trae / Cherry Studio / LobeChat 等任意支持自定义 Base URL 的客户端可直连。附网页控制台与 DeepSeek Harness 原生插件。零依赖，仅监听 127.0.0.1。 JavaScript JavaScript · 52 au total
    Gagnées en 30 j +52 Score 5,2 Total 52
Afficher la suite

Émergents en Inférence & passerelles LLM Semaines passées : 26 oct.19 oct.