flashinfer

In short

FlashInfer is a GPU kernel library for LLM inference, providing attention, GEMM and MoE operations with multiple backends. It is aimed at engineers building low-latency LLM serving systems.

Summary written by AI from the repository’s description and README.