sglang

MemT-1.0

+90

Official repository for MemT-1.0

kvcached

+85

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

zero-to-sglang

+87

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

EAGLE-4

+76

Official Inference Code of EAGLE-4

sglang-omni

+47

High Performance Serving Runtime For Interactive Super Intelligence System

gpustack

+50

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

Mooncake

+49

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

InferenceX

+41

Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

dynamo

+36

A Datacenter Scale Distributed Inference Serving Framework