The Gateway Index / Routers / #55
azrtydxb/Fastllm-proxy
by azrtydxb · Routers · updated today
The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLLM/SGLang — 0.76 µs per request, no I/O on the request path, cache-affinity routing, RBAC, budgets and a 13-screen UI in the binary.
64
momentum
105
stars
93
forks
#55
rank
ai-gatewayinferencelitellmllmllm-gatewayload-balancingobservabilityopenai-apirbacreverse-proxyrustvllm
View on GitHub →