The Gateway Index / Gateways & Proxies / #125
yalun753/moe-l2
by yalun753 · Gateways & Proxies · updated 25d ago
MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert cache with LRU hot experts, OpenAI-compatible proxy, GGUF multi-shard.
55
momentum
146
stars
5
forks
#125
rank
consumer-gpudeepseekexpert-cachingllama-cppllm-inferencemoevram-optimization
View on GitHub →
More in Gateways & Proxies
OmniRoute
cheaper-inference
Never stop coding. Free MIT AI gateway: one endpoint, 375 providers (150+ free), 1,400+ mo
litellm
BerriAI
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (o
new-api
QuantumNous
A unified AI model hub for aggregation & distribution. It supports cross-converting variou
kong
Kong
🦍 The API and AI Gateway