The Gateway Index / Gateways & Proxies / #52
yalun753/moe-l2
by yalun753 · Gateways & Proxies · updated 2d ago
MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert cache with LRU hot experts, OpenAI-compatible proxy, GGUF multi-shard.
64
momentum
142
stars
2
forks
#52
rank
consumer-gpudeepseekexpert-cachingllama-cppllm-inferencemoevram-optimization
View on GitHub →
More in Gateways & Proxies
OmniRoute
diegosouzapw
Never stop coding. Free MIT AI gateway: one endpoint, 352 providers (150+ free), 1200+ mod
litellm
BerriAI
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (o
new-api
QuantumNous
A unified AI model hub for aggregation & distribution. It supports cross-converting variou
kong
Kong
🦍 The API and AI Gateway