The Gateway Index / Gateways & Proxies / #84
yalun753/moe-l2
by yalun753 · Gateways & Proxies · updated 4d ago
MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert cache with LRU hot experts, OpenAI-compatible proxy, GGUF multi-shard.
61
momentum
146
stars
4
forks
#84
rank
consumer-gpudeepseekexpert-cachingllama-cppllm-inferencemoevram-optimization
View on GitHub →
More in Gateways & Proxies
OmniRoute
diegosouzapw
Never stop coding. Free MIT AI gateway: one endpoint, 359 providers (150+ free), 1200+ mod
litellm
BerriAI
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (o
new-api
QuantumNous
A unified AI model hub for aggregation & distribution. It supports cross-converting variou
kong
Kong
🦍 The API and AI Gateway