The Gateway Index / Caching & Cost / #127
messkan/prompt-cache
by messkan · Caching & Cost · updated 1mo ago
Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic caching.A drop-in, provider-agnostic LLM proxy written in Go with sub-millisecond response
53
momentum
428
stars
44
forks
#127
rank
aibadgerdbcacheclaudecost-optimizationgolangchainllmmiddlewaremistralopenaiperformance
View on GitHub →
