LLM Failover Proxy
One endpoint that keeps working when an AI provider fails, rate-limits or slows down.
- problem
- Keep an OpenAI-compatible endpoint answering when a provider rate-limits, returns nothing usable or stalls mid-stream.
- my ownership
- Routing logic, failure classification, hedged requests, named model lists, CLI and terminal UI, persistence, tests and the npm release.
- architecture
- Node.js 22 behind one OpenAI-compatible HTTP and SSE contract, provider adapters, Promise.race hedging with AbortController, cooldowns persisted across restarts and zero runtime dependencies.
- delivered
- Pull request #4 merged upstream: 12 commits and +1,638 lines across 18 files, published as v1.8.0 on npm.
Forked from @ajoly21 and reviewed back into upstream, the shell in the agent lab below serves the same ordered model chain.