OctoFlux
LiveOne gateway for many OpenAI-compatible AI providers.
OctoFlux is an OpenAI-compatible LLM gateway. It sits between your application and multiple OpenAI-compatible AI providers and handles provider selection, API-key rotation, rate limiting, health tracking, retries and failover, so your application does not have to. The code is public on GitHub.
- OpenAI-compatible API, so an OpenAI SDK can use it with a new base URL and key
- Health-aware routing across providers, with priorities and model aliases
- API-key rotation and local rate-limit enforcement
- Failover that retries rate limits, timeouts and server errors, but not invalid requests
- Usage counters, metrics, an admin API and a web dashboard
- New providers added through configuration, without writing code