LLM gateway
Enterprise LLM gateway
Tokenjoy puts every team’s model calls behind one gateway. Apps use a single URL and a single API key. The gateway picks a channel by priority, checks permissions and quota, and records every call.
One endpoint, many models
Swap in the Tokenjoy base_url and API key and keep your existing OpenAI or Anthropic calls. One model can have several channels, and the gateway spreads requests across them by priority and weight.
- Compatible with OpenAI and Anthropic APIs
- Streaming, function calling and vision, depending on the model and channel
- Chat, image, audio, video, embedding and rerank models, managed in one model catalog
When a channel fails, requests keep flowing
- Failover: switch to another available channel as you configured
- Retries: retry on upstream errors by policy, with status and errors on record
- Rate limits: per API key request rate and burst caps
You decide who can use which model
Set the allowed models per department and per API key. Every call is checked, and a request for an unauthorized model is blocked and logged.
Every call leaves a trace
Each call records the caller, model, token usage and cost. Billing can be per token, per request or tiered, so you pay for what you use. Filter the call log by department, member, model or channel, and export it as CSV.
FAQ
What is an LLM gateway?
An LLM gateway sits between your applications and model providers. It gives apps one endpoint and adds routing, access control, budgets and audit logs on top.
Do I need to change my code to use Tokenjoy?
Usually you only change the base_url, the API key and the model name. What a model can do still depends on its protocol and channel.