Rate limits
For configuration precedence, Workspace balance, spend windows, and the complete request check order, start with Limits and request enforcement.
ApiGo runtime-limit responses
ApiGo enforces independent API key and Workspace layers for RPM, TPM, and concurrency. A rejection returns HTTP 429, a scope- and dimension-specific error.code, and Retry-After.
| Layer | RPM | TPM | Concurrency |
|---|---|---|---|
| API key | api_key_rpm_exceeded |
api_key_tpm_exceeded |
api_key_concurrency_exceeded |
| Workspace | workspace_rpm_exceeded |
workspace_tpm_exceeded |
workspace_concurrency_exceeded |
RPM and TPM refill continuously. Concurrency becomes available when in-flight requests finish, fail, or are cancelled.
Retry policy
- Respect
Retry-Afterwhen present. - Use exponential backoff with jitter and a maximum retry count.
- Do not retry
402balance or spend-cap failures until the underlying condition changes. - Cap worker concurrency before fan-out reaches the gateway.
- Keep
X-Request-Idfor Call Logs and support.
Provider throttling
After ApiGo admits a request, the selected provider can still enforce account, model, region, RPM, TPM, or concurrency limits. Provider errors keep the original status and response body, so do not assume one stable provider error code.
A Routing fallback can move eligible traffic to another configured model or provider. It cannot bypass Workspace funds, API key spend caps, or ApiGo runtime limits.
Engineering checklist
- Set explicit timeouts and retry budgets on every client.
- Alert separately on API key
429, Workspace429,402, and provider failures. - Use separate keys per service or environment so one workload's key layer cannot starve another.
- Monitor Workspace aggregate traffic because every key shares that hard ceiling.
