Limits
Limits are per key and per workspace, set by plan, shown on the key in the Console and reported in the response headers. Every number is a placeholder until decided Not decided yet.
| Limit | Scope | Value | Over the limit |
|---|---|---|---|
| Requests per minute | Key, workspace | Not decided yet by plan | 429 rate_limit_exceeded, param: "requests" |
| Tokens per minute (language models) | Key, workspace | Not decided yet | 429 rate_limit_exceeded, param: "tokens" |
| Task units per minute | Key, workspace | Not counted today: a task request counts toward requests per minute | None |
| Concurrent streams | Key | Not counted today | None |
Jobs in queued, assigned or running | Workspace | No job route is built yet | None |
| Request body size | Request | Not decided yet | 400 request_too_large |
| Inline input bytes per request (task API) | Request | Not decided yet and the model's hips.limits | 400 input_too_large (send it as an upload) |
| Inputs per task request | Request | The model's hips.limits.max_inputs Not decided yet | 400 too_many_inputs |
messages size and context_length | Request | The model's context_length, or the largest window any GPU serves it at now (the pool's context_window on Models) | 400 context_length_exceeded, the count and the limit in the message; the prompt is never truncated |
max_tokens | Request | The model's max_output_tokens | 400 max_tokens_exceeded |
stop entries (stop_sequences on Messages) | Request | 16 (decided 2026-11-01) | 400 invalid_value naming the field |
tools per request, json_schema size | Request | Not decided yet | 400 invalid_value naming the field |
prompt_cache_key | Request | Default proposed: 512 characters Not decided yet | 400 invalid_value |
| Embedding inputs, rerank documents | Request | 2,048 strings on /v1/embeddings; 1,000 documents on /v1/rerank, each and the query at most 32 KiB | 400 too_many_inputs; 400 input_too_long |
| Upload size, uploads per day | Upload, workspace | Not decided yet by plan; uploads are not offered in production yet | 400 upload_too_large, 429 rate_limit_exceeded param: "uploads" |
| Body arrival | Connection | A stalled body is closed after Not decided yet | 400 request_timeout |
A task's idempotency-key after its request completed | Request | Held for the window Not decided yet; no answer is kept | 404 result_expired, naming the first request's req_ id |
| Embedding and rerank time | Request | The gateway's time limit for a whole answer | 400 exceeds_time_limit before any work, when the measured speed shows it cannot finish |
- Limit
- Requests per minute
- Scope
- Key, workspace
- Value
- Not decided yet by plan
- Over the limit
- 429
rate_limit_exceeded,param: "requests"
- Limit
- Tokens per minute (language models)
- Scope
- Key, workspace
- Value
- Not decided yet
- Over the limit
- 429
rate_limit_exceeded,param: "tokens"
- Limit
- Task units per minute
- Scope
- Key, workspace
- Value
- Not counted today: a task request counts toward requests per minute
- Over the limit
- None
- Limit
- Concurrent streams
- Scope
- Key
- Value
- Not counted today
- Over the limit
- None
- Limit
- Jobs in
queued,assignedorrunning - Scope
- Workspace
- Value
- No job route is built yet
- Over the limit
- None
- Limit
- Request body size
- Scope
- Request
- Value
- Not decided yet
- Over the limit
- 400
request_too_large
- Limit
- Inline input bytes per request (task API)
- Scope
- Request
- Value
- Not decided yet and the model's
hips.limits - Over the limit
- 400
input_too_large(send it as an upload)
- Limit
- Inputs per task request
- Scope
- Request
- Value
- The model's
hips.limits.max_inputsNot decided yet - Over the limit
- 400
too_many_inputs
- Limit
messagessize andcontext_length- Scope
- Request
- Value
- The model's
context_length, or the largest window any GPU serves it at now (the pool'scontext_windowon Models) - Over the limit
- 400
context_length_exceeded, the count and the limit in the message; the prompt is never truncated
- Limit
max_tokens- Scope
- Request
- Value
- The model's
max_output_tokens - Over the limit
- 400
max_tokens_exceeded
- Limit
stopentries (stop_sequenceson Messages)- Scope
- Request
- Value
- 16 (decided 2026-11-01)
- Over the limit
- 400
invalid_valuenaming the field
- Limit
toolsper request,json_schemasize- Scope
- Request
- Value
- Not decided yet
- Over the limit
- 400
invalid_valuenaming the field
- Limit
prompt_cache_key- Scope
- Request
- Value
- Default proposed: 512 characters Not decided yet
- Over the limit
- 400
invalid_value
- Limit
- Embedding inputs, rerank documents
- Scope
- Request
- Value
- 2,048 strings on
/v1/embeddings; 1,000 documents on/v1/rerank, each and the query at most 32 KiB - Over the limit
- 400
too_many_inputs; 400input_too_long
- Limit
- Upload size, uploads per day
- Scope
- Upload, workspace
- Value
- Not decided yet by plan; uploads are not offered in production yet
- Over the limit
- 400
upload_too_large, 429rate_limit_exceededparam: "uploads"
- Limit
- Body arrival
- Scope
- Connection
- Value
- A stalled body is closed after Not decided yet
- Over the limit
- 400
request_timeout
- Limit
- A task's
idempotency-keyafter its request completed - Scope
- Request
- Value
- Held for the window Not decided yet; no answer is kept
- Over the limit
- 404
result_expired, naming the first request'sreq_id
- Limit
- Embedding and rerank time
- Scope
- Request
- Value
- The gateway's time limit for a whole answer
- Over the limit
- 400
exceeds_time_limitbefore any work, when the measured speed shows it cannot finish
x-ratelimit-limit-requests, x-ratelimit-remaining-requests and x-ratelimit-reset-requests carry the key's request limit in the current window, what remains, and the time to reset in OpenAI's duration format; the token equivalents do the same for tokens on language-model routes, and for task units on the task API under the same header names. Clients that already read them keep working.
A limit is never a 5xx. rate_limit_exceeded names the dimension in param (requests, tokens, units, uploads); concurrency_limit_exceeded names streams or jobs. Both carry Retry-After in seconds: wait that long, then retry; nothing was charged on the refusal, and the retry is a new quote. Rate-limit windows and burst behavior are the gateway's and will be documented with the numbers.
Related sheets
- Sheet
- 19 of 20
- Detail
- A
- Updated
- 2026-11-15