Retries and billing outcomes
Prelaunch
Do not assume that every failed or retried attempt has the same billing outcome.
The principle: the customer is charged once per completed answer, at the quote bound at acceptance. A failed, rejected, interrupted or timed-out attempt costs the customer nothing; what the provider is paid for it is defined independently. The outcomes below are proposals until decided Not decided yet, except where the rule is the company's.
| Situation | Who retries | Billing outcome |
|---|---|---|
| 429 or 503 | The client, after Retry-After; the SDK does it | Nothing on the refusal; the retry is a new quote (market) or the same stable price |
| An attempt is rejected or interrupted before any output | The scheduler, on another eligible worker under the same requirements, up to Not decided yet attempts | Nothing until a completed answer; then once |
| An attempt is interrupted after the first streamed token | Nobody: a stream cannot be restarted transparently. The stream ends with the error envelope | Proposed: nothing (no complete answer); the tokens delivered are in the usage record as interrupted |
| An attempt is interrupted after the first token on a non-streamed request | The scheduler, within the attempt count | Once, for the completed answer |
| Retries exhausted | Nobody; 500 attempt_failed or job interrupted | Nothing |
| A worker timed out | The scheduler, once, on another worker, when the timeout was the worker's | Nothing |
| The client closes a stream | Nobody; the job is canceled on the worker | Not decided yet (proposed: the tokens delivered are charged) |
| The client cancels a queued job | Nobody | Nothing |
| The client cancels a running job | Nobody | Not decided yet (proposed: nothing) |
| A market price changes while a job runs | Nobody | The bound quote applies to work already accepted |
A retried request with the same idempotency-key | The idempotency rules | Once |
- Situation
- 429 or 503
- Who retries
- The client, after
Retry-After; the SDK does it - Billing outcome
- Nothing on the refusal; the retry is a new quote (market) or the same stable price
- Situation
- An attempt is rejected or interrupted before any output
- Who retries
- The scheduler, on another eligible worker under the same requirements, up to Not decided yet attempts
- Billing outcome
- Nothing until a completed answer; then once
- Situation
- An attempt is interrupted after the first streamed token
- Who retries
- Nobody: a stream cannot be restarted transparently. The stream ends with the error envelope
- Billing outcome
- Proposed: nothing (no complete answer); the tokens delivered are in the usage record as
interrupted
- Situation
- An attempt is interrupted after the first token on a non-streamed request
- Who retries
- The scheduler, within the attempt count
- Billing outcome
- Once, for the completed answer
- Situation
- Retries exhausted
- Who retries
- Nobody; 500
attempt_failedor jobinterrupted - Billing outcome
- Nothing
- Situation
- A worker timed out
- Who retries
- The scheduler, once, on another worker, when the timeout was the worker's
- Billing outcome
- Nothing
- Situation
- The client closes a stream
- Who retries
- Nobody; the job is canceled on the worker
- Billing outcome
- Not decided yet (proposed: the tokens delivered are charged)
- Situation
- The client cancels a queued job
- Who retries
- Nobody
- Billing outcome
- Nothing
- Situation
- The client cancels a running job
- Who retries
- Nobody
- Billing outcome
- Not decided yet (proposed: nothing)
- Situation
- A market price changes while a job runs
- Who retries
- Nobody
- Billing outcome
- The bound quote applies to work already accepted
- Situation
- A retried request with the same
idempotency-key - Who retries
- The idempotency rules
- Billing outcome
- Once
All open Not decided yet: the synchronous request limit on chat completions and tasks; the idle limit between streamed tokens; the per-task maximum run time of a job; the worker's own limits. A request refused before it starts because it cannot fit is a 400 use_jobs or a 503, never a wait until the limit.
- Retry 429 and 503 after
Retry-After. - Retry a connection failure before any byte was received.
- Retry a 500 once.
- Never retry a 400, 401, 402, 403 or 404 unchanged.
- On the task API send an
idempotency-keyso a retry can never double-bill. - A retried chat completion is a new request, a new quote and a new charge: the route takes no idempotency key, because the gateway stores no completion body.
Related sheets
- Sheet
- 12 of 13
- Detail
- C
- Updated
- 2026-10-21
- Status
- Preview