Skip to content
HIPS
  • Earn with your GPU
  • Pricing
  • Docs
Sign inJoin early access
⌘K

Get started

  1. 01Developer hub
  2. 02Quickstart
  3. 03Migrate to HIPS

Guides

  1. 04HIPS SDK
  2. 05Import and run your model
  3. 06Model and task guides
    1. ATask guides
    2. BLanguage (chat)
    3. CClassification
    4. DImage
    5. EVideo
    6. FAudio
    7. GDocument processing
  4. 07Batch jobs
  5. 08Prompt caching
  6. 09Code with HIPS
  7. 10Coding-tool setup
    1. AHIPS Code
    2. BClaude Code
    3. CCodex
    4. DOpenCode
    5. EKilo
    6. FPi
    7. GQwen Code
    8. HCrush
    9. ICline
    10. JPrime Agent
    11. KAider
    12. LGoose
    13. MHermes Agent
    14. NGrok Build
    15. ODeepSeek Harness
    16. POther coding tools and editor integrations
  8. 11Command line
    1. AInstall and update
    2. BSign in
    3. CStatus and models
    4. DCoding agents
  9. 12Console
    1. AAccount
    2. BWorkspaces
    3. CAPI keys
    4. DSettings
    5. EUsage
    6. FModels
    7. GPlayground
    8. HDeployments
  10. 13Billing
  11. 14Provider setup
    1. AWhen it works
    2. BCommands
    3. CSettings
    4. DProvider Console
    5. EEarnings
    6. FCrash reports
    7. GData centers
  12. 15Semantic routing
  13. 16Guardrails
  14. 17Knowledge

Reference

  1. 18API reference
    1. AAuthentication
    2. BChat completions
    3. CResponses
    4. DMessages
    5. ECount tokens
    6. FModels
    7. GPrices API
    8. HTasks
    9. IEmbeddings and rerank
    10. JImages
    11. KJobs
    12. LUploads
    13. MFiles
    14. NBatches
    15. OWebhooks
    16. PVector stores
    17. QDeployments
    18. RRequirements and headers
    19. SErrors
    20. TUsage
    21. UVersioning
  2. 19Operating guidance
    1. ALimits
    2. BErrors
    3. CRetries and billing outcomes
    4. DIdempotency
    5. EStreaming
    6. FData handling
    7. GCompatibility matrix
  3. 20Changelog
Sheet indexOperating guidance

Get started

  1. 01Developer hub
  2. 02Quickstart
  3. 03Migrate to HIPS

Guides

  1. 04HIPS SDK
  2. 05Import and run your model
  3. 06Model and task guides
    1. ATask guides
    2. BLanguage (chat)
    3. CClassification
    4. DImage
    5. EVideo
    6. FAudio
    7. GDocument processing
  4. 07Batch jobs
  5. 08Prompt caching
  6. 09Code with HIPS
  7. 10Coding-tool setup
    1. AHIPS Code
    2. BClaude Code
    3. CCodex
    4. DOpenCode
    5. EKilo
    6. FPi
    7. GQwen Code
    8. HCrush
    9. ICline
    10. JPrime Agent
    11. KAider
    12. LGoose
    13. MHermes Agent
    14. NGrok Build
    15. ODeepSeek Harness
    16. POther coding tools and editor integrations
  8. 11Command line
    1. AInstall and update
    2. BSign in
    3. CStatus and models
    4. DCoding agents
  9. 12Console
    1. AAccount
    2. BWorkspaces
    3. CAPI keys
    4. DSettings
    5. EUsage
    6. FModels
    7. GPlayground
    8. HDeployments
  10. 13Billing
  11. 14Provider setup
    1. AWhen it works
    2. BCommands
    3. CSettings
    4. DProvider Console
    5. EEarnings
    6. FCrash reports
    7. GData centers
  12. 15Semantic routing
  13. 16Guardrails
  14. 17Knowledge

Reference

  1. 18API reference
    1. AAuthentication
    2. BChat completions
    3. CResponses
    4. DMessages
    5. ECount tokens
    6. FModels
    7. GPrices API
    8. HTasks
    9. IEmbeddings and rerank
    10. JImages
    11. KJobs
    12. LUploads
    13. MFiles
    14. NBatches
    15. OWebhooks
    16. PVector stores
    17. QDeployments
    18. RRequirements and headers
    19. SErrors
    20. TUsage
    21. UVersioning
  2. 19Operating guidance
    1. ALimits
    2. BErrors
    3. CRetries and billing outcomes
    4. DIdempotency
    5. EStreaming
    6. FData handling
    7. GCompatibility matrix
  3. 20Changelog
19
  1. Docs/
  2. Operating guidance/
  3. Streaming
(Detail E)
llms.txt

Streaming

On this sheet2 sections
  1. Cancellation
  2. Jobs are not streams

stream: true on POST /v1/chat/completions answers 200 with Content-Type: text/event-stream, one data: line per chunk in OpenAI's chat.completion.chunk shape (choices[].delta with role, content, tool_calls[] by index; finish_reason on the last content chunk), then data: [DONE].

F
Fig. 1, sheet 19Operating guidance
RuleBehavior
UsageWith stream_options.include_usage: true, one final chunk before [DONE] with choices: [] and usage. Without it, no usage chunk.
The hips objectThe headers that need the finished answer (x-hips-cost-micros, x-hips-latency-ms) cannot be sent at the head of a stream. They are omitted from the head, and the final usage chunk carries a top-level hips object: {"cost_micros", "latency_ms", "quote"}, then first_token_ms (from the request's receipt to the answer's first token) and gpu ({"model", "vendor", "backend"}) where they are known. gpu.model is the model of the GPU that served the answer, as the serving device reported it for that GPU, and is null only where the device reported no GPU product name; its vendor and backend are still named then. A non-streamed answer carries the same object as its last member, without first_token_ms: the engine answers it whole, so a client that wants the time to the first token streams. It is sent when include_usage is true; whether it is sent without it is open Not decided yet. The other x-hips-* headers are in the head as usual.
Keep-aliveA comment line (: keep-alive) at an interval Not decided yet while the engine produces nothing.
A failure after the stream startedOne data: line holding the error envelope, then the stream closes without [DONE]. OpenAI clients raise on it. The usage record shows the outcome.
Client disconnectClosing the connection cancels the job on the worker. The charge for tokens already delivered is Not decided yet (proposed: charged).
RelayThe gateway relays chunks as the worker emits them and buffers nothing beyond one chunk.
  • Rule
    Usage
    Behavior
    With stream_options.include_usage: true, one final chunk before [DONE] with choices: [] and usage. Without it, no usage chunk.
  • Rule
    The hips object
    Behavior
    The headers that need the finished answer (x-hips-cost-micros, x-hips-latency-ms) cannot be sent at the head of a stream. They are omitted from the head, and the final usage chunk carries a top-level hips object: {"cost_micros", "latency_ms", "quote"}, then first_token_ms (from the request's receipt to the answer's first token) and gpu ({"model", "vendor", "backend"}) where they are known. gpu.model is the model of the GPU that served the answer, as the serving device reported it for that GPU, and is null only where the device reported no GPU product name; its vendor and backend are still named then. A non-streamed answer carries the same object as its last member, without first_token_ms: the engine answers it whole, so a client that wants the time to the first token streams. It is sent when include_usage is true; whether it is sent without it is open Not decided yet. The other x-hips-* headers are in the head as usual.
  • Rule
    Keep-alive
    Behavior
    A comment line (: keep-alive) at an interval Not decided yet while the engine produces nothing.
  • Rule
    A failure after the stream started
    Behavior
    One data: line holding the error envelope, then the stream closes without [DONE]. OpenAI clients raise on it. The usage record shows the outcome.
  • Rule
    Client disconnect
    Behavior
    Closing the connection cancels the job on the worker. The charge for tokens already delivered is Not decided yet (proposed: charged).
  • Rule
    Relay
    Behavior
    The gateway relays chunks as the worker emits them and buffers nothing beyond one chunk.

Note

The cost of a streamed answer arrives in the hips object of the final usage chunk, not in x-hips-cost-micros: the cost is not known when the head is sent, and trailers do not pass the edge. Ask for stream_options.include_usage to receive it.

Cancellation

Close the connection. The job is canceled on the worker; an attempt interrupted after the first streamed token is never restarted transparently, because a stream cannot be resumed. A tool call streamed as delta.tool_calls by index is complete only when its finish_reason arrives.

Jobs are not streams

Long-running work goes through jobs: submit, poll GET /v1/jobs/{id}, cancel with POST /v1/jobs/{id}/cancel, retrieve the result until it expires. A completion webhook is open Not decided yet.

Related sheets

  • Chat completions reference
  • Retries and billing outcomes
Previous sheetIdempotency
Next sheetData handling
Sheet
19 of 20
Detail
E
Updated
2026-11-09
Sign in to the Console

On this sheet

  1. Cancellation
  2. Jobs are not streams

Product

  • Inference
  • Models
  • Deploy your model
  • Always-on GPU capacity
  • How it works
  • Use cases
  • Pricing

Developer

  • Docs
  • Quickstart
  • Migrate to HIPS
  • HIPS SDK
  • Batch jobs
  • API reference

Provider

  • Earn with your GPU
  • Register GPU interest
  • Download for Mac
  • Download for Windows
  • Download for Linux

Resource

  • Security and Data
  • Resources
  • llms.txt

Compare

  • OpenRouter alternatives
  • Together AI alternatives
  • Groq alternatives
  • All comparisons

Company

  • About
  • Contact
  • Affiliate program
  • HIPS Payments

Legal

  • Legal
  • Consumer Health Data Privacy Policy
  • Report content
  • Cookie preferences

Support

  • Contact HIPS
  • Looking for payments support?

HIPS. AI inference for builders.

hips.com

Hyperscale Inference Processing Systems OÜ