Skip to content
HIPS
  • Earn with your GPU
  • Pricing
  • Docs
Sign inJoin early access
⌘K

Get started

  1. 01Developer hub
  2. 02Quickstart
  3. 03Migrate to HIPS

Guides

  1. 04HIPS SDK
  2. 05Import and run your model
  3. 06Model and task guides
    1. ATask guides
    2. BLanguage (chat)
    3. CClassification
    4. DImage
    5. EVideo
    6. FAudio
    7. GDocument processing
  4. 07Batch jobs
  5. 08Prompt caching
  6. 09Code with HIPS
  7. 10Coding-tool setup
    1. AHIPS Code
    2. BClaude Code
    3. CCodex
    4. DOpenCode
    5. EKilo
    6. FPi
    7. GQwen Code
    8. HCrush
    9. ICline
    10. JPrime Agent
    11. KAider
    12. LGoose
    13. MHermes Agent
    14. NGrok Build
    15. ODeepSeek Harness
    16. POther coding tools and editor integrations
  8. 11Command line
    1. AInstall and update
    2. BSign in
    3. CStatus and models
    4. DCoding agents
  9. 12Console
    1. AAccount
    2. BWorkspaces
    3. CAPI keys
    4. DSettings
    5. EUsage
    6. FModels
    7. GPlayground
    8. HDeployments
  10. 13Billing
  11. 14Provider setup
    1. AWhen it works
    2. BCommands
    3. CSettings
    4. DProvider Console
    5. EEarnings
    6. FCrash reports
    7. GData centers
  12. 15Semantic routing
  13. 16Guardrails
  14. 17Knowledge

Reference

  1. 18API reference
    1. AAuthentication
    2. BChat completions
    3. CResponses
    4. DMessages
    5. ECount tokens
    6. FModels
    7. GPrices API
    8. HTasks
    9. IEmbeddings and rerank
    10. JImages
    11. KJobs
    12. LUploads
    13. MFiles
    14. NBatches
    15. OWebhooks
    16. PVector stores
    17. QDeployments
    18. RRequirements and headers
    19. SErrors
    20. TUsage
    21. UVersioning
  2. 19Operating guidance
    1. ALimits
    2. BErrors
    3. CRetries and billing outcomes
    4. DIdempotency
    5. EStreaming
    6. FData handling
    7. GCompatibility matrix
  3. 20Changelog
Sheet indexPrompt caching

Get started

  1. 01Developer hub
  2. 02Quickstart
  3. 03Migrate to HIPS

Guides

  1. 04HIPS SDK
  2. 05Import and run your model
  3. 06Model and task guides
    1. ATask guides
    2. BLanguage (chat)
    3. CClassification
    4. DImage
    5. EVideo
    6. FAudio
    7. GDocument processing
  4. 07Batch jobs
  5. 08Prompt caching
  6. 09Code with HIPS
  7. 10Coding-tool setup
    1. AHIPS Code
    2. BClaude Code
    3. CCodex
    4. DOpenCode
    5. EKilo
    6. FPi
    7. GQwen Code
    8. HCrush
    9. ICline
    10. JPrime Agent
    11. KAider
    12. LGoose
    13. MHermes Agent
    14. NGrok Build
    15. ODeepSeek Harness
    16. POther coding tools and editor integrations
  8. 11Command line
    1. AInstall and update
    2. BSign in
    3. CStatus and models
    4. DCoding agents
  9. 12Console
    1. AAccount
    2. BWorkspaces
    3. CAPI keys
    4. DSettings
    5. EUsage
    6. FModels
    7. GPlayground
    8. HDeployments
  10. 13Billing
  11. 14Provider setup
    1. AWhen it works
    2. BCommands
    3. CSettings
    4. DProvider Console
    5. EEarnings
    6. FCrash reports
    7. GData centers
  12. 15Semantic routing
  13. 16Guardrails
  14. 17Knowledge

Reference

  1. 18API reference
    1. AAuthentication
    2. BChat completions
    3. CResponses
    4. DMessages
    5. ECount tokens
    6. FModels
    7. GPrices API
    8. HTasks
    9. IEmbeddings and rerank
    10. JImages
    11. KJobs
    12. LUploads
    13. MFiles
    14. NBatches
    15. OWebhooks
    16. PVector stores
    17. QDeployments
    18. RRequirements and headers
    19. SErrors
    20. TUsage
    21. UVersioning
  2. 19Operating guidance
    1. ALimits
    2. BErrors
    3. CRetries and billing outcomes
    4. DIdempotency
    5. EStreaming
    6. FData handling
    7. GCompatibility matrix
  3. 20Changelog
08
  1. Docs/
  2. Prompt caching
llms.txt

Prompt caching

On this sheet5 sections
  1. What it costs today
  2. Read it in the usage
  3. What the gateway takes today
  4. What HIPS has decided
  5. Limits

HIPS keeps no prompt cache yet: every GPU engine that serves a request runs with its prompt cache off, so each request reads its whole prompt, every answer reports cached_tokens: 0, and every input token is billed at the model's input rate. There is nothing to switch on, and writing a cache is never charged.

F
Fig. 1, sheet 08Prompt caching

What it costs today

TokenRate
InputThe model's input rate, for every input token
Cached inputNone: no prefix is reused, so no request pays a cached-input rate
Cache writeNo charge. cache_write_tokens is reported as 0
OutputThe model's output rate
In a batchThe batch rates, every input token at the batch input rate
  • Token
    Input
    Rate
    The model's input rate, for every input token
  • Token
    Cached input
    Rate
    None: no prefix is reused, so no request pays a cached-input rate
  • Token
    Cache write
    Rate
    No charge. cache_write_tokens is reported as 0
  • Token
    Output
    Rate
    The model's output rate
  • Token
    In a batch
    Rate
    The batch rates, every input token at the batch input rate

Read it in the usage

Parts list: Usage fields

  1. 01
    usage.prompt_tokens

    Every input token, as OpenAI counts them.

  2. 02
    usage.prompt_tokens_details.cached_tokens

    0 on every answer today: no prefix is reused yet.

  3. 03
    cache_write_tokens

    Always 0: writing a cache is not charged.

  4. 04
    x-hips-cost-micros

    The request's cost, every input token at the input rate; on a stream, the hips object of the final usage chunk.

The Messages API reports the same counts as cache_read_input_tokens and cache_creation_input_tokens, and the Responses API as input_tokens_details.cached_tokens: each is 0 today. The Console's Usage and activity view counts cached input apart from the rest of the input, and it reads 0.

What the gateway takes today

  • prompt_cache_key, a string of at most 512 characters: accepted, and never logged or stored. It does nothing more yet.
  • cache_salt: refused with 400 unknown_parameter. What keeps one workspace's cache apart from another's is HIPS's to derive, never the client's.
  • prompt_cache_options, prompt_cache_breakpoint and prompt_cache_retention: refused with 400 unsupported_parameter, since HIPS promises no cache lifetime and takes no breakpoint.
  • On the Messages API, cache_control {"type": "ephemeral"} where Anthropic places it: accepted, and it changes nothing. A ttl is a 400.

What HIPS has decided

How prompt caching will work is decided, for both pools, and it is not built yet:

  • It will run on every GPU that serves a language model: providers' devices in the Community pool and HIPS's own servers in the Certified pool.
  • A workspace's prompt prefix will be kept only in the memory of the engine on the GPU that ran the request. HIPS's software never writes it to a disk, a database or a log, and no answer is ever cached.
  • It will be reused only by the same workspace, never by another workspace's request on any GPU, and prompt_cache_key narrows it further within the workspace.
  • A cache nobody has used for 30 minutes will be dropped, and nothing will reuse it after that. It will also end at once when the workspace is deleted. A busy GPU may overwrite a cache sooner, and no lifetime is promised.
  • It will come first for speed, and change no price: until HIPS can verify each hit and a price for cached input is decided, every input token stays billed at the input rate.

Limits

  • Every input token counts toward your tokens-per-minute limit.
  • No cache lifetime is promised.

Related sheets

  • Batch jobs
  • Usage reference
  • Pricing
Previous sheetBatch jobs
Next sheetCode with HIPS
Sheet
08 of 20
Updated
2026-11-16
Sign in to the Console

On this sheet

  1. What it costs today
  2. Read it in the usage
  3. What the gateway takes today
  4. What HIPS has decided
  5. Limits

Product

  • Inference
  • Models
  • Deploy your model
  • Always-on GPU capacity
  • How it works
  • Use cases
  • Pricing

Developer

  • Docs
  • Quickstart
  • Migrate to HIPS
  • HIPS SDK
  • Batch jobs
  • API reference

Provider

  • Earn with your GPU
  • Register GPU interest
  • Download for Mac
  • Download for Windows
  • Download for Linux

Resource

  • Security and Data
  • Resources
  • llms.txt

Compare

  • OpenRouter alternatives
  • Together AI alternatives
  • Groq alternatives
  • All comparisons

Company

  • About
  • Contact
  • Affiliate program
  • HIPS Payments

Legal

  • Legal
  • Consumer Health Data Privacy Policy
  • Report content
  • Cookie preferences

Support

  • Contact HIPS
  • Looking for payments support?

HIPS. AI inference for builders.

hips.com

Hyperscale Inference Processing Systems OÜ