Batch jobs
Send many requests at once and collect the results when they are done. HIPS runs batch lines when GPU capacity is free, at 50% less than the on-demand price of the same model, and tells you when the batch is finished. The API is OpenAI's Batch API, so the official SDKs and the batch scripts you already have work against HIPS unchanged.
Note
Batch is decided and not yet served. The queue of enqueued tokens per model and workspace is set per plan, and its values are still open Not decided yet.
Step 1: Write a JSONL file, one request per line, each with your own
custom_id.Step 2: Upload it with
POST /v1/filesand the purposebatch.Step 3: Create the batch with
POST /v1/batches, naming the endpoint and the completion window. The price is quoted and bound here.Step 4: HIPS runs the lines when capacity is free. Live requests always come first.
Step 5: When the batch ends, a signed webhook and a mail tell you; download the output and error files.
import jsonimport osfrom openai import OpenAI client = OpenAI( api_key=os.environ["HIPS_API_KEY"], base_url=os.environ["HIPS_BASE_URL"], # https://ai.hips.com/v1) # One request per line, each with your own custom_id.with open("requests.jsonl", "w") as f: for i, text in enumerate(["First document.", "Second document."]): f.write(json.dumps({ "custom_id": f"doc-{i}", "method": "POST", "url": "/v1/chat/completions", "body": { "model": os.environ["HIPS_MODEL"], "messages": [{"role": "user", "content": f"Summarize: {text}"}], }, }) + "\n") batch_file = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")batch = client.batches.create( input_file_id=batch_file.id, endpoint="/v1/chat/completions", completion_window="24h",)print(batch.id, batch.status)The output file holds one line per finished request, matched to your input by custom_id, in no particular order. Lines that failed, and lines still unfinished when the window closed, come back in the error file.
Python
batch = client.batches.retrieve(batch.id) if batch.status == "completed": for line in client.files.content(batch.output_file_id).text.splitlines(): record = json.loads(line) print(record["custom_id"], record["response"]["status_code"]) # Lines that failed, or expired when the window closed, are in the error file. if batch.error_file_id: print(client.files.content(batch.error_file_id).text)On https://ai.hips.com, in OpenAI's Batch API shape.
/v1/filespreviewUpload the JSONL input file with the purpose batch.
/v1/batchespreviewCreate a batch from an input file: input_file_id, endpoint, completion_window.
/v1/batches/{id}previewA batch with its status, its line counts and its output and error file ids.
/v1/batches/{id}/cancelpreviewCancel a batch. Finished lines are delivered and charged; the rest are not.
/v1/batchespreviewThe workspace's batches, newest first.
/v1/files/{id}/contentpreviewDownload an output or error file.
/v1/chat/completions, for language models.- Embeddings.
- The HIPS task API, for every other model type: a line that OpenAI's Batch API does not offer.
- One model per input file; your own model's lines follow the pools it may run in.
You choose how long HIPS may take: 24h by default, or any whole number of hours or days up to 7d. The price is the same for every window. No finish time is promised inside it: everything that finishes is delivered, and lines still unfinished when the window closes come back as expired and are not charged.
| Batch | |
|---|---|
| Discount | 50% less than the on-demand rate of the same model, pool and unit, applied to every rate a line uses: input, cached input, output and every task unit |
| Quote | Bound when POST /v1/batches accepts the batch, and held for every line through the window |
| Prompt caching | The batch discount and the cached-input rate multiply (Prompt caching) |
| Balance | The batch's maximum charge (input at batch rates plus each line's output cap) is held against your prepaid balance at creation and released line by line as lines finish or expire |
| Expired lines | Not charged |
| A model without batch | A model whose price cannot carry the discount is sold without batch rather than at a smaller discount, and the catalog says so |
- Discount
- Batch
- 50% less than the on-demand rate of the same model, pool and unit, applied to every rate a line uses: input, cached input, output and every task unit
- Quote
- Batch
- Bound when
POST /v1/batchesaccepts the batch, and held for every line through the window
- Prompt caching
- Batch
- The batch discount and the cached-input rate multiply (Prompt caching)
- Balance
- Batch
- The batch's maximum charge (input at batch rates plus each line's output cap) is held against your prepaid balance at creation and released line by line as lines finish or expire
- Expired lines
- Batch
- Not charged
- A model without batch
- Batch
- A model whose price cannot carry the discount is sold without batch rather than at a smaller discount, and the catalog says so
Actual prices are set per model in the catalog; the Pricing page shows the shape.
The limits match OpenAI's, so existing scripts run unchanged
| Limit | Value |
|---|---|
| Lines per input file | 50,000 |
| Input file size | 200 MB |
| Models per input file | One |
| Embedding inputs per batch | 50,000 |
| Batches created per hour | 2,000 |
| Open batches per workspace | 100 |
| Enqueued input tokens per model and workspace | Set per plan Not decided yet |
- Limit
- Lines per input file
- Value
- 50,000
- Limit
- Input file size
- Value
- 200 MB
- Limit
- Models per input file
- Value
- One
- Limit
- Embedding inputs per batch
- Value
- 50,000
- Limit
- Batches created per hour
- Value
- 2,000
- Limit
- Open batches per workspace
- Value
- 100
- Limit
- Enqueued input tokens per model and workspace
- Value
- Set per plan Not decided yet
- On every eligible idle GPU: personal devices the moment they come online, and spare capacity in the approved data-center pool.
- Live requests always come first. A device finishes the batch line it is running before it takes a live request.
- Lines that share a long prompt prefix are grouped and run in order on one worker, so the shared part is cached once.
- A batch line on your own model runs only where the model may run: the approved pool, at launch.
Batch input and output files are kept 30 days and then deleted, sooner when you set a shorter expiry (expires_after, output_expires_after) or delete them. Their content follows the same training rule as a live request on the same plan (Data handling).
Related sheets
- Sheet
- 07 of 13
- Updated
- 2026-10-21
- Status
- Preview