Router

A router is a model name that chooses the model. Send hips-router-chat, hips-router-agent, hips-router-code or hips-router-research in model on chat completions, Responses or Messages, streamed or not, with your own tools, and HIPS chooses a model for each request among the models that can do that work, the fastest or the cheapest as you set it. The request is served and billed as the model chosen, and the answer names it.

A model joins and leaves a router by itself as it is imported, tested and priced; no list is kept by hand. A router chooses among catalog models only: never a workspace's own deployment (dep_...), never a draft, deprecated or retired model, and never a model the catalog offers only to buyers in certain countries. For the Coding router, a test with the coding agent of the route you call counts first: HIPS Code or OpenCode on chat completions, Claude Code on Messages, Codex on Responses. Where no model has passed that agent's test, a test with any of the four counts.

The request is the route's own, with a router's name in model. These samples read HIPS_API_KEY and HIPS_BASE_URL from the environment, as every sample on these pages does, and print the two headers that say what was chosen.

00000000000000000000000000000000000000001import os2from openai import OpenAI3 4client = OpenAI(5    api_key=os.environ["HIPS_API_KEY"],6    base_url=os.environ["HIPS_BASE_URL"],  # https://ai.hips.com/v17)8 9raw = client.chat.completions.with_raw_response.create(10    model="hips-router-code",  # or hips-router-code:fast, hips-router-code:cheap11    messages=[{"role": "user", "content": "Write a Python function that checks whether a number is prime."}],12)13print(raw.headers.get("x-hips-model"))   # the model chosen14print(raw.headers.get("x-hips-router"))  # how it was chosen15print(raw.parse().choices[0].message.content)

The same names work on POST /v1/responses and POST /v1/messages. For a coding agent, set the agent's model to the router's name, for example HIPS_MODEL=hips-router-code.

Each router has a preference for the workspace: Fastest sends each request to the model with the shortest measured time for its size, Cheapest to the lowest expected charge for its size at the rates its quote would bind. A router never set reads Cheapest. An owner or admin sets it on the Console's Router page; the change applies to the next requests and is recorded in the audit log. A suffix overrides the setting for one request:

• The measured time is the prompt's tokens over the measured prefill rate plus the expected output over the measured decode rate, on GPUs that hold the model loaded and free now. A model with no measured speed ranks after every model that has one, those among themselves by expected charge. Nothing is estimated from another model.
• The expected output is the request's own cap where it sets one, else the same length for every model, so a model with a long default is not made to look dear.
• A name and its suffix are lowercase, exactly as written. An empty or unknown suffix (hips-router-code:, hips-router-code:best) is 400 invalid_value at model: "The suffix of a router is :fast or :cheap, as in hips-router-code:fast."
• Any other name that starts with hips-router, a router written in another case included, is 404 model_not_found, and the message names the four. No catalog id, alias or pinned id starts with hips-router, so a router never collides with a model.
• A router chooses language models on chat completions, Responses and Messages only. Named on /v1/tasks, /v1/embeddings or /v1/rerank, it is 400 invalid_value at model.

The Router makes no promise about which model serves a request. It chooses before the first byte of the answer, in this order:

1. 01

   The candidates. The router's models, narrowed to those that can serve this request in a pool it may use: an image part needs a model that takes images, tools one that supports tool calls, response_format, logprobs, logit_bias and reasoning_effort a model that advertises them (parallel_tool_calls narrows nothing: it is a permission, served as false on a model without the feature), the prompt a window served now that holds it, and an output cap a model whose output limit reaches it. Then the requirements (x-hips-pool, x-hips-region, x-hips-pricing-mode), the pools the workspace may use and your guardrails. Then the supply now: a model no GPU can serve in any of those pools is out.

2. 02

   The read. Where two models or more remain, the request may be read first: a decision service judges its task (conversation, writing, extraction, reasoning, code or tool_use) and its difficulty (simple, moderate or hard) from a masked excerpt. A harder request is held to larger models, and the task ranks the models that fit it first: models that reason for reasoning, models tested with a coding agent for code, validated tool calls for tool_use, JSON schema output for extraction. Without an answer the request is chosen by rules alone, among the same candidates with no size floor and no task preference. A request with a region requirement is never read. A read that fails, times out or finds the day's reads used up never fails the request: it is chosen by rules alone.

3. 03

   The order. Models with a GPU ready now go before models that start on demand, since loading a model can take minutes. Within each, the preference's measure decides, ties going to the other measure, then to the catalog id, so every gateway chooses alike. Where the read asked for a larger model and none is ready now, a smaller one that is ready goes first, and the answer says floor=below.

4. 04

   The conversation. A conversation's next turn goes to the model its earlier turn was answered by, while that model is still a candidate, ready now and large enough for the new turn, so a coding agent's session stays on one model.

5. 05

   The tries. The first model is tried as if the request had named it. If it is refused before its first byte for capacity (no_capacity, pool_unavailable, overloaded, context_size_not_measured), the next one is tried, up to three models in all, within the request's own deadline. After the first byte the request is that model's: a stream that fails is never restarted on another model.

Everything about the answer is the chosen model's: model in the body names it, never the router, x-hips-model carries its pinned id, and usage its counts in the route's own names. x-hips-router says how it was chosen:

00000000000000000000000000000000000000001HTTP/2 2002content-type: application/json3x-hips-request-id: req_...4x-hips-model: <pinned id of the model chosen>5x-hips-router: hips-router-code; preference=cheapest; source=setting; read=decision; task=code; difficulty=hard

A request the read could not answer in time is chosen by rules alone, and the header says so:

00000000000000000000000000000000000000001x-hips-router: hips-router-code; preference=cheapest; source=setting; read=rules; reason=timeout

x-hips-router is on every answer to a request that named a router, a refusal after the name was read included: a refusal found among the candidates says read=rules with no reason, and a content moderation block, which comes before any choice, carries the name, preference and source alone. The answer's hips object carries the same as router, with model, the pinned id served, and each member only where it is known:

00000000000000000000000000000000000000001"hips": {2  "cost_micros": 0,3  "latency_ms": 0,4  "quote": "quote_...",5  "router": {6    "name": "hips-router-code",7    "preference": "cheapest",8    "source": "setting",9    "read": "decision",10    "task": "code",11    "difficulty": "hard",12    "model": "<pinned id of the model chosen>"13  }14}

The choice is made before the first byte, so x-hips-model and x-hips-router are in a stream's head, and a refusal is an ordinary status and body, never an error inside the stream.

A routed request is billed exactly as the model chosen: its quote (x-hips-quote), its rates in the pool that served it, its usage units, and x-hips-cost-micros or the hips object's cost_micros. There is no router fee, and reading the request is never on your usage. The usage record keeps the model chosen and, beside it, the router's name, the preference, its source, how it chose and why rules, never the task or the difficulty. Rate limits count a routed request as one request with its own tokens.

Every code of the Errors page applies to a routed request as to any. The Router's own answers:

router_no_candidate is never worth sending again unchanged: its message says why and what would serve. For example, while no model has passed a coding agent's test:

00000000000000000000000000000000000000001HTTP/2 4002x-hips-router: hips-router-code; preference=cheapest; source=setting; read=rules

GET /v1/models lists the four routers first, each with its use case, the workspace's preference, its suffixes, how many models it can choose for your key's default requirements and guardrails (candidates), and how many of those have a GPU ready now (ready, null while the capacity reading is stale). GET /v1/models/{id} answers a router's name with or without its suffix. The Models reference has the rest of the list.

00000000000000000000000000000000000000001{2  "id": "hips-router-code",3  "object": "model",4  "created": 0,5  "owned_by": "hips",6  "hugging_face_id": null,7  "hips": {8    "task": "chat",9    "router": {10      "use_case": "Coding",11      "preference": "cheapest",12      "suffixes": ["fast", "cheap"],13      "candidates": 0,14      "ready": 015    }16  }17}

• The requirement headers apply to every model a router may choose and narrow its candidates; none is weakened to find a model. Under Auto the pool is chosen for the model chosen exactly as if you had named it, and x-hips-pool names the pool that served.
• A model and pool access guardrail narrows the candidates to the models it allows, each judged as a request naming it would be. A router itself is kept out only where a guardrail names it: a deny list naming it, or a family such as hips-router-*, or an allow list that names another router and not this one.
• An allow list that names a router must also name the models it may choose: hips-router-code with language lets the Coding router choose among every language model, while hips-router-code alone allows no model, 403 model_not_allowed.
• Content moderation runs before any read or choice: a guardrail that blocks refuses the request first. A budget holds the chosen model's maximum charge, as for any request. Guardrails has the rules.

A batch runs one model, so a batch file whose lines name a router is refused at its check, 400 invalid_file_format at body.model on the first such line: "A router chooses a model for each request; a batch runs one model. Name a model in the file's lines." Batches are not offered in production yet.

• Router, in the Inference menu after Guardrails: one panel per use case with the name to call and its Copy, an example API call, what the router is for, its preference (Fastest or Cheapest; an owner or admin changes it, and anyone else sees why the control is closed), what it would choose now for a request of a sample size at each difficulty and without a read, under the workspace's tool defaults or a chosen key's requirements, how many models it can choose and why the others are out, the suffix that overrides it, and that it is billed as the model it chooses. Above the panels, the last 24 hours: how many requests were routed, how many were read first, how many were chosen by rules alone, and why.
• Models lists the four routers in a group of their own above the models; each opens its panel on the Router page.
• The Playground's model picker shows the routers first, with a preference control. Each answer says which model was chosen and how, and its Details show the router, the preference and where it came from, the read and the tries.
• Usage names a routed request "hips-router-code, served by <model>" and filters by router.

The read receives a masked excerpt of the request, at most 4,096 bytes: the newest user message's text, then the system text, with every email address, phone number, Social Security number, card number, IBAN, IP address and secret replaced by a typed placeholder, and the custom patterns of your content moderation guardrails masked too. Images are never read. The excerpt goes to a decision service run outside HIPS for the one call, and is never stored, logged or sent to a model's server; a masking that cannot finish within its bound sends nothing. A verdict and a conversation's model are kept under keyed hashes for a bounded time, never as text.