Deployments and imports

My models and deployments takes a customer model from import to inference, and pins catalog models to always-on capacity. The rules behind each step are on Import and run your model; this page is the Console's side.

1. 01

   Hub: Hugging Face.

2. 02

   Repository: a model repository in the hub's own form, and a revision. A Space is an application and a dataset is data; neither is a model source.

3. 03

   Access: a public repository needs no token; a private or gated one needs a read-only access token for its hub, chosen or added here.

4. 04

   Model: the task the model serves.

5. 05

   Where it runs: a language model runs in data centers only, in the Certified pool; another model type runs in the Certified pool at launch. The copy is stored in the region the workspace's deployments run in.

6. 06

   License: the license the hub states, and your attestation that you have the right to run this model as a hosted service under it.

7. 07

   Review, and import.

Whatever revision you enter, the import pins the exact commit or version id it resolves to. Only the weight files are fetched; what it accepts is re-saved and signed with a HIPS key, and workers and devices load only signed copies. Embedding and rerank models are not served for customer models yet: they open once HIPS's data-center workers serve them. The form also shows what an import is held to, as the platform states it: the hubs enabled, whether imports are open, the licenses allowed, the size limits per kind of model, the imports a day with what is left, how many run at once, and the engine releases pinned.

An import's page follows it by itself, names each step with its time, and can cancel it. The refusal codes and what to change for each are on Import and run your model.

Each import that ends Ready makes a new version, with its source, pinned revision and license. Versions are immutable; importing a newer revision makes a new one. A version is checked, then profiled, then given its tariff, and a deployment makes a version runnable.

1. 01

   Model: a version of one of your models that its import left ready, or a catalog model to pin.

2. 02

   Compatibility checks: the import's checks, then engine fit and hardware fit beyond memory: compatible, needs profiling, or unsupported with each reason.

3. 03

   Pool: Auto first and selected, then the two pools side by side: quote, availability, runtime-access conditions, region eligibility. Customer models run in the Certified pool for now.

4. 04

   Fallback: off by default. If you enable it, it must still respect your region and data-use rules.

5. 05

   Policies: the request-data policy of the plan, the weights commitment, the retention statement.

6. 06

   Lane: shared where the model supports usage-based access, or always_on from the Certified pool only.

7. 07

   Review, and Deploy.

The deployment's id goes in the model field of the OpenAI-compatible routes for a language model, or in deployment on the task API.

An always-on allocation keeps a model loaded on Certified-pool capacity, charged by the hour, including time between requests. A GPU being allocated and a model being ready to serve are separate steps. A capacity snapshot is not a reservation: if the allocation cannot be made, the deployment shows Stopped with the reason and nothing is charged. Charges stop when the allocation ends.

• Stop ends the allocation; no allocation charge accrues from then. Start again makes a new allocation.
• Changing the version moves the deployment to another version; requests in flight finish on the version they started with.
• Delete is refused while the deployment is not Stopped. It removes the deployment and, if you choose, the customer model's stored copies. Nothing at the hub changes.
• Stored models and a stopped deployment's weights cost $0.10 per GB-month while they are kept.