> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a single model

To serve one model, pair it with a profile and deploy it directly with a `ModelDeployment`. No bundle is created.

Create a bundle instead when you want to serve several models as one unit, configure speculative decoding, or validate and share a named configuration. See [Create a custom bundle](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/create-a-custom-bundle).

## Prerequisites

Before deploying, complete the quickstart that applies to you:

> **[Quickstart - Hosted](/docs/en/v2.2.6/sambastack/getting-started/hosted)**
>
> System set up for hosted SambaStack

> **[Quickstart - On-prem](/docs/en/v2.2.6/sambastack/getting-started/on-prem)**
>
> System set up for on-prem SambaStack

Additionally:

* Review the [deployment overview](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/overview) for how models, profiles, and deployments fit together.
* Identify the model and profile you want to pair, and confirm they are compatible. See [Identify available models and profiles](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/overview#identify-available-models-and-profiles).
* Check [Supported models and bundles](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/supported-models-and-bundles) for the available model checkpoints.

## Deploy the model

**1. Create the YAML file**

Create a YAML file that declares the model and profile inline under `spec.models`:

```yaml theme={}
apiVersion: sambanova.ai/v1alpha1
kind: ModelDeployment
metadata:
  name: md-gpt120
spec:
  models:
    modelConfigs:
    - model: gpt-oss-120b:1
      profile: gpt-oss-fp8-dyt
  groups:
  - minReplicas: 1
    name: default
    qosList:
    - free
  owner: no-reply@sambanova.ai
  secretNames:
  - sambanova-artifact-reader
  engineConfig:
    startupTimeout: 7200
```

A checkpoint path is not required, because the checkpoint is resolved from the referenced `Model`.

`startupTimeout` is in seconds. The `7200` here gives the engine two hours to download and load the model before its startup probe fails, well above the 45-minute default. See [Startup timeouts](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/custom-resources/model-deployment#startup-timeouts).

**2. Apply the ModelDeployment**

**Hosted**

```bash theme={}
kubectl apply -f <modeldeployment-file>.yaml
```

**On Premise**

```bash theme={}
kubectl -n <namespace> apply -f <modeldeployment-file>.yaml
```

**3. Monitor deployment status**

**Hosted**

```bash theme={}
kubectl get modeldeployments
kubectl describe modeldeployment <deployment-name>
```

**On Premise**

```bash theme={}
kubectl -n <namespace> get modeldeployments.sambanova.ai
kubectl -n <namespace> describe modeldeployment.sambanova.ai <deployment-name>
```

The deployment is complete once all of its serving pods report a ready status.

## Verify the deployment

Pods reporting ready means the serving containers started, not that the model answers requests. Confirm the serving name, then send one inference request.

### Resolve the serving name

The name you send to the API is not the Kubernetes resource name. Three names are involved:

| Name | Where it lives | Used for |
| - | - | - |
| `metadata.name` on a Model | The `Model` resource | `kubectl` commands, and `modelConfigs[].model` references in bundles and deployments |
| `spec.name` on a Model | The `Model` resource | The `model` field of an inference request |
| `spec.aliases` on a Model | The `Model` resource | Additional names that route to the same model |

Read the serving name from the Model you deployed:

**Hosted**

```bash theme={}
kubectl get model <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
```

**On Premise**

```bash theme={}
kubectl -n <namespace> get model.sambanova.ai <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
```

Then confirm the gateway is serving it. Each `id` in the response is a servable name:

```bash theme={}
curl https://<your-api-domain>/v1/models \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY"
```

> [!NOTE]
>
> A model set to `modelSettings.routable: false`, such as a speculative decoding draft model, is deliberately absent from this list and cannot be addressed directly.

### Send a request

Use the serving name in the `model` field:

```bash theme={}
curl https://<your-api-domain>/v1/chat/completions \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Say hi in one sentence."}
    ],
    "max_tokens": 128,
    "stream": false
  }'
```

A successful response returns the serving name in `model` and a populated `choices[0].message`:

```json theme={}
{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Hi there, it's a sunny day today!",
        "role": "assistant"
      }
    }
  ],
  "created": 1770927918,
  "id": "c754b468-05e1-4fe4-8a28-f51ecc74f4f2",
  "model": "gpt-oss-120b",
  "object": "chat.completion",
  "system_fingerprint": "fastcoe",
  "usage": {
    "completion_tokens": 121,
    "prompt_tokens": 91,
    "total_tokens": 212,
    "stop_reason": "stop",
    "time_to_first_token": 0.05286884307861328,
    "total_latency": 0.29257988929748535
  }
}
```

The `usage` object carries additional throughput and latency fields; for the full body, and for the API domain to use in each environment, see [Quickstart - Hosted](/docs/en/v2.2.6/sambastack/getting-started/hosted) or [Quickstart - On-prem](/docs/en/v2.2.6/sambastack/getting-started/on-prem).

> [!WARNING]
>
> A `model not found` error with pods running usually means the request used `metadata.name` instead of `spec.name`. An inference error on a model that loads and lists usually means the checkpoint and the profile disagree on `model_arch`. See [Deployment failures](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting#deployment-failures).

## Update or remove the deployment

To change a deployment's configuration, edit its YAML file and reapply it. The legalizer automatically revalidates the changes.

**Hosted**

```bash theme={}
kubectl apply -f <modified-file>.yaml
```

**On Premise**

```bash theme={}
kubectl -n <namespace> apply -f <modified-file>.yaml
```

To remove the deployment:

**Hosted**

```bash theme={}
kubectl delete modeldeployment <deployment-name>
```

**On Premise**

```bash theme={}
kubectl -n <namespace> delete modeldeployment <deployment-name>
```

## Troubleshooting

A deployment created from an inline `spec.models` specification is legalized in place, so both classes of failure can apply to it. See [Troubleshooting deployments](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting) for [legalizer validation failures](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting#legalizer-validation-failures) and [deployment failures](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting#deployment-failures).

## Related documentation

> **[ModelDeployment](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/custom-resources/model-deployment)**
>
> Full field reference for the deployment resource

> **[Deploy a bundle](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-a-bundle)**
>
> Serve several models together as one unit