> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a bundle

This page describes how to serve a bundle by referencing it from a `ModelDeployment`. It applies both to the bundles that SambaNova provides and to bundles you authored yourself.

To author your own bundle first, see [Create a custom bundle](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/create-a-custom-bundle). To serve a single model without a bundle, see [Deploy a single model](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-a-single-model).

## Prerequisites

Before deploying, complete the quickstart that applies to you:

> **[Quickstart - Hosted](/docs/en/v2.2.6/sambastack/getting-started/hosted)**
>
> System set up for hosted SambaStack

> **[Quickstart - On-prem](/docs/en/v2.2.6/sambastack/getting-started/on-prem)**
>
> System set up for on-prem SambaStack

Additionally:

* Review the [ModelDeployment](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/custom-resources/model-deployment) field reference.
* Confirm the bundle you intend to deploy has passed validation. See [Legalizer results](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/custom-resources/model-bundle#legalizer-results).
* Review [Supported models and bundles](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/supported-models-and-bundles) for the bundles SambaNova provides.

## Deploy the bundle

**1. Discover the available bundles**

List the bundles present in the cluster and note the name of the one you want to deploy:

**Hosted**

```bash theme={}
kubectl get modelbundles
```

**On Premise**

```bash theme={}
kubectl -n <namespace> get modelbundles.sambanova.ai
```

Output:

```text theme={}
mb-gpt120                      17h
mb-gpt120-llama70sd8-llama8    17h
ds-v32-gemma-4-31b             3h
```

Before deploying, confirm the bundle passed validation, as described in [Legalizer results](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/custom-resources/model-bundle#legalizer-results).

**2. Create a ModelDeployment**

Insert the bundle name into `spec.bundle`:

```yaml theme={}
apiVersion: sambanova.ai/v1alpha1
kind: ModelDeployment
metadata:
  name: md-gpt120
spec:
  bundle: mb-gpt120
  groups:
  - minReplicas: 1
    name: default
    qosList:
    - free
  owner: no-reply@sambanova.ai
  secretNames:
  - sambanova-artifact-reader
```

**3. Apply the ModelDeployment**

**Hosted**

```bash theme={}
kubectl apply -f <modeldeployment-file>.yaml
```

**On Premise**

```bash theme={}
kubectl -n <namespace> apply -f <modeldeployment-file>.yaml
```

**4. Monitor deployment status**

**Hosted**

```bash theme={}
kubectl get modeldeployments
kubectl describe modeldeployment <deployment-name>
```

**On Premise**

```bash theme={}
kubectl -n <namespace> get modeldeployments.sambanova.ai
kubectl -n <namespace> describe modeldeployment.sambanova.ai <deployment-name>
```

## Verify the deployment

Pods reporting ready means the serving containers started, not that the model answers requests. Confirm the serving name, then send one inference request.

### Resolve the serving name

The name you send to the API is not the Kubernetes resource name. Three names are involved:

| Name | Where it lives | Used for |
| - | - | - |
| `metadata.name` on a Model | The `Model` resource | `kubectl` commands, and `modelConfigs[].model` references in bundles and deployments |
| `spec.name` on a Model | The `Model` resource | The `model` field of an inference request |
| `spec.aliases` on a Model | The `Model` resource | Additional names that route to the same model |

Read the serving name from the Model you deployed:

**Hosted**

```bash theme={}
kubectl get model <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
```

**On Premise**

```bash theme={}
kubectl -n <namespace> get model.sambanova.ai <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
```

Then confirm the gateway is serving it. Each `id` in the response is a servable name:

```bash theme={}
curl https://<your-api-domain>/v1/models \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY"
```

> [!NOTE]
>
> A model set to `modelSettings.routable: false`, such as a speculative decoding draft model, is deliberately absent from this list and cannot be addressed directly.

### Send a request

Use the serving name in the `model` field:

```bash theme={}
curl https://<your-api-domain>/v1/chat/completions \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Say hi in one sentence."}
    ],
    "max_tokens": 128,
    "stream": false
  }'
```

A successful response returns the serving name in `model` and a populated `choices[0].message`:

```json theme={}
{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Hi there, it's a sunny day today!",
        "role": "assistant"
      }
    }
  ],
  "created": 1770927918,
  "id": "c754b468-05e1-4fe4-8a28-f51ecc74f4f2",
  "model": "gpt-oss-120b",
  "object": "chat.completion",
  "system_fingerprint": "fastcoe",
  "usage": {
    "completion_tokens": 121,
    "prompt_tokens": 91,
    "total_tokens": 212,
    "stop_reason": "stop",
    "time_to_first_token": 0.05286884307861328,
    "total_latency": 0.29257988929748535
  }
}
```

The `usage` object carries additional throughput and latency fields; for the full body, and for the API domain to use in each environment, see [Quickstart - Hosted](/docs/en/v2.2.6/sambastack/getting-started/hosted) or [Quickstart - On-prem](/docs/en/v2.2.6/sambastack/getting-started/on-prem).

> [!WARNING]
>
> A `model not found` error with pods running usually means the request used `metadata.name` instead of `spec.name`. An inference error on a model that loads and lists usually means the checkpoint and the profile disagree on `model_arch`. See [Deployment failures](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting#deployment-failures).

## Switch bundles

A ModelDeployment's bundle reference cannot be changed in place. To serve a different bundle, delete the existing deployment and apply a new one that references the new bundle.

**1. Delete the existing ModelDeployment**

**Hosted**

```bash theme={}
kubectl delete modeldeployment <deployment-name>
```

**On Premise**

```bash theme={}
kubectl -n <namespace> delete modeldeployment <deployment-name>
```

**2. Apply a ModelDeployment for the new bundle**

Author a new ModelDeployment whose `spec.bundle` names the bundle you want to serve, then apply it as described in [Deploy the bundle](#deploy-the-bundle).

```yaml theme={}
apiVersion: sambanova.ai/v1alpha1
kind: ModelDeployment
metadata:
  name: md-qwen3-32b-whisper
spec:
  bundle: mb-qwen3-32b-whisper
  groups:
  - minReplicas: 1
    name: default
    qosList:
    - free
  owner: no-reply@sambanova.ai
  secretNames:
  - sambanova-artifact-reader
```

To request new bundles, contact SambaNova support.

## Deploy multiple bundles

To serve more than one bundle at a time, apply a separate ModelDeployment for each bundle.

> [!WARNING]
>
> SambaStack supports only one bundle per node. When deploying multiple bundles, assign each bundle to separate nodes to avoid resource conflicts. Use `spec.nodeSelector` on each ModelDeployment to control placement.

Verify that the pods reflect the deployed bundles:

**Hosted**

```bash theme={}
kubectl get pods
```

**On Premise**

```bash theme={}
kubectl -n <namespace> get pods
```

## Update or remove a deployment or bundle

**Update a configuration**

**1. Modify the YAML file**

Edit the ModelBundle, ModelProfile, or ModelDeployment YAML file with your changes.

**2. Reapply the configuration**

**Hosted**

```bash theme={}
kubectl apply -f <modified-file>.yaml
```

**On Premise**

```bash theme={}
kubectl -n <namespace> apply -f <modified-file>.yaml
```

The legalizer automatically revalidates the changes. Changing a profile or a model triggers a reconcile of every bundle and deployment that references it.

**Remove a configuration**

**1. Delete the ModelDeployment**

**Hosted**

```bash theme={}
kubectl delete modeldeployment <deployment-name>
```

**On Premise**

```bash theme={}
kubectl -n <namespace> delete modeldeployment <deployment-name>
```

**2. Delete the ModelBundle (if one was used)**

**Hosted**

```bash theme={}
kubectl delete modelbundle <bundle-name>
```

**On Premise**

```bash theme={}
kubectl -n <namespace> delete modelbundle <bundle-name>
```

**3. Delete the ModelProfile (optional)**

Delete a profile only if you authored it for a custom architecture. Profiles provided with the SambaStack installation are shared across models and should be left in place.

**Hosted**

```bash theme={}
kubectl delete modelprofile <profile-name>
```

**On Premise**

```bash theme={}
kubectl -n <namespace> delete modelprofile <profile-name>
```

## Troubleshooting

For the failures the operator reports once it starts creating serving pods, and for the failures the legalizer reports before that, see [Troubleshooting deployments](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting).

## Related documentation

> **[Supported models and bundles](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/supported-models-and-bundles)**
>
> Catalogue of models and bundles available for deployment

> **[ModelDeployment](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/custom-resources/model-deployment)**
>
> Full field reference for the deployment resource

> **[Deploying custom checkpoints](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-custom-checkpoints)**
>
> Deploy your own custom or fine-tuned checkpoints

> **[Migrating bundle configuration](/docs/en/v2.2.6/sambastack/service-administration/model-deployment/deploying-models-and-bundles/sambastack-v2-bundle-migration)**
>
> Migrate bundles built with the deprecated custom resources