> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# ModelDeployment

A ModelDeployment instantiates one or more replicas on the cluster and creates the serving pods behind a routable inference endpoint. It takes its models either from a [ModelBundle](https://docs.sambanova.ai/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-bundle) or from an inline model and profile pair.

`ModelDeployment` resources use `apiVersion: sambanova.ai/v1alpha1`.

## Structure

Set **exactly one** model source:

| Deployment path | Field | Description |
| - | - | - |
| Direct deployment | `spec.models` | Declares a model and profile inline. The specification is validated and legalized in place, and no bundle is used. |
| Bundle deployment | `spec.bundle` | References an existing `ModelBundle` by name. The bundle is trusted once it has been validated. |

```yaml theme={}
apiVersion: sambanova.ai/v1alpha1
kind: ModelDeployment
metadata:
  name: md-gpt120-llama70sd8-llama8
spec:
  bundle: mb-gpt120-llama70sd8-llama8
  groups:
  - minReplicas: 1
    name: default
    qosList:
    - free
  owner: no-reply@sambanova.ai
  secretNames:
  - sambanova-artifact-reader
  engineConfig:
    startupTimeout: 7200   # seconds; see Startup timeouts below
```

| Field | Required | Description |
| - | - | - |
| `spec.bundle` | Conditional | Name of the `ModelBundle` to deploy. Set exactly one of `spec.bundle` or `spec.models`. |
| `spec.models` | Conditional | Inline definition of the models and profiles to deploy, using the same `modelConfigs` structure as a `ModelBundle`. Set exactly one of `spec.bundle` or `spec.models`. |
| `spec.groups[].name` | Yes | Name identifier for the deployment group. Use `default` unless you are defining more than one group. |
| `spec.groups[].minReplicas` | Yes | Minimum number of replicas to maintain. |
| `spec.groups[].qosList` | Yes | QoS tags that determine which service tiers can route requests to this group, for example `web`, `free`, or `pro`. The tags available to you are the ones defined by the service tiers configured on your system. See [Service tiers](https://docs.sambanova.ai/docs/en/sambastack/service-administration/platform-admin/usergroup-management#service-tiers). |
| `spec.owner` | Yes | Email address of the deployment owner for tracking and notifications. |
| `spec.secretNames` | Yes | Credentials used to access artifacts. Must match secrets configured in your environment. |
| `spec.engineConfig` | No | Configures the inference engine container: image, pull policy, environment variables, and its startup probe. See [Startup timeouts](#startup-timeouts). |
| `spec.cacheConfig` | No | Configures the artifact cache container, including its startup probe. See [Startup timeouts](#startup-timeouts). |
| `spec.tokenizerConfig` | No | Configures the tokenizer container, including its startup probe. See [Startup timeouts](#startup-timeouts). |
| `spec.storage`, `spec.storageClass` | No | Storage configuration for the deployment. |
| `spec.nodeSelector`, `spec.tolerations` | No | Default scheduling controls for the deployment. |
| `spec.skip_legalizer` | No | Skips the legalizer when deploying an inline `spec.models` specification. Spelled in snake case in the API, unlike the camel case fields above. |

## Startup timeouts

A deployment brings up three containers, each guarded by a Kubernetes startup probe. `startupTimeout` sets how long that probe keeps waiting before the container is considered failed, and `startupInterval` sets how often it polls. **Both are in seconds.**

| Field | Guards | Default timeout | Default interval |
| - | - | - | - |
| `engineConfig.startupTimeout` | Inference engine container | `2700` (45 minutes) | `10` |
| `cacheConfig.startupTimeout` | Artifact cache container | `21600` (6 hours) | `1` |
| `tokenizerConfig.startupTimeout` | Tokenizer container | `60` (1 minute) | `1` |

The timeout divided by the interval becomes the probe's failure threshold, so raising the timeout extends the window rather than changing how often the container is checked.

### When to raise engineConfig.startupTimeout

The engine container is ready only once every PEF and checkpoint in the deployment has been downloaded and loaded onto the RDUs. How long that takes scales with the number and size of the configurations being served, so a large bundle can outlast the 45-minute default and be killed mid-load.

Setting `startupTimeout: 7200` gives the engine two hours, which is enough for the large model bundles SambaNova provides. This is why the examples throughout these pages use that value:

```yaml theme={}
spec:
  engineConfig:
    startupTimeout: 7200   # seconds; 2 hours, enough for large bundles to load
```

Raising the timeout costs nothing when startup is quick — the probe succeeds as soon as the container is ready and does not wait out the remaining time. A deployment stuck in a restart loop while loading a large bundle is the signal that the value is too low.

## Related resources

> **[Deploy a single model](https://docs.sambanova.ai/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-a-single-model)**
>
> Deploy with an inline `spec.models` specification

> **[Deploy a bundle](https://docs.sambanova.ai/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-a-bundle)**
>
> Deploy with a `spec.bundle` reference