> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom bundle deployment

This guide describes how to create custom **BundleTemplates** and **Bundles** to deploy models with specific configurations on SambaStack.

In SambaStack, bundles are the fundamental deployment unit. Rather than deploying individual models, you deploy bundles that group one or more models together with their deployment configurations, including batch sizes and sequence lengths. This approach uses the SambaNova Reconfigurable Dataflow Unit (RDU) to support multiple models and configurations in a single deployment, enabling instant switching between configurations for improved efficiency and flexibility.

<Note>
  This guide covers creating custom bundles. For deploying pre-configured bundles provided by SambaNova, see [Deploying Bundles](/en/v1.2.0/sambastack/service-administration/deploying-model-bundles).
</Note>

## Prerequisites

Before creating custom bundles, complete the following that applies to you:

<CardGroup cols={3}>
  <Card icon="list-check" href="/en/v1.2.0/sambastack/getting-started/hosted" title="Quickstart - Hosted">
    System set up for hosted SambaStack
  </Card>

  <Card icon="list-check" href="/en/v1.2.0/sambastack/getting-started/on-prem" title="Quickstart - On-prem">
    System set up for On-prem Sambastack
  </Card>
</CardGroup>

Additionally, review the following documentation:

* [Model Bundle Deployment](/en/v1.2.0/sambastack/service-administration/deploying-model-bundles) - Bundle deployment concepts and workflows
* [Supported Models and Bundles](/en/v1.2.0/sambastack/service-administration/supported-models-and-bundles) - Available model checkpoints
* [Speculative Decoding Deployment Guidelines](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding) - Required if configuring speculative decoding pairs

### Terminology

| Term                     | Definition                                                                                               |
| ------------------------ | -------------------------------------------------------------------------------------------------------- |
| **RDU**                  | Reconfigurable Dataflow Unit - SambaNova's proprietary processor architecture                            |
| **PEF**                  | Processor Executable Format - Compiled model binaries that run on RDUs                                   |
| **Expert**               | A sequence length profile configuration (for example, 8k, 16k, 32k) within a model                       |
| **Speculative Decoding** | An optimization technique using a smaller draft model to accelerate inference from a larger target model |
| **Legalizer**            | A validation process that verifies a bundle fits within RDU memory constraints                           |
| **CR (Custom Resource)** | A Kubernetes extension that defines custom resource types such as `Pef`, `Bundle`, and `BundleTemplate`  |

## Concepts

### Bundle architecture

For an overview of core bundle concepts including what bundles and bundle templates are see [Deploying Model Bundles](/en/v1.2.0/sambastack/service-administration/deploying-model-bundles). This section covers the specific three-tier structure used when creating **custom** bundles:

1. **BundleTemplate** - Defines *how* models can be run: available sequence length profiles, batch sizes, and PEF mappings. Also defines target and draft model relationships for speculative decoding pairs.
2. **Bundle** - Binds a template to *specific checkpoints* in storage, making it ready for deployment
3. **BundleDeployment** - Instantiates one or more replicas of a bundle on the cluster

This separation allows you to:

* Reuse templates across multiple checkpoints, including custom checkpoints for fine-tuned models
* Bundle different checkpoints of the same underlying model while sharing the same template
* Deploy the same bundle configuration with different replica counts
* Update checkpoints without modifying deployment configurations

### BundleTemplate structure

A BundleTemplate defines the deployment capabilities for one or more models. The following example shows a multi-model template with speculative decoding configuration:

```yaml theme={null}
apiVersion: sambanova.ai/v1alpha1
kind: BundleTemplate
metadata:
  name: bt-gpt120-llama70sd8-llama8
spec:
  models:
    gpt-oss-120b:
      experts:
        8k:
          configs:
          - pef: gpt-oss-fp8-ss8192-bs2:1
        32k:
          configs:
          - pef: gpt-oss-fp8-ss32768-bs2:1
    Meta-Llama-3.3-70B-Instruct:
      experts:
        4k:
          configs:
          - pef: llama-3p1-70b-ss4096-bs4-sd5:3
            spec_decoding:
              draft_model: Meta-Llama-3.1-8B-Instruct
          - pef: llama-3p1-70b-ss4096-bs32-sd5:2
        8k:
          configs:
          - pef: llama-3p1-70b-ss8192-bs1-sd5:1
          - pef: llama-3p1-70b-ss8192-bs8-sd5:2
          default_config_values:
            spec_decoding:
              draft_model: Meta-Llama-3.1-8B-Instruct
        128k:
          configs:
          - pef: llama-3p1-70b-ss131072-bs1-sd5:2
    Meta-Llama-3.1-8B-Instruct:
      experts:
        4k:
          configs:
          - pef: llama-3p1-8b-ss4096-bs4:1
        8k:
          configs:
          - pef: llama-3p1-8b-ss8192-bs1:1
          - pef: llama-3p1-8b-ss8192-bs8:1
  owner: no-reply@sambanova.ai
  secretNames:
  - sambanova-artifact-reader
  usePefCRs: true
```

The example above includes configurations for three models:

* **gpt-oss-120b**: 2 configs
* **Meta-Llama-3.3-70B-Instruct**: 5 configs, 3 of which use speculative decoding with Meta-Llama-3.1-8B-Instruct as the draft model
* **Meta-Llama-3.1-8B-Instruct**: 3 configs

For more details on the speculative decoding fields, see the [Speculative Decoding Deployment Guidelines](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding).

**BundleTemplate Top-Level Fields**

| Field              | Required | Description                                                                                             |
| ------------------ | -------- | ------------------------------------------------------------------------------------------------------- |
| `spec.models`      | Yes      | Defines the models and their expert configurations. See [Models and Experts](#models-and-experts).      |
| `spec.owner`       | Yes      | Email address of the bundle template owner for tracking and notifications.                              |
| `spec.secretNames` | Yes      | List of Kubernetes secrets used to access artifacts. Must match secrets configured in your environment. |
| `spec.usePefCRs`   | Yes      | Set to `true` to use PEF custom resources for deployment.                                               |

**Models and Experts**

Each model in a BundleTemplate contains one or more **experts**, which represent sequence length profiles. Common profiles include:

* `8k`, `16k`, `32k`, `64k`, `128k` - Fixed sequence length configurations
* `default` - Standard configuration when no specific length is required

```yaml theme={null}
spec:
  models:
    Meta-Llama-3.1-8B-Instruct:
      experts:
        128k:
          configs:
            - <config>
        64k:
          configs:
            - <config>
        32k:
          configs:
            - <config>
        16k:
          configs:
            - <config>
        8k:
          configs:
            - <config>
        default:
          configs:
            - <config>
```

**Expert Configuration Parameters**

Each expert contains one or more configurations with the following parameters:

| Parameter       | Required | Description                                                                                                                                                                                                               |
| --------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pef`           | Yes      | Reference to a PEF custom resource in format `<pef-name>:<version>`. Use version `1` unless a higher version is confirmed via `kubectl describe pef`. The `<pef-name>` includes the batch size after the `bs` characters. |
| `spec_decoding` | No       | Speculative decoding configuration. **Only specify for target models, not draft models.**                                                                                                                                 |

<Accordion title="Speculative decoding parameters">
  Parameters within `spec_decoding` (target models only):

  | Parameter      | Description                                                                                                                                                           |
  | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | `draft_model`  | Name of the draft model in the same BundleTemplate                                                                                                                    |
  | `draft_expert` | Expert profile of the draft model to use. Should match the sequence length of the target model expert (for example, use a 16k draft expert with a 16k target expert). |

  For detailed guidance, see [Speculative Decoding Deployment Guidelines](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding).
</Accordion>

### Bundle structure

A Bundle binds a BundleTemplate to specific checkpoints. The following sections explain each part of the Bundle manifest.

**Resource Identity**

Define the Bundle name and resource type:

```yaml theme={null}
apiVersion: sambanova.ai/v1alpha1
kind: Bundle
metadata:
  name: b-70b-3dot3-ss-16-32k-bs-4
```

* `metadata.name` - The **Bundle name** used to reference this bundle in deployments
* `apiVersion` and `kind` - Keep these values the same for all bundles

**Checkpoints**

Define the model checkpoints to use:

```yaml theme={null}
spec:
  checkpoints:
    GPT_OSS_120B_CKPT:
      source: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/checkpoint
      toolSupport: true
    META_LLAMA_3_3_70B_INSTRUCT_CKPT:
      source: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/checkpoint
      toolSupport: true
    META_LLAMA_3_1_8B_INSTRUCT_CKPT:
      source: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/checkpoint
      toolSupport: true
```

| Field         | Description                                                                                                                                                                       |
| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `source`      | GCS path pointing to the model checkpoint. Find available checkpoints in [Model and Bundle Directory](/en/v1.2.0/sambastack/service-administration/supported-models-and-bundles). |
| `toolSupport` | Boolean flag indicating whether this checkpoint is compatible with tools and function-calling (if supported by the product).                                                      |

**Models**

Map model names to checkpoints and templates:

```yaml theme={null}
spec:
  models:
    gpt-oss-120b:
      checkpoint: GPT_OSS_120B_CKPT
      template: gpt-oss-120b
    Meta-Llama-3.3-70B-Instruct:
      checkpoint: META_LLAMA_3_3_70B_INSTRUCT_CKPT
      template: Meta-Llama-3.3-70B-Instruct
    Meta-Llama-3.1-8B-Instruct:
      checkpoint: META_LLAMA_3_1_8B_INSTRUCT_CKPT
      template: Meta-Llama-3.1-8B-Instruct
```

| Field                                                      | Description                                                                                                                                                        |
| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `<model-key>` (for example, `Meta-Llama-3.3-70B-Instruct`) | The **API model name** that users will send inference requests to. Must match a name in the BundleTemplate's `spec.models` section.                                |
| `checkpoint`                                               | The checkpoint alias (from `spec.checkpoints`) this model should use.                                                                                              |
| `template`                                                 | The model template in the BundleTemplate's `spec.models` to use. This value **must exactly match** a model name defined under `spec.models` in the BundleTemplate. |

**Template and Secrets**

Connect the Bundle to its BundleTemplate and credentials:

```yaml theme={null}
spec:
  template: bt-gpt120-llama70sd8-llama8
  secretNames:
  - sambanova-artifact-reader
```

| Field         | Description                                                                                                                               |
| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `template`    | References the BundleTemplate by its `metadata.name`. This connects the Bundle to the deployment configurations defined in that template. |
| `secretNames` | Credentials used to read checkpoints from GCS. Must match the secrets configured in your environment.                                     |

**Complete Bundle Example**

The following example shows a complete multi-model Bundle:

```yaml theme={null}
apiVersion: sambanova.ai/v1alpha1
kind: Bundle
metadata:
  name: b-gpt120-llama70sd8-llama8
spec:
  checkpoints:
    GPT_OSS_120B_CKPT:
      source: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/checkpoint
      toolSupport: true
    META_LLAMA_3_3_70B_INSTRUCT_CKPT:
      source: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/checkpoint
      toolSupport: true
    META_LLAMA_3_1_8B_INSTRUCT_CKPT:
      source: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/checkpoint
      toolSupport: true
  models:
    gpt-oss-120b:
      checkpoint: GPT_OSS_120B_CKPT
      template: gpt-oss-120b
    Meta-Llama-3.3-70B-Instruct:
      checkpoint: META_LLAMA_3_3_70B_INSTRUCT_CKPT
      template: Meta-Llama-3.3-70B-Instruct
    Meta-Llama-3.1-8B-Instruct:
      checkpoint: META_LLAMA_3_1_8B_INSTRUCT_CKPT
      template: Meta-Llama-3.1-8B-Instruct
  secretNames:
  - sambanova-artifact-reader
  template: bt-gpt120-llama70sd8-llama8
```

The paths to checkpoints hosted by SambaNova will be provided to you by your SambaNova contact. If you have hosted your own checkpoints, you can include those paths in the `source` fields above.

### BundleDeployment structure

A BundleDeployment instantiates a bundle on the cluster. For detailed deployment information, see [Quickstart - Hosted](/en/v1.2.0/sambastack/getting-started/hosted) or [Quickstart - On-prem](/en/v1.2.0/sambastack/getting-started/on-prem).

```yaml theme={null}
apiVersion: sambanova.ai/v1alpha1
kind: BundleDeployment
metadata:
  name: bd-gpt120-llama70sd8-llama8
spec:
  bundle: b-gpt120-llama70sd8-llama8
  groups:
  - minReplicas: 1
    name: default
    qosList:
    - free
  owner: no-reply@sambanova.ai
  secretNames:
  - sambanova-artifact-reader
```

| Field                       | Description                                                                              |
| --------------------------- | ---------------------------------------------------------------------------------------- |
| `spec.bundle`               | Name of the Bundle to deploy                                                             |
| `spec.groups[].name`        | Name identifier for the deployment group                                                 |
| `spec.groups[].minReplicas` | Minimum number of bundle replicas to maintain                                            |
| `spec.groups[].qosList`     | Quality of service classes for request prioritization                                    |
| `spec.owner`                | Email address of the deployment owner for tracking and notifications                     |
| `spec.secretNames`          | Credentials used to access artifacts. Must match secrets configured in your environment. |

### PEF and checkpoint lifecycle status

SambaStack assigns a `pef_status` field to PEF CR versions and a `checkpoint_status` field to model CR checkpoint versions to indicate their support lifecycle. Understanding these statuses helps you make informed decisions when selecting PEF or checkpoint versions for custom bundles.

**PEF and checkpoint version status values**

Each version entry in a PEF CR includes a `pef_status` field. Checkpoint CR versions use `checkpoint_status`. Both share the same set of values:

| Status       | Description                                                                                                                                                                                            |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `preview`    | Not fully tested or supported. May have unknown reliability or performance issues, or limited functionality (for example, partial function calling support). Not recommended for production workloads. |
| `stable`     | Fully supported and tested.                                                                                                                                                                            |
| `deprecated` | Has known reliability or performance issues. Still available for a limited transition period (up to 3 months from the deprecation announcement) to allow migration to a stable version.                |
| `removed`    | No longer usable. The version entry is retained in the PEF CR or model CR for traceability and auditability, but the path may no longer exist, causing deployment to fail if referenced.               |

**Example PEF CR versions with status**

```yaml theme={null}
versions:
  '1':
    source: gs://ext-sambastack-artifacts-prod-0/path/to/pef_v1.pef
    pef_status: deprecated
  '2':
    source: gs://ext-sambastack-artifacts-prod-0/path/to/pef_v2.pef
    pef_status: stable
```

To check version statuses, run `kubectl describe pef <pef-name>` or `kubectl describe model <model-name>` and review the `pef_status` or `checkpoint_status` field in the `Versions` section.

<Tip>
  The following procedures describe the step-by-step workflow for creating and deploying custom bundles using the concepts and structures described above.
</Tip>

## Procedures

### Identify available PEFs

Before creating a BundleTemplate, identify the PEF resources available for your model.

<Steps>
  <Step title="List available PEFs">
    List PEFs matching your model and sequence length requirements:

    <Tabs>
      <Tab title="Hosted">
        ```bash theme={null}
        kubectl get pefs | grep <model-pattern>
        ```

        Example:

        ```bash theme={null}
        kubectl get pefs | grep llama-3p1-70b-ss4096
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={null}
        kubectl -n <namespace> get pefs.sambanova.ai | grep <model-pattern>
        ```

        Example:

        ```bash theme={null}
        kubectl -n <namespace> get pefs.sambanova.ai | grep llama-3p1-70b-ss4096
        ```
      </Tab>
    </Tabs>

    Output:

    ```text theme={null}
    llama-3p1-70b-ss4096-bs1-sd9     17h
    llama-3p1-70b-ss4096-bs16-sd5    17h
    llama-3p1-70b-ss4096-bs2-sd5     17h
    llama-3p1-70b-ss4096-bs32-sd5    17h
    llama-3p1-70b-ss4096-bs4-sd5     17h
    llama-3p1-70b-ss4096-bs8-sd5     17h
    ```
  </Step>

  <Step title="View PEF details">
    View PEF details to understand supported configurations and check for higher versions:

    <Tabs>
      <Tab title="Hosted">
        ```bash theme={null}
        kubectl describe pef <pef-name>
        ```

        Example output:

        ```text theme={null}
        $ kubectl describe pef deepseek-ss131072-bs1
        Name:         deepseek-ss131072-bs1
        ...
        Spec:
          copy_pef:                gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/copy_pef
          copy_pef_name_override:  COPY_PEF_DEEPSEEK_R1_128K_PEF_BS1
          Metadata:
            batch_size:             1
            is_prompt_caching:      false
            job_type:               infer
            max_completion_tokens:  131072
            max_seq_length:         131072
            num_rdus:               16
            rdu_arch:               SN40L-16
            seq_lengths:
              65536
              131072
          model_arch:         deepseek
          pef_name_override:  DEEPSEEK_R1_128K_PEF_BS1
          Versions:
            1:
              cached_path:  gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/pef_v1.pef
              pef_status:   deprecated
              Source:       gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/pef_v1.pef
            2:
              cached_path:  gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/pef_v2.pef
              pef_status:   stable
              Source:       gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/pef_v2.pef
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={null}
        kubectl -n <namespace> describe pef.sambanova.ai <pef-name>
        ```

        Example output:

        ```text theme={null}
        $ kubectl -n <namespace> describe pef.sambanova.ai deepseek-ss131072-bs1
        Name:         deepseek-ss131072-bs1
        ...
        Spec:
          copy_pef:                gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/copy_pef
          copy_pef_name_override:  COPY_PEF_DEEPSEEK_R1_128K_PEF_BS1
          Metadata:
            batch_size:             1
            is_prompt_caching:      false
            job_type:               infer
            max_completion_tokens:  131072
            max_seq_length:         131072
            num_rdus:               16
            rdu_arch:               SN40L-16
            seq_lengths:
              65536
              131072
          model_arch:         deepseek
          pef_name_override:  DEEPSEEK_R1_128K_PEF_BS1
          Versions:
            1:
              cached_path:  gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/pef_v1.pef
              pef_status:   deprecated
              Source:       gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/pef_v1.pef
            2:
              cached_path:  gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/pef_v2.pef
              pef_status:   stable
              Source:       gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/pef_v2.pef
        ```
      </Tab>
    </Tabs>

    Review the `Spec.Metadata` section for:

    * `batch_size` - Supported batch size
    * `max_seq_length` - Maximum sequence length
    * `num_rdus` - Required RDU count
    * `rdu_arch` - Required RDU architecture
    * `seq_lengths` - Supported sequence lengths

    Check the `Versions` section to determine if a higher PEF version is available and to review each version's `pef_status` before referencing it in a BundleTemplate.
  </Step>
</Steps>

<Tip>
  Use version `1` in your PEF references (for example, `llama-3p1-70b-ss16384-bs4-sd5:1`) unless `kubectl describe pef` confirms a higher version is available.
</Tip>

### Create a BundleTemplate

<Steps>
  <Step title="Create the YAML file">
    Create a YAML file for your BundleTemplate. For a single-model template:

    ```yaml theme={null}
    apiVersion: sambanova.ai/v1alpha1
    kind: BundleTemplate
    metadata:
      name: bt-gpt120
    spec:
      models:
        gpt-oss-120b:
          experts:
            8k:
              configs:
              - pef: gpt-oss-fp8-ss8192-bs2:1
            32k:
              configs:
              - pef: gpt-oss-fp8-ss32768-bs2:1
            64k:
              configs:
              - pef: gpt-oss-fp8-ss65536-bs2:1
            128k:
              configs:
              - pef: gpt-oss-fp8-ss131072-bs2:1
      owner: no-reply@sambanova.ai
      secretNames:
      - sambanova-artifact-reader
      usePefCRs: true
    ```

    For multi-model templates with speculative decoding, see the [BundleTemplate Structure](#bundletemplate-structure) example.
  </Step>

  <Step title="Apply the BundleTemplate">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={null}
        kubectl apply -f <bundletemplate-file>.yaml
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={null}
        kubectl -n <namespace> apply -f <bundletemplate-file>.yaml
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Verify creation">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={null}
        kubectl get bundletemplates
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={null}
        kubectl -n <namespace> get bundletemplates.sambanova.ai <bundle-name>
        ```
      </Tab>
    </Tabs>
  </Step>
</Steps>

<Tip>
  Including multiple batch sizes for each expert allows the inference engine to select the smallest and fastest configuration based on current workload.
</Tip>

### Create a bundle

<Steps>
  <Step title="Create the YAML file">
    Create a YAML file for your Bundle:

    ```yaml theme={null}
    apiVersion: sambanova.ai/v1alpha1
    kind: Bundle
    metadata:
      name: b-gpt120
    spec:
      checkpoints:
        GPT_OSS_120B_CKPT:
          source: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/checkpoint
          toolSupport: true
      models:
        gpt-oss-120b:
          checkpoint: GPT_OSS_120B_CKPT
          template: gpt-oss-120b
      secretNames:
      - sambanova-artifact-reader
      template: bt-gpt120
    ```

    For multi-model bundles, see the [Complete Bundle Example](#complete-bundle-example).
  </Step>

  <Step title="Apply the Bundle">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={null}
        kubectl apply -f <bundle-file>.yaml
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={null}
        kubectl -n <namespace> apply -f <bundle-file>.yaml
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Verify legalizer validation">
    The legalizer automatically runs when you apply the bundle and validates whether the bundle fits in RDU memory.

    <Tabs>
      <Tab title="Hosted">
        ```bash theme={null}
        kubectl describe bundle <bundle-name>
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={null}
        kubectl -n <namespace> describe bundle.sambanova.ai <bundle-name>
        ```
      </Tab>
    </Tabs>

    <Tabs>
      <Tab title="Successful validation">
        ```text theme={null}
        Status:
          Conditions:
            Last Transition Time:  2025-12-22T21:11:05.689262+00:00
            Message:               Bundle is Valid
            Observed Generation:   1
            Reason:                ValidationSucceeded
            Status:                True
            Type:                  Valid
        ```
      </Tab>

      <Tab title="Failed validation">
        ```text theme={null}
        Status:
          Conditions:
            Last Transition Time:  2025-12-22T21:11:54.975311+00:00
            Message:               <error-details>
            Reason:                ValidationFailed
            Status:                False
            Type:                  Valid
        ```

        The `Message` field contains error details, including legalizer errors if any.
      </Tab>
    </Tabs>
  </Step>
</Steps>

<Warning>
  Do not proceed to deployment until the bundle shows `ValidationSucceeded`.
</Warning>

### Deploy the bundle

<Steps>
  <Step title="Create a BundleDeployment">
    ```yaml theme={null}
    apiVersion: sambanova.ai/v1alpha1
    kind: BundleDeployment
    metadata:
      name: bd-gpt120
    spec:
      bundle: b-gpt120
      groups:
      - minReplicas: 1
        name: default
        qosList:
        - free
      owner: no-reply@sambanova.ai
      secretNames:
      - sambanova-artifact-reader
    ```
  </Step>

  <Step title="Apply the BundleDeployment">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={null}
        kubectl apply -f <bundledeployment-file>.yaml
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={null}
        kubectl -n <namespace> apply -f <bundledeployment-file>.yaml
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Monitor deployment status">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={null}
        kubectl get bundledeployments
        kubectl describe bundledeployment <deployment-name>
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={null}
        kubectl -n <namespace> get bundledeployments.sambanova.ai
        kubectl -n <namespace> describe bundledeployment.sambanova.ai <deployment-name>
        ```
      </Tab>
    </Tabs>
  </Step>
</Steps>

### Update or remove a bundle/BundleTemplate

<Tabs>
  <Tab title="Update a bundle">
    <Steps>
      <Step title="Modify the YAML file">
        Edit the Bundle or BundleTemplate YAML file with your changes.
      </Step>

      <Step title="Reapply the configuration">
        <Tabs>
          <Tab title="Hosted">
            ```bash theme={null}
            kubectl apply -f <modified-file>.yaml
            ```
          </Tab>

          <Tab title="On Premise">
            ```bash theme={null}
            kubectl -n <namespace> apply -f <modified-file>.yaml
            ```
          </Tab>
        </Tabs>

        The legalizer automatically revalidates the changes.
      </Step>
    </Steps>
  </Tab>

  <Tab title="Remove a bundle">
    <Steps>
      <Step title="Delete the BundleDeployment">
        <Tabs>
          <Tab title="Hosted">
            ```bash theme={null}
            kubectl delete bundledeployment <deployment-name>
            ```
          </Tab>

          <Tab title="On Premise">
            ```bash theme={null}
            kubectl -n <namespace> delete bundledeployment <deployment-name>
            ```
          </Tab>
        </Tabs>
      </Step>

      <Step title="Delete the Bundle">
        <Tabs>
          <Tab title="Hosted">
            ```bash theme={null}
            kubectl delete bundle <bundle-name>
            ```
          </Tab>

          <Tab title="On Premise">
            ```bash theme={null}
            kubectl -n <namespace> delete bundle <bundle-name>
            ```
          </Tab>
        </Tabs>
      </Step>

      <Step title="Delete the BundleTemplate (optional)">
        <Tabs>
          <Tab title="Hosted">
            ```bash theme={null}
            kubectl delete bundletemplate <template-name>
            ```
          </Tab>

          <Tab title="On Premise">
            ```bash theme={null}
            kubectl -n <namespace> delete bundletemplate <template-name>
            ```
          </Tab>
        </Tabs>
      </Step>
    </Steps>
  </Tab>
</Tabs>

## Troubleshooting

### Legalizer validation failures

| Error Pattern                                                     | Cause                                                                 | Resolution                                                           |
| ----------------------------------------------------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------- |
| `PEF pef1 and pef2 are not checkpoint compatible (checkpoint #0)` | PEFs with the same `ckpt_sharing_uuid` cannot share checkpoint memory | Assign different `ckpt_sharing_uuid` values to the incompatible PEFs |
| `Bundle exceeds memory constraints`                               | Combined PEF and checkpoint size exceeds RDU memory                   | Reduce the number of experts or batch sizes in the template          |
| `PEF not found: <pef-name>`                                       | Referenced PEF does not exist                                         | Verify PEF name with `kubectl get pefs`                              |

### Deployment failures

| Symptom                      | Possible Cause                          | Resolution                                                              |
| ---------------------------- | --------------------------------------- | ----------------------------------------------------------------------- |
| Deployment stuck in pending  | Insufficient RDU resources              | Check cluster capacity; reduce `minReplicas`                            |
| Checkpoint download fails    | Invalid GCS path or missing credentials | Verify `source` path; confirm `sambanova-artifact-reader` secret exists |
| Model not accessible via API | Model name mismatch                     | Verify `spec.models.<name>` matches expected API endpoint               |

## Related documentation

<CardGroup cols={2}>
  <Card icon="rocket" href="/en/v1.2.0/sambastack/service-administration/deploying-model-bundles" title="Model Deployment">
    Bundle deployment concepts and workflows
  </Card>

  <Card icon="cube" href="/en/v1.2.0/sambastack/service-administration/supported-models-and-bundles" title="Supported Models and Bundles">
    Catalogue of models and bundles available for deployment
  </Card>

  <Card icon="compress" href="/en/v1.2.0/sambastack/service-administration/deploy-custom-checkpoints" title="Custom checkpoint deployment">
    Deploy your own custom or fine-tuned checkpoints
  </Card>

  <Card icon="arrows-rotate" href="/en/v1.2.0/sambastack/service-administration/checkpoint-conversion-tool" title="Checkpoint Conversion Tool">
    Convert Checkpoints to Compatible formats
  </Card>
</CardGroup>