> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Supported models and bundles

SambaStack supports a variety of models that can be deployed to both on-prem and hosted environments. Contact your system administrator to determine which models are available on your deployment.

## Deployment options

When deploying models in SambaStack, administrators can select from various context length and batch size combinations.

* Smaller batch sizes provide higher token throughput (tokens/second).
* Larger batch sizes provide better concurrency for multiple users.

SambaStack supports two deployment configurations – high-interactivity and high-throughput. See [High-throughput deployment](https://docs.sambanova.ai/docs/en/v1.0.57/sambastack/service-administration/deployment-configurations.md) for when to use each.

## Finding models on your cluster

You can run the following command to discover available models in your cluster:

```shellscript theme={}
kubectl -n <namespace> get models
```

## All supported models

Search or filter to find a model, then select it to jump to its full configuration.

All providers

All modalities

— select one to jump to its full
configuration below

## Meta

### Meta-Llama-3.3-70B-Instruct

<details>
  <summary>
    Meta

    Meta-Llama-3.3-70B-Instruct

    Task agent, Tool-calling agent, Text to SQL/Cipher
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)
</details>

### Meta-Llama-3.1-8B-Instruct

<details>
  <summary>
    Meta

    Meta-Llama-3.1-8B-Instruct

    Gateway agent, Validation agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)
</details>

### Meta-Llama-3.1-405B-Instruct

<details>
  <summary>
    Meta

    Meta-Llama-3.1-405B-Instruct

    Task agent, Tool-calling agent, Code generation
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct)
</details>

### Llama-4-Maverick-17B-128E-Instruct

<details>
  <summary>
    Meta

    Llama-4-Maverick-17B-128E-Instruct

    Image understanding, Task agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct)
</details>

## MiniMax

### MiniMax-M2.5

<details>
  <summary>
    MiniMax

    MiniMax-M2.5

    Coding agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)
</details>

## DeepSeek

### DeepSeek-R1-0528

<details>
  <summary>
    DeepSeek

    DeepSeek-R1-0528

    Complex reasoning
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-R1)
</details>

### DeepSeek-R1-Distill-Llama-70B

<details>
  <summary>
    DeepSeek

    DeepSeek-R1-Distill-Llama-70B

    Complex reasoning
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B)
</details>

### DeepSeek-V3-0324

<details>
  <summary>
    DeepSeek

    DeepSeek-V3-0324

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324)
</details>

### DeepSeek-V3.1

<details>
  <summary>
    DeepSeek

    DeepSeek-V3.1

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-V3.1)
</details>

### DeepSeek-V3.2

<details>
  <summary>
    DeepSeek

    DeepSeek-V3.2

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-V3.2)
</details>

## OpenAI

### gpt-oss-120b

<details>
  <summary>
    OpenAI

    gpt-oss-120b

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/openai/gpt-oss-120b)
</details>

### Whisper-Large-v3

<details>
  <summary>
    OpenAI

    Whisper-Large-v3

    Automatic speech recognition (ASR), Audio transcription
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/openai/whisper-large-v3)
</details>

## Google

### gemma-3-27b-it

<details>
  <summary>
    Google

    gemma-3-27b-it

    Image understanding, Task agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/google/gemma-3-27b-it)
</details>

### gemma-3-12b-it

<details>
  <summary>
    Google

    gemma-3-12b-it

    Image understanding, Task agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/google/gemma-3-12b-it)
</details>

## Alibaba Cloud

### Qwen3-235B-A22B-Instruct-2507

<details>
  <summary>
    Alibaba Cloud

    Qwen3-235B-A22B-Instruct-2507

    Agentic planner, Multilingual instruction following
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507)
</details>

### Qwen3-235B

<details>
  <summary>
    Alibaba Cloud

    Qwen3-235B

    Agentic planner, Multilingual instruction following
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507)
</details>

### Qwen3-32B

<details>
  <summary>
    Alibaba Cloud

    Qwen3-32B

    Task agent, Multilingual instruction following
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/Qwen/Qwen3-32B)
</details>

## Tokyotech-llm

### Llama-3.3-Swallow-70B-Instruct-v0.4

<details>
  <summary>
    Tokyotech-llm

    Llama-3.3-Swallow-70B-Instruct-v0.4

    Japanese instruction following, Task agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4)
</details>

## Other

### E5-Mistral-7B-Instruct

<details>
  <summary>
    Other

    E5-Mistral-7B-Instruct

    Vector storage and retrieval (RAG)
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/intfloat/e5-mistral-7b-instruct)
</details>

## Recommended model bundles

In SambaStack, models are not deployed individually; they are deployed as bundles. A bundle is a packaged deployment that groups one or more models together with their associated configurations, such as batch size and sequence length.

For example, deploying the `Meta‑Llama‑3.3‑70B` model with a batch size of 4 and a sequence length of 16K tokens constitutes a single configuration. A bundle, however, can contain multiple such configurations, either for the same model or for different models.

SambaNova’s RDU technology enables several models and configurations to be loaded simultaneously in a single deployment. This allows you to switch instantly between models and between batch‑/sequence‑size profiles as needed. In contrast to traditional GPU systems, where deployments are typically single‑model and static, SambaStack supports multi‑model, multi‑configuration bundles. This approach delivers higher efficiency, greater flexibility, and increased throughput while preserving low latency.

You can run the following command to discover available bundles in your cluster:

```shellscript theme={}
kubectl -n <namespace> get bundles
```

The table below lists the recommended bundle templates for the models currently available in SambaStack. Each entry pairs a model with its recommended deployment bundle.

> [!NOTE]
>
> If the bundles listed below do not satisfy your inference requirements, you can create [custom bundles](https://docs.sambanova.ai/docs/en/v1.0.57/sambastack/service-administration/custom-bundle-deployment.md) that combine any mix of models and configurations so long as they fit in DDR memory.

| Model name | Bundle template | Bundle description | Bundle configuration |
| - | :- | :- | :- |
| `DeepSeek-R1-0528` / `DeepSeek-V3.1` | deepseek-r1-v31-fp8-16k | <ul><li>Combination of: <ul><li>`DeepSeek-R1-0528` </li><li>`DeepSeek-V3.1` </li></ul></li><li>Medium context length with low batch size</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`DeepSeek-R1-0528`<ul><li>Seq Length: 16K, BS: 1</li></ul></li><li>`DeepSeek-V3.1`<ul><li>Seq Length: 16K, BS: 1</li></ul></li></ul></details> |
| `DeepSeek-V3-0324` | deepseek-r1-v3-fp8-16k | <ul><li>Combination of: <ul><li>`DeepSeek-R1-0528` </li><li>`DeepSeek-V3-0324` </li></ul></li><li>Medium context length with low batch size</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`DeepSeek-R1-0528`<ul><li>Seq Length: 16K, BS: 1</li></ul></li><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 16K, BS: 1</li></ul></li></ul></details> |
| `E5-Mistral-7B-Instruct` / `Meta-Llama-3.1-8B-Instruct` | us-agentic-rag-1-1 | <ul><li>Combination of: <ul><li>`gpt-oss-120b`</li><li>`Llama-4-Maverick-17B-128E-Instruct`</li><li>`Meta-Llama-3.1-8B-Instruct`</li><li>`Meta-Llama-3.3-70B` (Target)</li><li>`Meta-Llama-3.2-1B` (Draft)</li><li>`E5-Mistral-7B-Instruct`</li></ul></li><li>Small to medium context length with varied batch size</li><li>[Speculative decoding](https://docs.sambanova.ai/docs/en/v1.0.57/sambastack/service-administration/deploy-with-speculative-decoding.md) supported for `Meta-Llama-3.3-70B`</li></ul> | <details><summary>View</summary><ul><li>`gpt-oss-120b`<ul><li>Seq Length: 32K, BS: 4</li><li>Seq Length: 64K, BS: 2</li><li>Seq Length: 128K, BS: 2</li></ul></li><li>`Llama-4-Maverick-17B-128E-Instruct`<ul><li>Seq Length: 8K, BS: 1</li><li>Seq Length: 16K, BS: 1</li></ul></li><li>`Meta-Llama-3.3-70B` (Target)/ `Meta-Llama-3.2-1B` (Draft)<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 1, 4, 8</li><li>Seq Length: 16K, BS: 1, 4</li><li>Seq Length: 32K, BS: 1, 4</li><li>Seq Length: 64K, BS: 1</li><li>Seq Length: 128K, BS: 1</li></ul></li><li>`Meta-Llama-3.1-8B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 16, 32</li><li>Seq Length: 8K, BS: 1, 4, 16, 32</li><li>Seq Length: 16K, BS: 1, 4, 8</li></ul></li><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li></ul></li></ul></details> |
| `gemma-3-27b-it` | gemma3-27b-32-128k | Homogeneous bundle containing `gemma-3-27b-it` configurations. | <details><summary>View</summary><ul><li>`gemma-3-27b-it`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `gemma-3-12b-it` | gemma3-v3 | <ul><li>Homogeneous bundle containing `gemma-3-12b-it` configurations.</li><li>Large context length with medium batch size</li></ul> | <details><summary>View</summary><ul><li>`gemma-3-12b-it`<ul><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `gpt-oss-120b` | <ul><li>`cd-dyt-gpt-oss-120b-32-64-128k` †</li><li>`cd-dyt-gpt-oss-120b-8-32-64-128k` †</li></ul> | <ul><li>Homogeneous bundles with constrained decoding for `gpt-oss-120b`.</li><li>`cd-dyt-gpt-oss-120b-8-32-64-128k` adds 8K context support.</li></ul> | <details><summary>View</summary><ul><li>`cd-dyt-gpt-oss-120b-32-64-128k`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 128K, BS: 2</li></ul></li><li>`cd-dyt-gpt-oss-120b-8-32-64-128k`<ul><li>Seq Length: 8K, BS: 2, 4, 6, 8</li><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details> |
| `Llama-4-Maverick-17B-128E-Instruct` | llama-4-medium-8-16-32-64-128k | <ul><li>Homogeneous bundles containing `Llama-4-Maverick-17B-128E-Instruct` configurations.</li><li>Small to large context length with low batch</li></ul> | <details><summary>View</summary><ul><li>`Llama-4-Maverick-17B-128E-Instruct`<ul><li>Seq Length: 8K, BS: 1</li><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1</li><li>Seq Length: 64K, BS: 1</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details> |
| `Meta-Llama-3.3-70B-Instruct` | 70b-3dot3-ss-4-8-16-32-64-128k | <ul><li>[Speculative decoding](https://docs.sambanova.ai/docs/en/v1.0.57/sambastack/service-administration/deploy-with-speculative-decoding.md) of: <ul><li>`Meta-Llama-3.3-70B` (Target)</li><li>`Meta-Llama-3.2-1B` (Draft)</li></ul></li><li>Medium to large context length with low batch size</li></ul> | <details><summary>View</summary>**Target Models:**<ul><li>`Meta-Llama-3.3-70B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul>**Draft Models:**<ul><li>`Meta-Llama-3.2-1B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4; private: true</li><li>Seq Length: 128K, BS: 1; private: true</li></ul></li></ul></details> |
| `MiniMax-M2.5` | <ul><li>dyt-minimax-m2p5-32k</li><li>dyt-minimax-m2p5-32-160k</li></ul> | <ul><li>Homogeneous bundles containing `MiniMax-M2.5` configurations.</li><li>dyt-minimax-m2p5-32k is better for medium sequence lengths and high batching.</li><li>dyt-minimax-m2p5-32-160k is better for higher sequence lengths and low batching</li></ul> | <details><summary>View</summary><ul><li>dyt-minimax-m2p5-32k<ul><li>Seq Length: 4K-32K, BS: 2, 4, 6, 8</li></ul></li></ul><ul><li>dyt-minimax-m2p5-32-160k<ul><li>Seq Length: 32K, BS: 2</li><li>Seq Length: 160K, BS: 2</li></ul></li></ul></details> |
| `Qwen3-235B-A22B-Instruct-2507` | dyt-qwen3-235b-32-128k | Homogeneous bundle containing `Qwen3-235B-A22B-Instruct-2507` configurations. | <details><summary>View</summary><ul><li>`Qwen3-235B-A22B-Instruct-2507`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details> |
| `Qwen3-235B` | <ul><li>qwen3-235b-8k</li><li>qwen3-235b-16-32-64k</li><li>qwen3-235b-128k</li></ul> | <ul><li>Homogeneous bundles containing `Qwen3-235B` configurations.</li><li>qwen3-235b-8k: small context with higher batch size.</li><li>qwen3-235b-16-32-64k: medium context lengths.</li><li>qwen3-235b-128k: large context length.</li></ul> | <details><summary>View</summary><ul><li>qwen3-235b-8k<ul><li>Seq Length: 8K, BS: 2, 8</li></ul></li><li>qwen3-235b-16-32-64k<ul><li>Seq Length: 16K, BS: 2</li><li>Seq Length: 32K, BS: 2</li><li>Seq Length: 64K, BS: 2</li></ul></li><li>qwen3-235b-128k<ul><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details> |
| `Whisper-Large-v3` / `Qwen3-32B` | qwen3-32b-whisper-e5-mistral | <ul><li>Combination of: <ul><li>`Qwen3-32B`</li><li>`Whisper-Large-v3`</li><li>`E5-Mistral-7B-Instruct`</li></ul></li><li>Small to medium context length with varied batch size</li></ul> | <details><summary>View</summary><ul><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li></ul></li><li>`Qwen3-32B`<ul><li>Seq Length: 8K, BS: 1, 4</li><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1, 2</li></ul></li><li>`Whisper-Large-v3`<ul><li>BS: 1, 16, 32</li></ul></li></ul></details> |

† These bundles support sequence lengths from 8K–128K. If you require shorter context lengths (4K–16K) for `gpt-oss-120b`, contact your SambaNova representative.