> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Supported models and bundles

SambaStack supports a variety of models that can be deployed to both on-prem and hosted environments. Contact your system administrator to determine which models are available on your deployment. For definitions of the **Preview** and **Production** designations, see the [Glossary](/en/resources/glossary).

## Deployment options

When deploying models in SambaStack, administrators can select from various context length and batch size combinations.

* Smaller batch sizes provide higher token throughput (tokens/second).
* Larger batch sizes provide better concurrency for multiple users.

SambaStack supports two deployment configurations – high-interactivity and high-throughput. See [High-throughput deployment](/en/v1.2.0/sambastack/service-administration/deployment-configurations) for when to use each.

## Supported models

You can run the following command to discover available models in your cluster:

```shellscript theme={null}
kubectl -n <namespace> get models
```

The table below lists supported models, context lengths, batch sizes, and features.

| Developer/Model ID                                                                                                                                                                                                                                                                         | Modalities                                         | Suggested Use                                                                                                   | Context length (batch size)                                                                                                                                                                                                                                                           | Features and optimizations                                                                                                                                                                                                                                                                                                                 |
| :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Meta**                                                                                                                                                                                                                                                                                   |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`Meta-Llama-3.3-70B-Instruct`](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)                                                                                                                                                                                                  | <ul><li>Text</li></ul>                             | <ul><li>Task agent</li><li>Tool-calling agent</li><li>Text to SQL/Cipher</li></ul>                              | <details><summary>View</summary><ul><li>4K (1, 2, 4, 8, 16, 32)</li><li>8K (1, 2, 4, 8, 16, 32)</li><li>16K (1, 2, 4)</li><li>32K (1, 2, 4)</li><li>64K (1, 2, 4)</li><li>128K (1)</li></ul></details>                                                                                | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: Yes</li><li>Optimizations: [Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding)</li></ul></details>                            |
| [`Meta-Llama-3.1-8B-Instruct`](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)                                                                                                                                                                                                    | <ul><li>Text</li></ul>                             | <ul><li>Gateway agent</li><li>Validation agent</li></ul>                                                        | <details><summary>View</summary><ul><li>4K (1, 2, 4, 8, 16, 32, 64, 128)</li><li>8K (1, 2, 4, 8, 16, 32, 64)</li><li>16K (1, 2, 4, 8)</li></ul></details>                                                                                                                             | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: Yes</li><li>Optimizations: None</li></ul></details>                                                                                                                             |
| [`Meta-Llama-3.1-70B-Instruct`](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) ‡                                                                                                                                                                                                | <ul><li>Text</li></ul>                             | <ul><li>Task agent</li><li>Tool-calling agent</li></ul>                                                         | <details><summary>View</summary><ul><li>8K (1, 2, 4, 8)</li><li>16K (1, 2, 4)</li><li>32K (1, 4)</li></ul></details>                                                                                                                                                                  | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: [Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding)</li></ul></details>                             |
| [`Meta-Llama-3.1-405B-Instruct`](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct)                                                                                                                                                                                                | <ul><li>Text</li></ul>                             | <ul><li>Task agent</li><li>Tool-calling agent</li><li>Code generation</li></ul>                                 | <details><summary>View</summary><ul><li>4K (1, 2, 4)</li><li>8K (1)</li><li>16K (1)</li></ul></details>                                                                                                                                                                               | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: [Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding)</li></ul></details>                             |
| [`Llama-4-Maverick-17B-128E-Instruct`](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct)                                                                                                                                                                               | <ul><li>Image</li><li>Text</li></ul>               | <ul><li>Image understanding</li><li>Task agent</li><li>Tool-calling agent</li></ul>                             | <details><summary>View</summary><ul><li>8K (1)</li><li>16K (1)</li><li>32K (1)</li><li>64K (1)</li><li>128K (1)</li></ul></details>                                                                                                                                                   | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                              |
| **MiniMax**                                                                                                                                                                                                                                                                                |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`MiniMax-M2.7`](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                                             | <ul><li>Text</li></ul>                             | <ul><li>Coding agent</li></ul>                                                                                  | <details><summary>View</summary><ul><li>8K-32K (2, 4, 6, 8)</li><li>160K (2)</li><li>192K (2)</li></ul></details>                                                                                                                                                                     | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, Structured output</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                      |
| [`MiniMax-M2.5`](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)                                                                                                                                                                                                                            | <ul><li>Text</li></ul>                             | <ul><li>Coding agent</li></ul>                                                                                  | <details><summary>View</summary><ul><li>8K-32K (2, 4, 6, 8)</li><li>160K (2)</li><li>192K (2)</li></ul></details>                                                                                                                                                                     | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, Structured output</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                      |
| **Mistral AI**                                                                                                                                                                                                                                                                             |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`Mistral-Large-3-675B-Instruct-2512`](https://huggingface.co/mistralai/Mistral-Large-3-675B-Instruct-2512)<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup> | <ul><li>Text</li></ul>                             | <ul><li>Multilingual instruction following</li><li>Task agent</li><li>Tool-calling agent</li></ul>              | <details><summary>View</summary><ul><li>8K (1)</li></ul></details>                                                                                                                                                                                                                    | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                                         |
| **DeepSeek**                                                                                                                                                                                                                                                                               |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`DeepSeek-R1-0528`](https://huggingface.co/deepseek-ai/DeepSeek-R1)                                                                                                                                                                                                                       | <ul><li>Reasoning</li><li>Text</li></ul>           | <ul><li>Complex reasoning</li></ul>                                                                             | <details><summary>View</summary><ul><li>4K (4)</li><li>8K (1)</li><li>16K (1)</li><li>32K (1)</li></ul></details>                                                                                                                                                                     | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                              |
| [`DeepSeek-R1-Distill-Llama-70B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B) ‡                                                                                                                                                                                      | <ul><li>Reasoning</li><li>Text</li></ul>           | <ul><li>Complex reasoning</li></ul>                                                                             | <details><summary>View</summary><ul><li>4K (1, 2, 4, 8, 16, 32)</li><li>8K (1, 2, 4, 8)</li><li>16K (1, 2, 4)</li><li>32K (1, 2, 4)</li><li>64K (1)</li><li>128K (1)</li></ul></details>                                                                                              | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: None</li><li>Custom checkpoints supported: Yes</li><li>Optimizations: [Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding)</li></ul></details>                                                   |
| [`DeepSeek-V3-0324`](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324)                                                                                                                                                                                                                  | <ul><li>Text</li></ul>                             | <ul><li>Main/planner agent</li><li>Tool-calling agent</li></ul>                                                 | <details><summary>View</summary><ul><li>4K (4)</li><li>8K (1, 4)</li><li>16K (1)</li><li>32K (1)</li></ul></details>                                                                                                                                                                  | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: [High-throughput](/en/v1.2.0/sambastack/service-administration/deployment-configurations)</li></ul></details>                                         |
| [`DeepSeek-V3.1`](https://huggingface.co/deepseek-ai/DeepSeek-V3.1)                                                                                                                                                                                                                        | <ul><li>Reasoning</li><li>Text</li></ul>           | <ul><li>Main/planner agent</li><li>Tool-calling agent</li></ul>                                                 | <details><summary>View</summary><ul><li>4K (4)</li><li>8K (1, 4)</li><li>16K (1)</li><li>32K (1)</li></ul></details>                                                                                                                                                                  | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: [High-throughput](/en/v1.2.0/sambastack/service-administration/deployment-configurations)</li></ul></details>                                         |
| [`DeepSeek-V3.2`](https://huggingface.co/deepseek-ai/DeepSeek-V3.2)                                                                                                                                                                                                                        | <ul><li>Text</li></ul>                             | <ul><li>Main/planner agent</li><li>Tool-calling agent</li></ul>                                                 | <details><summary>View</summary><ul><li>8K (1, 4)</li><li>16K (1)</li><li>32K (1)</li><li>128K (1)</li></ul></details>                                                                                                                                                                | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Optional thinking mode, Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: [High-throughput](/en/v1.2.0/sambastack/service-administration/deployment-configurations)</li></ul></details>                 |
| [`DeepSeek-V3.1-Terminus`](https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus)                                                                                                                                                                                                      | <ul><li>Reasoning</li><li>Text</li></ul>           | <ul><li>Main/planner agent</li><li>Tool-calling agent</li></ul>                                                 | <details><summary>View</summary><ul><li>8K (1, 4)</li><li>16K (1)</li><li>32K (1)</li><li>128K (1)</li></ul></details>                                                                                                                                                                | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: [High-throughput](/en/v1.2.0/sambastack/service-administration/deployment-configurations)</li></ul></details>                                         |
| **OpenAI**                                                                                                                                                                                                                                                                                 |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`gpt-oss-120b`](https://huggingface.co/openai/gpt-oss-120b)                                                                                                                                                                                                                               | <ul><li>Text</li></ul>                             | <ul><li>Main/planner agent</li><li>Tool-calling agent</li></ul>                                                 | <details><summary>View</summary><ul><li>8K-32K (2, 4, 6, 8)</li><li>64K (2, 4)</li><li>128K (2)</li></ul></details>                                                                                                                                                                   | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Reasoning, Function calling, JSON mode, Logit masking, `logit_bias` sampling parameter (see [logit\_bias](/en/features/openai-compatibility#logit_bias))</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details> |
| [`gpt-oss-20b`](https://huggingface.co/openai/gpt-oss-20b)<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                                                  | <ul><li>Text</li></ul>                             | <ul><li>Main/planner agent</li><li>Tool-calling agent</li><li>Reasoning</li></ul>                               | <details><summary>View</summary><ul><li>8K-32K (2, 4, 6, 8)</li><li>64K (2, 4)</li><li>128K (2)</li></ul></details>                                                                                                                                                                   | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                              |
| [`Whisper-Large-v3`](https://huggingface.co/openai/whisper-large-v3)                                                                                                                                                                                                                       | <ul><li>Audio</li></ul>                            | <ul><li>Automatic speech recognition (ASR)</li><li>Audio transcription</li></ul>                                | <details><summary>View</summary><ul><li>4K (1,16,32)</li></ul></details>                                                                                                                                                                                                              | <details><summary>View</summary><ul><li>Endpoint: Translation, Transcription</li><li>Capabilities: None</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                                           |
| **Google**                                                                                                                                                                                                                                                                                 |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`gemma-3-27b-it`](https://huggingface.co/google/gemma-3-27b-it)<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                                            | <ul><li>Image</li><li>Text</li></ul>               | <ul><li>Image understanding</li><li>Task agent</li></ul>                                                        | <details><summary>View</summary><ul><li>4K-128K (2, 4, 6, 8)</li></ul></details>                                                                                                                                                                                                      | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Image understanding, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                           |
| [`gemma-3-12b-it`](https://huggingface.co/google/gemma-3-12b-it)<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                                            | <ul><li>Image</li><li>Text</li></ul>               | <ul><li>Image understanding</li><li>Task agent</li></ul>                                                        | <details><summary>View</summary><ul><li>128K (2, 4, 6, 8)</li></ul></details>                                                                                                                                                                                                         | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: Image understanding, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                           |
| [`gemma-4-31B-it`](https://huggingface.co/google/gemma-4-31B-it)<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                                            | <ul><li>Image</li><li>Text</li><li>Video</li></ul> | <ul><li>[Image](/en/features/vision) and [video](/en/features/video) understanding</li><li>Task agent</li></ul> | <details><summary>View</summary><ul><li>32K-128K (2, 4, 6, 8)</li><li>32K-256K<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup> (2)</li></ul></details> | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: [Image](/en/features/vision) and [video](/en/features/video) understanding, Optional thinking mode, Function calling, JSON mode</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                          |
| **Alibaba Cloud**                                                                                                                                                                                                                                                                          |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`Qwen3-235B-A22B-Instruct-2507`](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507)                                                                                                                                                                                               | <ul><li>Reasoning</li><li>Text</li></ul>           | <ul><li>Agentic planner</li><li>Multilingual instruction following</li></ul>                                    | <details><summary>View</summary><ul><li>32K (2, 4, 6, 8)</li><li>128K (2)</li></ul></details>                                                                                                                                                                                         | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: None</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                                                     |
| [`Qwen3-32B`](https://huggingface.co/Qwen/Qwen3-32B)                                                                                                                                                                                                                                       | <ul><li>Reasoning</li><li>Text</li></ul>           | <ul><li>Task agent</li><li>Multilingual instruction following</li></ul>                                         | <details><summary>View</summary><ul><li>8K (1, 4)</li><li>16K (1)</li><li>32K (1, 2)</li></ul></details>                                                                                                                                                                              | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: None</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                                                     |
| [`Qwen3-TTS-Talker`](https://huggingface.co/Qwen/Qwen3-TTS)<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                                                 | <ul><li>Audio</li><li>Text</li></ul>               | <ul><li>Text-to-speech</li></ul>                                                                                | <details><summary>View</summary><ul><li>4K (2)</li></ul></details>                                                                                                                                                                                                                    | <details><summary>View</summary><ul><li>Endpoint: Speech</li><li>Capabilities: None</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                                                               |
| [`Qwen3-TTS-Vocoder`](https://huggingface.co/Qwen/Qwen3-TTS)<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                                                | <ul><li>Audio</li></ul>                            | <ul><li>Audio synthesis</li></ul>                                                                               | <details><summary>View</summary><ul><li>4K (1)</li></ul></details>                                                                                                                                                                                                                    | <details><summary>View</summary><ul><li>Endpoint: Speech</li><li>Capabilities: None</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                                                               |
| **Tokyotech-llm**                                                                                                                                                                                                                                                                          |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`Llama-3.3-Swallow-70B-Instruct-v0.4`](https://huggingface.co/tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4) ‡                                                                                                                                                                        | <ul><li>Text</li></ul>                             | <ul><li>Japanese instruction following</li><li>Task agent</li></ul>                                             | <details><summary>View</summary><ul><li>4K (1, 2, 4, 8, 16)</li><li>8K (1, 2, 4, 8, 16)</li><li>16K (1, 2, 4)</li><li>32K (1, 2, 4)</li><li>64K (1)</li><li>128K (1)</li></ul></details>                                                                                              | <details><summary>View</summary><ul><li>Endpoint: Chat completions</li><li>Capabilities: None</li><li>Custom checkpoints supported: No</li><li>Optimizations: [Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding)</li></ul></details>                                                    |
| **Other**                                                                                                                                                                                                                                                                                  |                                                    |                                                                                                                 |                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                            |
| [`E5-Mistral-7B-Instruct`](https://huggingface.co/intfloat/e5-mistral-7b-instruct)                                                                                                                                                                                                         | <ul><li>Embedding</li></ul>                        | <ul><li>Vector storage and retrieval (RAG)</li></ul>                                                            | <details><summary>View</summary><ul><li>4K (1, 4, 8, 16, 32)</li></ul></details>                                                                                                                                                                                                      | <details><summary>View</summary><ul><li>Endpoint: Embeddings</li><li>Capabilities: None</li><li>Custom checkpoints supported: No</li><li>Optimizations: None</li></ul></details>                                                                                                                                                           |

‡ No prebuilt bundle is available for this model in SambaStack. You can use this model to create a [custom bundle](/en/v1.2.0/sambastack/service-administration/custom-bundle-deployment).

## Recommended model bundles

In SambaStack, models are not deployed individually; they are deployed as bundles. A bundle is a packaged deployment that groups one or more models together with their associated configurations, such as batch size and sequence length.

For example, deploying the `Meta‑Llama‑3.3‑70B` model with a batch size of 4 and a sequence length of 16K tokens constitutes a single configuration. A bundle, however, can contain multiple such configurations, either for the same model or for different models.

SambaNova’s RDU technology enables several models and configurations to be loaded simultaneously in a single deployment. This allows you to switch instantly between models and between batch‑/sequence‑size profiles as needed. In contrast to traditional GPU systems, where deployments are typically single‑model and static, SambaStack supports multi‑model, multi‑configuration bundles. This approach delivers higher efficiency, greater flexibility, and increased throughput while preserving low latency.

You can run the following command to discover available bundles in your cluster:

```shellscript theme={null}
kubectl -n <namespace> get bundles
```

The table below lists the recommended bundle templates for the models currently available in SambaStack. Each entry pairs a model with its recommended deployment bundle.

<Note>
  If the bundles listed below do not satisfy your inference requirements, you can create [custom bundles](/en/v1.2.0/sambastack/service-administration/custom-bundle-deployment) that combine any mix of models and configurations so long as they fit in DDR memory.
</Note>

### Suggested bundles per model

For each model, this section lists the suggested bundle for typical use and any alternative bundles that trade off context length, batch size, or modality support. See [Bundle configurations](#bundle-configurations) below for the seq length/batch size details of each bundle.

#### Meta

<CardGroup cols={2}>
  <Card>
    **`Meta-Llama-3.3-70B-Instruct`**

    **Suggested:** `70b-3dot3-ss-4-8-16-32-64-128k`

    **Alternatives:** `70b-3dot3-ss-full-whisper`, `us-agentic-rag-1-1`, `e5-mistral-70b-64k-128k`
  </Card>

  <Card>
    **`Meta-Llama-3.1-8B-Instruct`**

    **Suggested:** `us-agentic-rag-1-1`

    **Alternatives:** `qwen3-32b-llama405b-s-m`
  </Card>

  <Card>
    **`Meta-Llama-3.1-405B-Instruct`**

    **Suggested:** `qwen3-32b-llama405b-s-m`
  </Card>

  <Card>
    **`Llama-4-Maverick-17B-128E-Instruct`**

    **Suggested:** `llama-4-medium-8-16-32-64-128k`

    **Alternatives:** `llama-4-medium-ss-16k-bs24`, `us-agentic-rag-1-1`
  </Card>
</CardGroup>

#### MiniMax

<CardGroup cols={2}>
  <Card>
    **`MiniMax-M2.7`**<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>

    **Suggested:** `dyt-minimax-m2p7-32-64-192k-pc`

    **Alternatives:** `dyt-minimax-m2p7-32-160-192k`, `dyt-minimax-m2p7-32k-v2`
  </Card>

  <Card>
    **`MiniMax-M2.5`**

    **Suggested:** `dyt-minimax-m2p5-32-160k`

    **Alternatives:** `dyt-minimax-m2p5-32k`
  </Card>
</CardGroup>

#### Mistral AI

<CardGroup cols={2}>
  <Card>
    **`Mistral-Large-3-675B-Instruct-2512`**<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>

    **Suggested:** `mistral-large-3-fp8-8-16-32k`

    **Alternatives:** `mistral-large-3-fp8-8k`
  </Card>
</CardGroup>

#### DeepSeek

<CardGroup cols={2}>
  <Card>
    **`DeepSeek-R1-0528`**

    **Suggested:**

    * `deepseek-5in1-fp8-16-32k` (higher interactivity)
    * `deepseek-4in1-fp8-128k` (higher context length)

    **Alternatives:** `deepseek-r1-v3-fp8-8k`, `deepseek-r1-v31-fp8-8k`
  </Card>

  <Card>
    **`DeepSeek-V3-0324`**

    **Suggested:**

    * `deepseek-5in1-fp8-16-32k` (higher interactivity)
    * `deepseek-4in1-fp8-128k` (higher context length)

    **Alternatives:** `deepseek-r1-v3-fp8-8k`, `deepseek-v3-v31-fp8-8k`, `deepseek-v3-v3termi-fp8-8k`
  </Card>

  <Card>
    **`DeepSeek-V3.1`**

    **Suggested:**

    * `deepseek-5in1-fp8-16-32k` (higher interactivity)
    * `deepseek-4in1-fp8-128k` (higher context length)

    **Alternatives:** `deepseek-r1-v31-fp8-8k`, `deepseek-v3-v31-fp8-8k`
  </Card>

  <Card>
    **`DeepSeek-V3.1-Terminus`**

    **Suggested:**

    * `deepseek-5in1-fp8-16-32k` (higher interactivity)
    * `deepseek-4in1-fp8-128k` (higher context length)

    **Alternatives:** `deepseek-v3-v3termi-fp8-8k`
  </Card>

  <Card>
    **`DeepSeek-V3.2`**

    **Suggested:**

    * `deepseek-5in1-fp8-16-32k` (higher interactivity)
    * `deepseek-4in1-fp8-128k` (higher context length)
  </Card>
</CardGroup>

#### OpenAI

<CardGroup cols={2}>
  <Card>
    **`gpt-oss-120b`**

    **Suggested:** `us-agentic-rag-1-1`

    **Alternatives:** `cd-dyt-gpt-oss-120b-8-32-64-128k`, `gpt-gemma-whisper-mistral`
  </Card>

  <Card>
    **`gpt-oss-20b`**<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>

    **Suggested:** `dyt-gpt-oss-20b-32-64-128k`
  </Card>

  <Card>
    **`Whisper-Large-v3`**

    **Suggested:** `qwen3-32b-whisper-e5-mistral`

    **Alternatives:** `70b-3dot3-ss-full-whisper` (known issue: does not load within default startup time), `gpt-gemma-whisper-mistral`
  </Card>
</CardGroup>

#### Google

<CardGroup cols={2}>
  <Card>
    **`gemma-3-27b-it`**<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>

    **Suggested:** `gemma3-27b-32-128k`
  </Card>

  <Card>
    **`gemma-3-12b-it`**<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>

    **Suggested:** `gemma3-v3`

    **Alternatives:** `gpt-gemma-whisper-mistral`
  </Card>

  <Card>
    **`gemma-4-31B-it`**<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>

    **Suggested:**

    * `cd-gemma-4-31b-32-128-256k` (text only; higher context length and constrained decoding support, no image/video support)
    * `gemma-4-31b-32-128k` (image/video support, higher throughput)
  </Card>
</CardGroup>

#### Alibaba Cloud

<CardGroup cols={2}>
  <Card>
    **`Qwen3-235B-A22B-Instruct-2507`**

    **Suggested:** `dyt-qwen3-235b-32-128k`
  </Card>

  <Card>
    **`Qwen3-32B`**

    **Suggested:** `qwen3-32b-whisper-e5-mistral`

    **Alternatives:** `qwen3-32b-llama405b-s-m`
  </Card>

  <Card>
    **`Qwen3-TTS-Talker`**<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>

    **Suggested:** `qwen3-tts-talker`
  </Card>

  <Card>
    **`Qwen3-TTS-Vocoder`**<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>

    **Suggested:** `qwen3-tts-vocoder`
  </Card>
</CardGroup>

#### Other

<CardGroup cols={2}>
  <Card>
    **`E5-Mistral-7B-Instruct`**

    **Suggested:** `us-agentic-rag-1-1`

    **Alternatives:** `e5-mistral-70b-64k-128k`, `qwen3-32b-whisper-e5-mistral`, `gpt-gemma-whisper-mistral`
  </Card>
</CardGroup>

### Bundle configurations

The table below lists the configuration details for each bundle template referenced above.

| Model name                                                                                                                                                                                                          | Bundle template                                                                                                                                                                                             | Bundle description                                                                                                                                                                                                                                              | Bundle configuration                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DeepSeek-R1-0528` / <br />`DeepSeek-V3.1`                                                                                                                                                                          | deepseek-r1-v31-fp8-8k                                                                                                                                                                                      | <ul><li>Medium context length with low batch size</li></ul>                                                                                                                                                                                                     | <details><summary>View</summary><p><strong>Models:</strong></p><ul><li>`DeepSeek-R1-0528`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li><li>`DeepSeek-V3.1`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `DeepSeek-R1-0528` / <br />`DeepSeek-V3-0324`                                                                                                                                                                       | deepseek-r1-v3-fp8-8k                                                                                                                                                                                       | <ul><li>Medium context length with low batch size</li></ul>                                                                                                                                                                                                     | <details><summary>View</summary><p><strong>Models:</strong></p><ul><li>`DeepSeek-R1-0528`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `DeepSeek-V3-0324` / <br />`DeepSeek-V3.1`                                                                                                                                                                          | deepseek-v3-v31-fp8-8k                                                                                                                                                                                      | <ul><li>Medium context length with low batch size</li></ul>                                                                                                                                                                                                     | <details><summary>View</summary><p><strong>Models:</strong></p><ul><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li><li>`DeepSeek-V3.1`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `DeepSeek-V3-0324` / <br />`DeepSeek-V3.1-Terminus`                                                                                                                                                                 | deepseek-v3-v3termi-fp8-8k                                                                                                                                                                                  | <ul><li>Medium context length with low batch size</li></ul>                                                                                                                                                                                                     | <details><summary>View</summary><p><strong>Models:</strong></p><ul><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li><li>`DeepSeek-V3.1-Terminus`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `DeepSeek-R1-0528` / <br />`DeepSeek-V3-0324` / <br />`DeepSeek-V3.1` / <br />`DeepSeek-V3.1-Terminus`                                                                                                              | deepseek-4in1-fp8-128k                                                                                                                                                                                      | <ul><li>Large context length with single batch size</li><li>Four DeepSeek models in one bundle</li></ul>                                                                                                                                                        | <details><summary>View</summary><p><strong>Models:</strong></p><ul><li>`DeepSeek-R1-0528`<ul><li>Seq Length: 128K, BS: 1</li></ul></li><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 128K, BS: 1</li></ul></li><li>`DeepSeek-V3.1`<ul><li>Seq Length: 128K, BS: 1</li></ul></li><li>`DeepSeek-V3.1-Terminus`<ul><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `DeepSeek-R1-0528` / <br />`DeepSeek-V3-0324` / <br />`DeepSeek-V3.1` / <br />`DeepSeek-V3.1-Terminus` / <br />`DeepSeek-V3.2`                                                                                      | deepseek-5in1-fp8-16-32k                                                                                                                                                                                    | <ul><li>Medium context length with single batch size</li><li>Five DeepSeek models in one bundle</li></ul>                                                                                                                                                       | <details><summary>View</summary><p><strong>Models (each):</strong></p><ul><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1</li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `E5-Mistral-7B-Instruct` / <br />`Meta-Llama-3.1-8B-Instruct` / <br />`Llama-4-Maverick-17B-128E-Instruct` / <br />`Meta-Llama-3.3-70B-Instruct` / <br />`gpt-oss-120b`                                             | us-agentic-rag-1-1                                                                                                                                                                                          | <ul><li>Small to medium context length with varied batch size</li><li>[Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding) supported for `Meta-Llama-3.3-70B`</li></ul>                                        | <details><summary>View</summary><ul><li>`gpt-oss-120b`<ul><li>Seq Length: 32K, BS: 4</li><li>Seq Length: 64K, BS: 2</li><li>Seq Length: 128K, BS: 2</li></ul></li><li>`Llama-4-Maverick-17B-128E-Instruct`<ul><li>Seq Length: 8K, BS: 1</li><li>Seq Length: 16K, BS: 1</li></ul></li><li>`Meta-Llama-3.3-70B` (Target)/ `Meta-Llama-3.2-1B` (Draft)<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 1, 4, 8</li><li>Seq Length: 16K, BS: 1, 4</li><li>Seq Length: 32K, BS: 1, 4</li><li>Seq Length: 64K, BS: 1</li><li>Seq Length: 128K, BS: 1</li></ul></li><li>`Meta-Llama-3.1-8B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 16, 32</li><li>Seq Length: 8K, BS: 1, 4, 16, 32</li><li>Seq Length: 16K, BS: 1, 4, 8</li></ul></li><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li></ul></li></ul></details> |
| `E5-Mistral-7B-Instruct` / <br />`Meta-Llama-3.3-70B`                                                                                                                                                               | e5-mistral-70b-64k-128k                                                                                                                                                                                     | <ul><li>Large context length with low batch size</li><li>[Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding) supported for `Meta-Llama-3.3-70B`</li></ul>                                                     | <details><summary>View</summary><p><strong>Models:</strong></p><ul><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li></ul></li><li>`Meta-Llama-3.3-70B` (Target)/ `Meta-Llama-3.2-1B` (Draft)<ul><li>Seq Length: 64K, BS: 1</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `gemma-3-27b-it`                                                                                                                                                                                                    | gemma3-27b-32-128k                                                                                                                                                                                          | Homogeneous bundle containing `gemma-3-27b-it` configurations.                                                                                                                                                                                                  | <details><summary>View</summary><ul><li>`gemma-3-27b-it`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `gemma-3-12b-it`                                                                                                                                                                                                    | gemma3-v3                                                                                                                                                                                                   | <ul><li>Homogeneous bundle containing `gemma-3-12b-it` configurations.</li><li>Large context length with medium batch size</li></ul>                                                                                                                            | <details><summary>View</summary><ul><li>`gemma-3-12b-it`<ul><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `gemma-4-31B-it`                                                                                                                                                                                                    | gemma-4-31b-32-128k                                                                                                                                                                                         | Homogeneous bundle containing `gemma-4-31B-it` configurations.                                                                                                                                                                                                  | <details><summary>View</summary><ul><li>`gemma-4-31B-it`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `gemma-4-31B-it`                                                                                                                                                                                                    | `cd-gemma-4-31b-32-128-256k`<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup> | <ul><li>Homogeneous bundle with constrained decoding for `gemma-4-31B-it`. Adds 32K–256K context support (preview). Image and video inputs have known accuracy issues across all sequence lengths in this bundle.</li></ul>                                     | <details><summary>View</summary><ul><li>`gemma-4-31B-it`<ul><li>Seq Length: 32K, BS: 2</li><li>Seq Length: 128K, BS: 2</li><li>Seq Length: 256K, BS: 2</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| `gpt-oss-120b`                                                                                                                                                                                                      | <ul><li>`cd-dyt-gpt-oss-120b-32-64-128k` †</li><li>`cd-dyt-gpt-oss-120b-8-32-64-128k` †</li></ul>                                                                                                           | <ul><li>Homogeneous bundles with constrained decoding for `gpt-oss-120b`.</li><li>`cd-dyt-gpt-oss-120b-8-32-64-128k` adds 8K context support.</li><li>Refreshed PEFs include structured output (logit-masking) support.</li></ul>                               | <details><summary>View</summary><ul><li>`cd-dyt-gpt-oss-120b-32-64-128k`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 128K, BS: 2</li></ul></li><li>`cd-dyt-gpt-oss-120b-8-32-64-128k`<ul><li>Seq Length: 8K, BS: 2, 4, 6, 8</li><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| `gpt-oss-20b`                                                                                                                                                                                                       | `dyt-gpt-oss-20b-32-64-128k` †                                                                                                                                                                              | Homogeneous bundle with constrained decoding for `gpt-oss-20b`.                                                                                                                                                                                                 | <details><summary>View</summary><ul><li>`dyt-gpt-oss-20b-32-64-128k`<ul><li>Seq Length: 8K, BS: 2, 4, 6, 8</li><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `E5-Mistral-7B-Instruct` / <br />`Whisper-Large-v3` / <br />`gemma-3-12b-it` / <br />`gpt-oss-120b`                                                                                                                 | gpt-gemma-whisper-mistral                                                                                                                                                                                   | <ul><li>Small to medium context length combining embeddings, transcription, image understanding, and a tool-calling agent</li></ul>                                                                                                                             | <details><summary>View</summary><p><strong>Models:</strong></p><ul><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1</li></ul></li><li>`Whisper-Large-v3`<ul><li>BS: 1</li></ul></li><li>`gemma-3-12b-it`<ul><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li><li>`gpt-oss-120b`<ul><li>Seq Length: 8K, BS: 2, 4, 6, 8</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `Llama-4-Maverick-17B-128E-Instruct`                                                                                                                                                                                | llama-4-medium-8-16-32-64-128k                                                                                                                                                                              | <ul><li>Homogeneous bundles containing `Llama-4-Maverick-17B-128E-Instruct` configurations.</li><li>Small to large context length with low batch</li></ul>                                                                                                      | <details><summary>View</summary><ul><li>`Llama-4-Maverick-17B-128E-Instruct`<ul><li>Seq Length: 8K, BS: 1</li><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1</li><li>Seq Length: 64K, BS: 1</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| `Llama-4-Maverick-17B-128E-Instruct`                                                                                                                                                                                | llama-4-medium-ss-16k-bs24                                                                                                                                                                                  | Homogeneous bundle containing `Llama-4-Maverick-17B-128E-Instruct` configurations; medium context length with higher batch size.                                                                                                                                | <details><summary>View</summary><ul><li>`Llama-4-Maverick-17B-128E-Instruct`<ul><li>Seq Length: 16K, BS: 2, 4</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `Meta-Llama-3.3-70B-Instruct`                                                                                                                                                                                       | 70b-3dot3-ss-4-8-16-32-64-128k                                                                                                                                                                              | <ul><li>Medium to large context length with low batch size</li></ul>                                                                                                                                                                                            | <details><summary>View</summary><p><strong>Target Models:</strong></p><ul><li>`Meta-Llama-3.3-70B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul><p><strong>Draft Models:</strong></p><ul><li>`Meta-Llama-3.2-1B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4; private: true</li><li>Seq Length: 128K, BS: 1; private: true</li></ul></li></ul></details>                                                                                                               |
| `Meta-Llama-3.3-70B-Instruct` / <br />`Whisper-Large-v3`                                                                                                                                                            | 70b-3dot3-ss-full-whisper                                                                                                                                                                                   | <ul><li>Full context-length range with low to medium batch size, plus Whisper transcription</li><li>[Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding) supported for `Meta-Llama-3.3-70B-Instruct`</li></ul> | <details><summary>View</summary><p><strong>Target Models:</strong></p><ul><li>`Meta-Llama-3.3-70B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1</li></ul></li><li>`Whisper-Large-v3`<ul><li>BS: 1, 16, 32</li></ul></li></ul><p><strong>Draft Models:</strong></p><ul><li>`Meta-Llama-3.2-1B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details>                                                                                   |
| `MiniMax-M2.5`                                                                                                                                                                                                      | <ul><li>dyt-minimax-m2p5-32k</li><li>dyt-minimax-m2p5-32-160k</li></ul>                                                                                                                                     | <ul><li>Homogeneous bundles containing `MiniMax-M2.5` configurations.</li><li>dyt-minimax-m2p5-32k is better for medium sequence lengths and high batching.</li><li>dyt-minimax-m2p5-32-160k is better for higher sequence lengths and low batching</li></ul>   | <details><summary>View</summary><ul><li>dyt-minimax-m2p5-32k<ul><li>Seq Length: 4K-32K, BS: 2, 4, 6, 8</li></ul></li></ul><ul><li>dyt-minimax-m2p5-32-160k<ul><li>Seq Length: 32K, BS: 2</li><li>Seq Length: 160K, BS: 2</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `MiniMax-M2.7`<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                       | dyt-minimax-m2p7-32k-v2                                                                                                                                                                                     | Homogeneous bundle containing `MiniMax-M2.7` configurations; medium context length with high batching.                                                                                                                                                          | <details><summary>View</summary><ul><li>`MiniMax-M2.7`<ul><li>Seq Length: 8K-32K, BS: 2, 4, 6, 8</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `MiniMax-M2.7`<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                       | dyt-minimax-m2p7-32-160-192k                                                                                                                                                                                | Homogeneous bundle containing `MiniMax-M2.7` configurations; better for higher sequence lengths and low batching.                                                                                                                                               | <details><summary>View</summary><ul><li>`MiniMax-M2.7`<ul><li>Seq Length: 8K-32K, BS: 2, 4, 6, 8</li><li>Seq Length: 160K-192K, BS: 2</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `MiniMax-M2.7`<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                       | dyt-minimax-m2p7-32-64-192k-pc                                                                                                                                                                              | Homogeneous bundle with prompt caching for `MiniMax-M2.7`.                                                                                                                                                                                                      | <details><summary>View</summary><ul><li>`MiniMax-M2.7`<ul><li>Seq Length: 32K-192K, BS: 2, 4, 6, 8</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| `Mistral-Large-3-675B-Instruct-2512`<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup> | mistral-large-3-fp8-8k                                                                                                                                                                                      | <ul><li>Homogeneous bundle containing `Mistral-Large-3-675B-Instruct-2512` configurations.</li><li>Preview model – text-only.</li></ul>                                                                                                                         | <details><summary>View</summary><ul><li>`Mistral-Large-3-675B-Instruct-2512`<ul><li>Seq Length: 8K, BS: 1</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `Mistral-Large-3-675B-Instruct-2512`<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup> | mistral-large-3-fp8-8-16-32k                                                                                                                                                                                | <ul><li>Homogeneous bundle containing `Mistral-Large-3-675B-Instruct-2512` configurations.</li><li>Wider context range than `mistral-large-3-fp8-8k`. Preview model – text-only.</li></ul>                                                                      | <details><summary>View</summary><ul><li>`Mistral-Large-3-675B-Instruct-2512`<ul><li>Seq Length: 8K, BS: 1, 4</li><li>Seq Length: 16K, BS: 1, 2</li><li>Seq Length: 32K, BS: 1</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `Qwen3-235B-A22B-Instruct-2507`                                                                                                                                                                                     | dyt-qwen3-235b-32-128k                                                                                                                                                                                      | Homogeneous bundle containing `Qwen3-235B-A22B-Instruct-2507` configurations.                                                                                                                                                                                   | <details><summary>View</summary><ul><li>`Qwen3-235B-A22B-Instruct-2507`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `Whisper-Large-v3` / <br />`Qwen3-32B` / <br />`E5-Mistral-7B-Instruct`                                                                                                                                             | qwen3-32b-whisper-e5-mistral                                                                                                                                                                                | <ul><li>Small to medium context length with varied batch size</li></ul>                                                                                                                                                                                         | <details><summary>View</summary><ul><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li></ul></li><li>`Qwen3-32B`<ul><li>Seq Length: 8K, BS: 1, 4</li><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1, 2</li></ul></li><li>`Whisper-Large-v3`<ul><li>BS: 1, 16, 32</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `Qwen3-32B` / <br />`Meta-Llama-3.1-405B-Instruct`                                                                                                                                                                  | qwen3-32b-llama405b-s-m                                                                                                                                                                                     | <ul><li>Small to medium context length</li><li>[Speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding) supported for `Meta-Llama-3.1-405B-Instruct`</li></ul>                                                     | <details><summary>View</summary><p><strong>Target Models:</strong></p><ul><li>`Meta-Llama-3.1-405B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 2, 4</li><li>Seq Length: 8K, BS: 1</li><li>Seq Length: 16K, BS: 1</li></ul></li></ul><p><strong>Draft Models:</strong></p><ul><li>`Meta-Llama-3.1-8B-Instruct`<ul><li>Seq Length: 16K, BS: 1</li></ul></li><li>`Meta-Llama-3.2-3B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 2, 4</li><li>Seq Length: 8K, BS: 1</li></ul></li></ul><p><strong>Routable Models:</strong></p><ul><li>`Qwen3-32B`<ul><li>Seq Length: 8K, BS: 1, 4</li><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1</li></ul></li></ul></details>                                                                                                                                                                                             |
| `Qwen3-TTS-Talker`<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                   | qwen3-tts-talker                                                                                                                                                                                            | Homogeneous bundle containing `Qwen3-TTS-Talker` configurations.                                                                                                                                                                                                | <details><summary>View</summary><ul><li>`Qwen3-TTS-Talker`<ul><li>Seq Length: 4K, BS: 2</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `Qwen3-TTS-Vocoder`<sup style={{background:"#4338ca",color:"#ffffff",fontWeight:700,fontSize:"0.62em",letterSpacing:"0.03em",padding:"1px 4px",borderRadius:"3px",marginLeft:"3px"}}>PREVIEW</sup>                  | qwen3-tts-vocoder                                                                                                                                                                                           | Homogeneous bundle containing `Qwen3-TTS-Vocoder` configurations.                                                                                                                                                                                               | <details><summary>View</summary><ul><li>`Qwen3-TTS-Vocoder`<ul><li>Seq Length: 4K, BS: 1</li></ul></li></ul></details>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |

† These bundles support sequence lengths from 8K–128K. If you require shorter context lengths (4K–16K) for `gpt-oss-120b`, contact your SambaNova representative.