> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# SambaStack release notes
Release notes for SambaStack, including new features, enhancements, and fixes. Model lifecycle stage changes are announced here — for what each stage means and the notice you receive before a model is retired, see [SambaStack model lifecycle](/docs/en/v2.2.5/sambastack/service-administration/model-deployment/model-lifecycle).
**Using SambaWiz to build bundles?** Always pull the latest [SambaWiz](https://github.com/sambanova/sambastack-tools/tree/main/sambawiz) version with each release so you have access to the models and features it introduces.
***
## SambaStack v2.2.5 release
**Release Date:** October 14, 2026
This release introduces the following key updates:
* **`gemma-4-31B-it` with prompt caching** – A new bundle that runs `gemma-4-31B-it` with prompt caching, up to 256K context, for text input only
* **`GLM-5.3` with prompt caching (Preview)** – A new bundle that runs `GLM-5.3` with prompt caching, up to 1M context
* **Faster `MiniMax-M3` prompt caching with image input** – The `MiniMax-M3` prompt caching bundles use an updated compiled model and accept image input
For deployment details, bundle configurations, and context lengths, see [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
### Upgrade notes
Read the items below, then follow [Upgrading SambaStack](/docs/en/sambastack/service-administration/platform-admin/upgrade-sambastack) to upgrade.
**Component versions**
| Component | v2.2.3 | v2.2.5 |
| :- | :- | :- |
| Helm chart | 2.2.3 | 2.2.5 |
| Runtime | 1.26.3-2 | 1.26.3-2 |
***
### New features and enhancements
**Model updates**
Updated model and bundle support in SambaStack v2.2.5:
* **`gemma-4-31b-acb-prefix-caching`** – Runs `gemma-4-31B-it` with prompt caching, continuous batching, multi-token prediction, and constrained decoding, up to 256K context. It accepts text input only.
* **`glm-5-3-acb-prefix-caching` – Preview** – Runs `GLM-5.3` with prompt caching, continuous batching, multi-token prediction, and constrained decoding, up to 1M context. It accepts text input only.
* **`MiniMax-M3` prompt caching bundles** – `minimax-m3-acb-prefix-caching` and the 32K, 256K, 512K, and 1M prompt caching bundles use an updated compiled model with a shorter time to first token and higher throughput. They also accept image input. Requests that contain an image are served without prompt caching, and video input is not supported.
**For `gemma-4-31B-it`, deploy either the prompt caching bundle or an image bundle, not both.**
`gemma-4-31b-acb-prefix-caching` has prompt caching but accepts text only. These image bundles accept image and video input but have no prompt caching:
* `gemma-4-31b-32-128k`
* `gemma-4-31b-32-128-256k`
* `gemma-4-31b-mtp-cd-8-32-64-128k`
All four bundles serve the same model, `gemma-4-31B-it`, and SambaStack sends each request to a bundle by model name only. If you deploy the prompt caching bundle with an image bundle, an image request can go to the prompt caching bundle, which does not accept image input.
If you don't need image input, deploy `gemma-4-31b-acb-prefix-caching` to use prompt caching.
See [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
***
**Client disconnects reported as HTTP 499**
When a client disconnects before a response finishes, the routing broker records the request as HTTP 499, not HTTP 502.
***
### Known issues
**`gemma-4-31B-it`**
These issues affect `gemma-4-31b-acb-prefix-caching`.
* **Slower with sampling** – Requests that sample, such as with `temperature` above 0, generate about half as many tokens per second per user as greedy requests.
* **Changes tool call arguments** – A string argument that contains a comma, braces, Unicode, or escape characters can be split into a shorter value and an extra key. For example, `Boston, MA` becomes `Boston` and a new `MA` key. Fields that follow a nested object in the arguments can be dropped.
* **Returns the wrong argument type with tool-local `$defs`** – When two tools define the same `$defs` name with different types, a call can return an object where its tool's schema requires a string.
* **Overgenerates on long tool arguments** – A forced tool call that asks for a long repeated string, such as 4,096 characters, can stream the string as text until `max_tokens` and return no tool call.
* **Returns HTTP 500 when the output budget is omitted** – A `/v1/responses` request without `max_output_tokens`, or a `/v1/chat/completions` request with `max_tokens` omitted or `null`, returns HTTP 500. A prompt near the 256K context limit also returns HTTP 500. Set `max_tokens` or `max_output_tokens`.
* **`/v1/responses` rejects parallel tool results** – An `input` history with two function calls and their paired outputs returns HTTP 400.
* **`/v1/responses` omits output token details** – `usage.output_tokens_details` is missing from responses.
* **Does not support `repetition_penalty`** – A request that sets `repetition_penalty` to a value other than `1.0` is rejected. Leave out `repetition_penalty` or set it to `1.0`.
* **Does not support `logprobs`** – A request with `logprobs: true` returns HTTP 500. Leave out `logprobs` or set it to `false`.
* **Accepts malformed content parts** – A message content part with missing `text`, a non-string `text` value, or an unsupported `type` returns HTTP 200, not HTTP 400.
**`GLM-5.3`**
* **Times out at 1M context** – The bundle processes one full-context request at a time, and a 1M-token prompt can take longer than the gateway's 600-second timeout, which returns HTTP 504. Requests near 1M tokens fail under concurrent load.
* **Always reasons** – `chat_template_kwargs` settings that turn thinking off have no effect, and the response includes reasoning. With a small `max_tokens`, reasoning can use the whole budget and return empty `content` with `finish_reason: length`. Set `max_tokens` high enough to cover reasoning and the answer.
* **Returns reasoning in `content` for system-only prompts** – A request with only a `system` message returns its reasoning, with a stray `` tag, in `content`.
* **Ignores `developer` messages** – Messages with `role: developer` are dropped without an error.
* **Rejects `json_schema` with a non-object root** – A strict `json_schema` whose root type is a boolean, string, number, integer, null, or a union returns HTTP 400 `Model did not output valid JSON.` Use a schema with an object root.
* **Does not report reasoning tokens** – `usage.completion_tokens_details.reasoning_tokens` is missing from responses.
* **Does not support `repetition_penalty`** – Only the default value of `1.0` is accepted. A request that sets any other value is rejected. `temperature`, `top_k`, and `top_p` work as expected. Leave out `repetition_penalty` or set it to `1.0`. The HTTP 400 for a non-zero `frequency_penalty` suggests `repetition_penalty`, which `GLM-5.3` also rejects.
* **Does not support `logprobs`** – A request with `logprobs: true` returns HTTP 500. Leave out `logprobs` or set it to `false`.
* **Does not support `/v1/responses`** – Use `/v1/chat/completions`.
* **Has no `GLM-5.3-1024k` alias** – The model cannot be called as `GLM-5.3-1024k`. Use `GLM-5.3`.
* **Accepts some invalid requests** – On `/v1/messages`, a `thinking.budget_tokens` that is 0, too small, or larger than `max_tokens` returns HTTP 200, not HTTP 400. A tool with an invalid schema also returns HTTP 200.
**`MiniMax-M3`**
* **Returns null `logprobs`** – A request with `logprobs: true` returns HTTP 200 with `logprobs: null` on every choice.
* **Prompt caching bundles stop early with `ignore_eos`** – With `ignore_eos: true`, a request can still stop at an end-of-sequence token before reaching `max_tokens`.
* **Prompt caching bundles return HTTP 500 for `repetition_penalty`** – A `repetition_penalty` other than `1.0` returns HTTP 500. The HTTP 400 for a non-zero `frequency_penalty` or `presence_penalty` suggests `repetition_penalty`, which these bundles also reject. Leave out `repetition_penalty` or set it to `1.0`.
* **Prompt caching bundles over-count `prompt_tokens`** – `usage.prompt_tokens` is about 126 tokens higher than the count from the model's tokenizer for the same input.
***
## SambaStack v2.2.3 release
**Release Date:** October 2, 2026
**Read the upgrade notes before you upgrade.** Support for V1 and V2 bundle resources ended on September 30, 2026. See **Deprecations and removals**.
This release introduces the following key updates:
* **Multi-token prediction for `gemma-4-31B-it`** – A new Preview bundle that generates several tokens per step for faster output
* **Prompt caching for `MiniMax-M3`** – A new Preview bundle with prompt caching for text input, up to 1M context
* **More transcription formats** – `/v1/audio/transcriptions` supports `verbose_json`, `srt`, and `vtt`, with segment timestamps
For deployment details, bundle configurations, and context lengths, see [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles). The model, API, and inference changes below also apply to SambaCloud.
### Upgrade notes
Read the items below, then follow [Upgrading SambaStack](/docs/en/sambastack/service-administration/platform-admin/upgrade-sambastack) to upgrade. Back up any applications you installed yourself before you start. SambaNova cannot back up or restore them.
**Edit your values files before you upgrade.** All model resources are in the models chart. Remove these keys:
| Key | What to do |
| :- | :- |
| `bundleSpecs`, `bundleDeploymentSpecs` | Remove. V1 and V2 bundle templates are no longer rendered |
| `global.modelComponents.enabled` | Remove. The models chart always renders model resources |
**Replace `spec.defaultBatchingConfig` in custom model profiles.** If you have a custom `ModelProfile` that uses `spec.defaultBatchingConfig`, replace it with `spec.batchingConfigs`. See [ModelProfile](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-profile).
**Service tiers and console configuration are validated.** Service tier values are checked when the chart renders. Each required model and parent tier must exist, and unknown keys are rejected, so a file that worked with 2.1.x can fail with 2.2.3. The console configuration is also checked before an install or upgrade, and an invalid configuration stops it.
***
**API behavior changes**
| Action | Old behavior | New behavior |
| :- | :- | :- |
| Send a non-zero `frequency_penalty` or `presence_penalty` | Accepted and ignored | Returns HTTP 400. `0` and `null` are accepted. Update clients that always send these parameters |
| Send `/v1/embeddings` input longer than the model's limit | Returns HTTP 500 | Returns HTTP 400 `context_length_exceeded` and names the input that is too long |
| Set `max_tokens` and `max_completion_tokens` to different values | Returns HTTP 500 | Returns HTTP 400 |
| Send a request to a retired model | Returns HTTP 410 with a plain-text body on `/v1/chat/completions` | Returns HTTP 410 with a `model_deprecated` JSON error and a `request_id`. See [SambaStack model lifecycle](/docs/en/sambastack/service-administration/model-deployment/model-lifecycle) |
| Send a `logit_bias` that the deployment cannot apply | Returns HTTP 500, or is ignored on some models | Returns HTTP 400 |
| Send a multimodal prompt over the context limit | Returns HTTP 500 | Returns HTTP 400 `context_length_exceeded` |
| Send a `gemma-4-31B-it` video longer than 60 seconds | Accepted, and sampled at a fixed 32 frames | Returns HTTP 400 with code `video_duration_exceeded`. Shorter videos are sampled at 2 frames per second |
***
**Component versions**
| Component | v2.1.2 | v2.2.3 |
| :- | :- | :- |
| Helm chart | 2.1.2 | 2.2.3 |
| Runtime | 1.26.3-2 | 1.26.3-2 |
| Kubernetes (RKE2) | 1.35 | 1.35 |
**No Kubernetes upgrade is required for this release.** RKE2 stays on 1.35.
***
**CVE remediation**
This release includes updated versions of third-party open source dependencies and base images, resolving a broad set of published security advisories (CVEs) affecting those components. No vulnerabilities were identified in SambaNova proprietary code.
***
### New features and enhancements
**Model updates**
Updated model and bundle support in SambaStack v2.2.3:
* **`gemma-4-31b-mtp-cd-8-32-64-128k` – Preview** – Runs `gemma-4-31B-it` with multi-token prediction and constrained decoding, at 8K to 128K context, with text, image, and video input. Multi-token prediction generates several tokens in each step, which speeds up output. Unlike speculative decoding, it does not need a separate draft model.
* **`minimax-m3-acb-prefix-caching` – Preview** – Runs `MiniMax-M3` with prompt caching, continuous batching, and constrained decoding, up to 1M context. It accepts text input only. For image or video input, use one of the other `MiniMax-M3` bundles.
See [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
***
**More transcription formats**
`/v1/audio/transcriptions` supports the `verbose_json`, `srt`, and `vtt` response formats, and `timestamp_granularities` with `segment`. Requests for `word` timestamps are rejected. Transcription and translation responses include usage data.
***
**API key expiry and rotation**
API keys can have an expiry date, and you can rotate a key without losing its name or usage history. You can set an expiry through the API on any deployment. By default, keys do not expire.
To show the expiry and rotation controls in the console, set both of these:
| Setting | Value |
| :- | :- |
| `cloud-ui.config.apiKeys.defaultExpiryDays` | A positive number |
| `apiKeyExpiry` flag | On |
***
### API improvements and fixes
**Cancellation for non-streaming requests**
Non-streaming requests are cancelled when the client disconnects, the same as streaming requests. Generation stops right away on continuous-batching bundles. See **Known issues** for the cases where it keeps running.
***
**Native tool-call format under constrained decoding**
With `tool_choice: required` or a named function, constrained decoding enforces each model's native tool-call format instead of bare JSON, and reasoning models can reason before the tool call.
***
**`seed` for vision-language models**
Setting `seed` gives reproducible output on vision-language models such as `gemma-4-31B-it`.
***
### Bug fixes
* **Gemma `logprobs`** – `logprobs: true` requests to `gemma-4-31B-it` and Gemma 3 bundles no longer return an internal error.
* **`MiniMax-M3` near the context limit** – Requests near the context limit no longer exceed it because of extra chat template tokens.
* **Truncated reasoning** – On reasoning models such as `MiniMax-M3`, when `max_tokens` cuts off a non-streaming response during reasoning, the text is returned in `reasoning`, not `content`.
* **Structured output** – For `response_format` requests, JSON is no longer taken from unfinished reasoning, and braces or reasoning markers inside JSON strings are handled correctly.
* **Large `response_format` schemas** – A large schema no longer pushes a request over its context limit.
* **Tool schemas that cannot be compiled** – A tool schema that cannot be compiled into a grammar no longer returns an internal error. The request is served with weaker or no grammar enforcement, so its tool call may not match the schema.
* **Batching** – Requests that set `logit_bias`, `seed`, `ignore_eos`, `logprobs`, `top_logprobs`, `tools`, `tool_choice`, `response_format`, `parallel_tool_calls`, or an alias of `stop` or `max_tokens` can share a batch with other requests.
* **API keys page** – The console API keys page (`/apis`) loads correctly.
***
### Deprecations and removals
**Support for the previous bundle resources ended on September 30, 2026.** The charts no longer render `BundleTemplate`, `Bundle`, or `BundleDeployment`. Move to `Model`, `ModelProfile`, `ModelDeployment`, and `ModelBundle`. For the field mapping and steps, see [Migrating bundle configuration](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/sambastack-v2-bundle-migration).
* **Air-gapped download script renamed** – `download_bundle_artifacts_v2.sh` is renamed `download_bundle_artifacts.sh` and handles `ModelBundle` manifests only. Update any runbooks or scripts that call the old name. The air-gapped values profile uses `modelBundleSpecs` and `modelDeploymentSpecs`.
***
### Known issues
* **`MiniMax-M3` prompt caching bundle limits** – `minimax-m3-acb-prefix-caching` accepts text only, and requests with `repetition_penalty` return an internal error. For image or video input, use another `MiniMax-M3` bundle. On the caching bundle, leave out `repetition_penalty`.
* **`MiniMax-M3` drops a second system message with images or video** – If a request with images or video starts with two `system` messages, the second one is ignored. Combine them into one.
* **`MiniMax-M3` returns HTTP 500 for some image URLs** – An `image_url` that does not start with `http://`, `https://`, or a lowercase `data:` returns HTTP 500 instead of HTTP 400. Send images as base64 `data:` URLs, with `data:` in lowercase.
* **`MiniMax-M3` ignores `thinking_mode` for text requests** – On text-only requests, `chat_template_kwargs.thinking_mode` has no effect, and `disabled` can return reasoning in `content`.
* **`gemma-4-31B-it` cannot fetch video from `https` URLs** – A `video_url` over `https` returns HTTP 400 `video_fetch_failed`. Send the video as a base64 `data:` URL.
* **`/v1/completions` fails for vision and audio models** – `MiniMax-M3`, `gemma-4-31B-it`, and other vision or audio models return an internal error on `/v1/completions`. Use `/v1/chat/completions`.
* **One rejected request can fail its whole batch** – If a request is rejected while its batch is being prepared, other requests in the same batch can get the same HTTP 400. For example, a `MiniMax-M3` request over the context limit, or a `gemma-4-31B-it` request with conflicting `max_tokens` values.
* **Some requests keep running after a disconnect** – These requests are not cancelled when the client disconnects:
* On bundles without continuous batching, a request that is not the longest in its batch. It keeps generating until the batch finishes.
* Streaming requests with a JSON `response_format` on reasoning models
* Requests that the global model router sends to another cluster
Set an output token bound to limit what an abandoned request uses.
* **Configuration check Jobs build up** – Each install or upgrade leaves a completed console configuration-check Job and pod. Delete them after you upgrade.
***
## SambaStack v2.1.2 release
**Release Date:** September 22, 2026
This release introduces the following key updates:
* **Output token bounds** – New `maxNewTokens` and `maxNewTokensHard` settings, applied on a model, a service tier, or one context group within a tier
* **Abandoned generations are cancelled** – A streaming client that disconnects stops its generation. Capacity returns immediately with continuous batching or a batch size of 1, otherwise when the rest of the batch finishes
* **Accurate `completion_tokens` on vision-language models** – Reported output tokens count what the model generated, not a re-encode of the response text
* **`minimax-m3-512k` bundle** – `MiniMax-M3` served with its context capped at 512K
* **Client sampling parameters in the inference log record** – Seven sampling parameters are recorded as the client sent them
### Upgrade notes
Review the items below, then follow [Upgrading SambaStack](/docs/en/sambastack/service-administration/platform-admin/upgrade-sambastack) for the upgrade procedure. Back up any self-installed applications first — SambaNova cannot back up or restore data for customer-installed applications.
This hotfix changes no defaults. Output token bounds are unset until you set them.
***
**Component versions**
| Component | v2.1.1 | v2.1.2 |
| :- | :- | :- |
| Helm chart | 2.1.1 | 2.1.2 |
| Runtime | 1.26.3-2 | 1.26.3-2 |
| Kubernetes (RKE2) | 1.35 | 1.35 |
**No Kubernetes upgrade is required for this release.** Only a Helm chart upgrade is required for this release.
***
**CVE remediation**
This release includes updated versions of third-party open source dependencies and base images, resolving a broad set of published security advisories (CVEs) affecting those components. No vulnerabilities were identified in SambaNova proprietary code.
***
### New features and enhancements
**Output token bounds**
Two settings bound how many output tokens a generation can produce:
* `maxNewTokens` supplies `max_tokens` when a request omits it. A request that sends its own is unaffected.
* `maxNewTokensHard` caps every generation, with or without a client `max_tokens`. A generation that reaches the cap stops there and reports the finish reason a client-supplied limit produces.
Either can be set on a [Model](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model) resource, on a service tier, or on a `contextGroups` entry inside a tier. contextGroup takes precedence over service tier, which takes precedence over Model.
See [Output token bounds](/docs/en/sambastack/service-administration/platform-admin/usergroup-management#output-token-bounds) for choosing a level and [SambaStack.yaml reference](/docs/en/sambastack/resources/sambastack-yaml#servicetiers) for the keys.
***
**Model updates**
* **`minimax-m3-512k` – Preview** – Serves `MiniMax-M3` capped at 512K context, with vision. Use `minimax-m3-32-64-128-256-512k-1m` for context above 512K. See [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
***
**Client sampling parameters in the inference log record**
The inference log record reports the client's sampling parameters: `max_tokens`, `temperature`, `top_p`, `stop`, `top_k`, `repetition_penalty`, and `ignore_eos`. A null means the client sent no value. Where `maxNewTokensHard` is set and the client's `max_tokens` exceeded it, the log reports the cap.
Two settings control what is recorded:
| Setting | Effect |
| :- | :- |
| `LOG_REQUEST_PARAMS` | Set to `false` to record none of the parameters |
| `LOG_REQUEST_PARAMS_EXCLUDE` | A list of individual parameters to leave out, for example `stop`, the only one that can carry text drawn from a prompt |
An excluded parameter is absent from the record rather than null.
***
### API improvements and fixes
**Abandoned generations are cancelled**
A streaming client that disconnects now stops its generation. When the capacity returns depends on how the model is served:
| Serving configuration | Capacity returns |
| :- | :- |
| [Continuous batching](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deployment-configurations) | Immediately |
| Static batching, batch size 1 | Immediately |
| Static batching, batch size above 1 | When the rest of the batch finishes |
A cancelled request reports `stop_reason: cancelled` with the tokens generated before the stop, and is not passed to another expert. Cancellations are held for `cancelled_in_flight_ttl_seconds` (default 300), which covers a request a worker has already picked up.
Non-streaming requests are not covered — see **Known issues**.
***
**`completion_tokens` reflects what the model generated**
On vision-language models, `usage.completion_tokens` counts the tokens the model generated. It previously re-encoded the response text, which could overstate the count.
A stop sequence that truncates output mid-step now counts the dropped tokens, which the model did generate.
***
### Known issues
* **Requests near the context limit on `minimax-m3-512k` fail with an internal error** – The backend counts 49 more prompt tokens than the token-count endpoint reports, so a text request within 49 tokens of the 512K limit is accepted and then fails. A request at or above the limit is still rejected with HTTP 400. Keep prompt plus `max_tokens` 50 tokens under 524,288.
* **Requests near the 128K boundary on `dyt-gpt-oss-120b-8-32-64-128k-scx` fail with an internal error** – The worker counts 14 more tokens than the gateway, so a prompt counted at 131,070 or 131,071 is accepted and then fails before generating. Keep prompts 15 tokens under 131,072.
* **Conflicting `max_tokens` and `max_completion_tokens` return an internal error** – On a vision-language model such as `gemma-4-31B-it`, sending both fields with different values returns HTTP 500 instead of a validation error when no output token bound is configured. Send one field only. Setting `maxNewTokens` or `maxNewTokensHard` also avoids it: the pair is resolved before the model sees it.
* **A truncated thinking block is returned as content on `gemma-4-31B-it`** – In a non-streaming request, if `max_tokens` cuts a thinking block short, the thought marker and reasoning text arrive in `message.content` with no reasoning field. Token counts stay correct. Stream the response, or raise `max_tokens` so the block completes.
* **Cancellation on disconnect covers streaming requests only** – An abandoned non-streaming request is not detected: it generates to its limit and holds its slot, with no `stop_reason: cancelled` record. Stream long generations, and set an output token bound to cap what an abandoned request consumes.
***
## SambaStack v2.1.1 release
**Release Date:** August 31, 2026
**This release requires action before you upgrade.** Work through **Upgrade notes** first. If you still use the previous bundle resource model, note the migration deadline under **Deprecations and removals**.
This release introduces the following key updates:
* **`MiniMax-M3`** – New model serving text, image, and video from 32K to 1M context, with function calling and reasoning support
* **Workload scheduling controls** – Platform services can be kept off inference nodes through a new set of `global.*` scheduling values
* **Constrained decoding on vision models** – Now applied during generation, where it was previously accepted and silently ignored
* A set of API, air-gapped install, and reliability fixes
For deployment details, bundle configurations, and context lengths, see [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles). The model, API, and inference changes below apply to SambaCloud as well.
SambaRack Manager ships its own changes in this release, including the `snctl` Go rewrite and the flag changes it brings. See [SambaRack Manager release notes](/docs/en/release-notes/sambarack-manager).
### Upgrade notes
Review the items below, then follow [Upgrading SambaStack](/docs/en/sambastack/service-administration/platform-admin/upgrade-sambastack) for the upgrade procedure. Back up any self-installed applications first — SambaNova cannot back up or restore data for customer-installed applications.
**API behavior changes**
| Change | Action |
| :- | :- |
| API error responses now use a JSON error envelope | Update any client that parses error bodies rather than relying on HTTP status codes. |
| `Mistral-Large-3` chat template updated | Re-validate prompts and evaluations pinned to the previous template. |
| Malformed image input returns HTTP 400 instead of HTTP 500 | Treat these as permanent request errors rather than retryable server errors. |
***
**Component versions**
| Component | v2.0.2 | v2.1.1 |
| :- | :- | :- |
| Helm chart | 2.0.2 | 2.1.1 |
| Runtime | 1.26.3-2 | 1.26.3-2 |
| Kubernetes (RKE2) | 1.35 | 1.35 |
**No Kubernetes upgrade is required for this release.** RKE2 stays on 1.35, so unlike v2.0.2 you do not need to plan a cluster upgrade window.
***
**CVE remediation**
This release includes updated versions of third-party open source dependencies and base images, resolving a broad set of published security advisories (CVEs) affecting those components. No vulnerabilities were identified in SambaNova proprietary code.
***
### New features and enhancements
**Model updates**
Updated model and bundle support in SambaStack v2.1.1:
* **`MiniMax-M3` – Preview** – A new model serving text, image, and video, available in the `minimax-m3-32-64-128-256-512k-1m` bundle at 32K, 64K, 128K, 256K, 512K, and 1M context, and in a dedicated 32K bundle. Function calling, reasoning output, and reasoning-token reporting are supported, and the model is available through the Responses API. Prompt caching is not supported on this model in this release — see **Known issues** below.
* **`MiniMax-M2.7` – Stable** – The model reaches Stable with KV cache integration, the last planned improvement for it. A new `dyt-minimax-m2p7-32k-pc` bundle serves the model at 32K with prompt caching.
A request that exceeds the KV sequence limit of the context length it targets is served by a larger expert through dynamic tiering rather than rejected. Such a request returns HTTP 200 with `finish_reason: length`.
***
**Workload scheduling controls**
Platform services can now be kept off inference (RDU) nodes. Scheduling is configured through a set of `global.*` values applied to every Deployment, StatefulSet, and Job in the chart:
* `global.affinity`
* `global.nodeSelector`
* `global.tolerations`
* `global.topologySpreadConstraints`
* `global.priorityClassName`
* `global.requireDefaultPodAntiAffinity`
Previously the gateway, inference router, operator, authentication, identity, Keycloak, database, console, and job workloads had no scheduling constraints and could be placed on inference nodes. When such a node failed or saturated, the kubelet could not report pod status, pods were left terminating, and the cluster did not recover automatically.
Any component can still override a single key for itself. `tolerations` and `topologySpreadConstraints` append to a component's own list rather than replacing it.
***
**Air-gapped and BYOC installs**
* **Inline profile definitions are resolved** – The artifact download path now reads PEFs from inline `profileDefinition` blocks as well as named `ModelProfile` references. Previously 312 PEF references across 27 bundles were skipped, so installs staged an incomplete artifact set, which crash-looped the cache pod and left workers in init.
* **The standalone models chart is detected automatically** – The download script now recognizes the standalone models chart, which is the default when `modelComponents.enabled` is `false`, in addition to the umbrella chart.
* **Third-party chart dependencies are mirrored** – Redis, OpenEBS, and CloudNativePG subcharts are mirrored into the private registry alongside images, so installs that previously could not resolve subchart dependencies now can.
***
### API improvements and fixes
**Constrained decoding now applied on vision models**
Constrained decoding is now applied during generation for vision models. Previously it was accepted and silently ignored: the grammar was compiled for each request and passed through, but the vision task never loaded the mask, so `tool_choice: required` and `response_format` requests fell through to unconstrained generation with best-effort output parsing and no error or warning.
This resolves the `gemma-4-31B-it` forced tool-call failure reported as a known issue in v2.0.2.
***
**Error responses use a JSON envelope**
**Backwards-incompatible.** Errors returned across the model-router endpoints now use a consistent JSON error envelope rather than the previous mixed formats. Clients that parse error bodies need updating. Clients that rely on HTTP status codes are unaffected.
***
**Tool-call fixes**
* **`tool_choice: none` on vision models** – Now honored on the vision model paths, where it was previously ignored.
* **`gpt-oss-120b` streaming** – Streaming tool calls no longer emit the model's raw channel scaffolding as visible assistant text in `delta.content`, and tool arguments are delivered once as a `tool_calls` delta instead of twice.
* **`gpt-oss-120b` code-mode tool calls** – Code-mode tool calls are now parsed and served rather than rejected with HTTP 400.
* **`finish_reason`** – Fixed a token-fallback defect that caused every request to report `finish_reason: tool_calls`.
***
**Reasoning output**
Reasoning output is now delivered consistently across `MiniMax-M2`, `gemma-4-31B-it`, `gpt-oss-120b`, and the DeepSeek V3 family (`DeepSeek-V3-0324`, `DeepSeek-V3.1`, `DeepSeek-V3.2`), streaming and non-streaming. `MiniMax-M3` reports reasoning tokens and supports thinking-mode selection.
***
**Image input on `/v1/responses`**
Requests to `/v1/responses` that carry an image are now served correctly. Previously they returned HTTP 500 or silently dropped the image.
***
**`/v1/models` is no longer cached at the edge**
`/v1/models` and `/v1/models/` now return `Cache-Control: no-store`, so the listing reflects current model availability instead of a value up to four hours old. Clients that relied on edge caching should cache the response themselves.
***
**Token accounting**
* **Reasoning tokens on vision-served models** – `usage.completion_tokens_details` is now populated for reasoning models served through the vision pipeline, such as `gemma-4-31B-it`.
* **Image and video token counting** – Expert routing now counts image and video tokens. This resolves the `gemma-4-31B-it` HTTP 500 near the 32K boundary reported as a known issue in v2.0.2, where routing counted text tokens only.
***
**Independent sampling parameters on co-batched `qwen3-tts` requests**
Co-batched talker requests now carry independent `temperature`, `top_k`, `top_p`, and `seed` values. Requests with differing sampling parameters were previously rejected with HTTP 400 when they happened to be batched together.
***
### Bug fixes
* **HTTP 500 on prompts near an expert's sequence ceiling** – A prompt that landed within the batching window of an expert's token-generation ceiling returned HTTP 500. Because the sequence dimension is resolved for the whole batch, one such prompt failed every request batched with it. Affects `MiniMax-M2.7`, `gpt-oss-120b`, and `gemma-4-31B-it`.
* **`prompt_tokens` under concurrent load** – Fixed the vision pipeline attributing prompt tokens to the wrong in-flight request, which made usage accounting inaccurate. This resolves the `prompt_tokens` mismatch reported as a known issue in v2.0.2.
* **Vision embedding corruption** – Fixed a padding defect that could silently garble image and video embeddings, producing incorrect answers rather than an error. Multiple image and video page groups are now packed into a single vision-encoder call.
* **Vision prefill padding** – Vision prefill is padded to the sequence-length bucket the runtime selected rather than the maximum, preventing a shape-mismatch crash on non-fixed sequence ranges.
* **Clusters with no models** – Fixed the API server and authentication and billing services crash-looping when no model configuration was present, which blocked reinstalls.
***
### Deprecations and removals
**The previous bundle resource model reaches end of support on September 30, 2026**, as announced in v2.0.2 — one month after this release. `BundleTemplate`, `Bundle`, and `BundleDeployment` still function until that date. Migrate to `Model`, `ModelProfile`, `ModelDeployment`, and `ModelBundle` before then. For the field mapping and the procedure, see [Migrating bundle configuration](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/sambastack-v2-bundle-migration).
* **`sn-redis-exporter` removed.** The Deployment, Service, and ServiceMonitor templates, the values block, and the on-premises override are all removed. Remove any dashboard or alert that targets it.
* **`modelConfigGenerator.enabled` values removed.** Delete the keys from your values file. The model-config generator is now always injected into the authentication and billing and gateway deployments.
* **v1 air-gapped artifact download script removed.** It is fully superseded by `download_bundle_artifacts_v2.sh`. Update any runbook or automation that calls the previous script.
***
### Known issues
* **Prompt caching is not supported on `MiniMax-M3`** – The model is available in Preview for text, image, and video, but prompt caching does not take effect on it in this release. Requests succeed; they are served without a cache hit. Use another prompt-caching bundle if you depend on cached prefixes.
* **Constrained decoding is not available on `MiniMax-M3`** – The bundle ships with constrained decoding disabled, so `response_format` (`json_schema` and `json_object`) is not enforced during generation on this model, and a forced `tool_choice` falls back to best-effort parsing of the model output. Use a constrained-decoding capable model if you need guaranteed structured output.
* **`swappable` applied per expert, not per checkpoint** – When two experts share a PEF but set different `swappable` values, both use the value from whichever expert was configured first. Set `swappable` consistently across experts that share a PEF.
***
## SambaStack v2.0.2 release
**Release Date:** August 5, 2026
**This release contains breaking changes.** Work through **Breaking changes** below before you upgrade.
This release introduces the following key updates:
* **New resource model** – `Model`, `ModelProfile`, `ModelDeployment`, and `ModelBundle` replace `BundleTemplate`, `Bundle`, and `BundleDeployment`
* **Infrastructure upgrades** – Kubernetes 1.30 → 1.35, plus Keycloak, CloudNativePG, OpenEBS, and Redis
* **Standalone models chart** – Model definitions now ship as a separate `sambastack-models` Helm release
* **Structured output enforcement** – `json_schema` and `json_object` are enforced during generation rather than validated afterward for constrained decoding enabled models
* **Text-to-speech improvements** – Multi-batch streaming and automatic language detection
* **Model updates** – `gemma-4-31B-it` gains image and video input at 256K context
* A set of API, deployment, and reliability fixes
For deployment details, bundle configurations, and context lengths, see [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles). The model, API, and inference changes below apply to SambaCloud as well.
### Breaking changes
| Change | Action |
| :- | :- |
| Kubernetes (RKE2) 1.30 → 1.35 | Plan an upgrade window with SambaNova support and back up self-installed applications. |
| The platform chart no longer creates model resources | Install the `sambastack-models` release as part of your upgrade. |
| `/v1/models` no longer returns `groups` or `display` | Read category information from `sn_metadata.category` instead. |
| `db-admin` component removed | Move any `db-admin.admins` list to `auth-and-billing.admins` and delete the `db-admin:` block. A leftover block is ignored silently, so admins lose Admin UI access with no error. Migrations, identity, and credits are now owned by the backend service. |
| Four model-router endpoints removed | Update tooling that calls `/v1/serving_config`, `/v1/check_auth`, `/v1/deployments`, or `/v1/bundle-templates`. |
| `cd-gemma-4-31b-32-128-256k` renamed to `gemma-4-31b-32-128-256k` | Update deployments referencing the old bundle name. |
### Infrastructure upgrades
**Major breaking changes – disruptive release.**
* All future infrastructure and model releases will only be supported on the new Kubernetes version 1.35 going forward.
* Back up all self-installed applications before upgrading. SambaNova cannot back up or restore data for customer installed applications.
* Contact your SambaNova support team for upgrade documentation and assistance planning your upgrade window.
This release upgrades core platform infrastructure:
* **Kubernetes (RKE2)** – 1.30 → 1.35
* **Keycloak** – 26.3.1 → 26.7.0
* **CloudNativePG** – 0.23.2 → 0.29.0 (Postgres 15 → Postgres 18)
* **OpenEBS** – 4.3.3 → 4.5.1
* **Redis** – 23.1.1 → 27.0.15 (Redis 8.2.1 → 8.8.0)
**CVE remediation**
This release includes updated versions of third-party open source dependencies and base images, resolving a broad set of published security advisories (CVEs) affecting those components. No vulnerabilities were identified in SambaNova proprietary code.
### New features and enhancements
**New resource model for model deployment**
SambaStack v2.0.2 introduces four composable custom resources:
* `Model`
* `ModelProfile`
* `ModelDeployment`
* `ModelBundle`
They replace:
* `BundleTemplate`
* `Bundle`
* `BundleDeployment`
A `ModelDeployment` can reference a model and a profile directly, so you no longer need a bundle to serve a model. The previous format remains functional, and both can coexist in the same cluster. See [Migrating bundle configuration](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/sambastack-v2-bundle-migration) for how the resources fit together.
**Previous resource model deprecation.** The previous resource model format is deprecated; however, it remains functional until **September 30, 2026**. Use the new format for new configurations and migrate existing bundles before that date. For the field mapping and the migration procedure, see [Migrating bundle configuration](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/sambastack-v2-bundle-migration).
***
**Standalone models chart**
Model definitions have moved out of the platform chart into a separate `sambastack-models` Helm release, so you can change which models you serve without touching the platform. A deployment is now three releases, installed in this order:
1. `sambastack-base`
2. `sambastack`
3. `sambastack-models`
**The platform chart no longer creates model resources.** Install the `sambastack-models` release as part of your upgrade, or your cluster will come up with no `Model`, `ModelProfile`, `ModelBundle`, or `Pef` resources.
For the chart layout and install order, see [Models chart](/docs/en/sambastack/service-administration/model-deployment/models-chart).
***
**Model updates**
Updated model and bundle support in SambaStack v2.0.2:
* **`gemma-4-31B-it`** – The `gemma-4-31b-32-128-256k` bundle now serves text, image, and video up to 256K in one bundle, with constrained decoding enabled.
* **`qwen3-tts`** – The talker bundle now supports batch sizes 2, 4, 8, and 16, up from 2.
* **Vision capabilities** – `gemma-3-12b-it`, `gemma-3-27b-it`, and `gemma-4-31B-it` now advertise text and vision capabilities through the v1/models API. This is a metadata correction; inference behavior is unchanged.
A request that exceeds the KV sequence limit of the context length it targets is served by a larger expert through dynamic tiering rather than rejected. Such a request returns HTTP 200 with `finish_reason: length`.
***
**Text-to-Speech / Audio API improvements**
The Qwen3-TTS pipeline behind `/v1/audio/speech` now supports concurrent generation.
* Multi-batch streaming generation at batch sizes 2, 4, 8, and 16.
* `language: auto` detects the input language instead of requiring an explicit value.
* Requests route to purpose-built vocoder experts based on output length.
* Lower audio transfer overhead in the generation path.
***
**Prompt caching**
Prompt caching is now backed by a dedicated KV cache store in the inference pipeline rather than per-request recomputation.
* Requests stay pinned to the worker already holding their cache instead of being routed to an idle worker and losing the cache hit.
* `repetition_penalty` is now honored on cached requests. Previously the value was ignored and treated as `1.0`.
***
### API improvements and fixes
**`/v1/models` changes**
**Backwards-incompatible.** The `groups` array and the per-model `display` object have been removed from the model listing response. Category information is available through `sn_metadata.category`.
***
**Structured output enforced during generation**
For constrained decoding enabled models, `json_schema` and `json_object` response formats are now enforced through constrained decoding while the response is generated, rather than validated after generation completes. The `json_object` grammar is limited to object literals, matching the OpenAI definition, instead of accepting any valid JSON value.
***
**`encoding_format` on `/v1/embeddings`**
The `/v1/embeddings` endpoint now accepts the OpenAI-compatible `encoding_format` parameter, supporting `float` and `base64` responses.
***
**Streaming truncation now OpenAI-compliant**
When a streaming tool-call or structured-output response is cut short by `max_tokens`, the response now retains `finish_reason: length` and the usage object.
***
**Tool-call fixes**
* **Forced `tool_choice`** – Responses produced under a forced tool choice now report `tool_calls` as the stop reason.
* **MiniMax streaming tool calls** – Tool calls are no longer dropped when the model emits text before the call.
* **`gemma-4-31B-it` tool calling** – Tool calling now succeeds in the Responses API. Forced tool calls under constrained decoding are a known issue in this release — see **Known issues** below.
* **`Llama-4-Maverick-17B-128E-Instruct`** – A null function name is now treated as no tool call instead of returning HTTP 400.
***
**`logprobs` for multimodal models**
`logprobs` and `top_logprobs` are now supported through the vision-language pipeline for `gemma-4-31B-it`.
***
**Multimodal input validation**
The `content` field is now optional in multimodal input validation, so requests that supply only non-text parts are accepted.
***
### Bug fixes
* **`gemma-4-31B-it` crash on vision input** – Fixed a buffer shape mismatch that crashed vision requests. Vision prefill is now padded to the sequence-length bucket the runtime selected rather than the maximum.
* **`MiniMax-M2.7` garbage output** – Fixed a batching defect that emitted repeated-byte garbage on requests with no cache hit.
* **`gemma-4-31B-it` token accounting** – Fixed a null `prompt_tokens_details` propagating into usage records.
* **`gemma-4-31B-it` tool-result images** – Media placeholders are now emitted in the chat template for images returned by tools.
* **Persistent errors after a router restart** – Fixed the router returning HTTP 500 indefinitely when it started before its configuration was available. The configuration loader failed silently while health checks continued to report healthy.
* **Bundle deployment names** – Names containing `.` are now rejected with an actionable error instead of failing validation deep inside the operator. Bundle name length is capped so generated cache service names stay within Kubernetes limits.
***
### Deprecations and removals
* **`db-admin` component removed.** Database migrations, identity, and credits are now owned by the backend service. Admin emails move from the top-level `db-admin:` block to `auth-and-billing.admins`:
```yaml theme={}
# Before (v1.x) — no longer has any effect
db-admin:
admins:
- abc@example.com
# After (v2.0.2+)
auth-and-billing:
admins:
- abc@example.com
```
A leftover `db-admin:` block does not fail the Helm upgrade — it is ignored. The install appears to succeed while every email in that list silently loses Admin UI access. Move the list before you upgrade. See [User access and permissions](/docs/en/sambastack/service-administration/platform-admin/usergroup-management#add-an-administrator).
* **Four model-router endpoints removed:** `/v1/serving_config`, `/v1/check_auth`, `/v1/deployments`, and `/v1/bundle-templates`. None were part of the documented public API. The `X-SN-AUTH-EMAIL` header is no longer read.
* **`cd-gemma-4-31b-32-128-256k` is replaced by `gemma-4-31b-32-128-256k`.** The 256K `gemma-4-31B-it` bundle no longer carries the `cd-` prefix, since constrained decoding is now standard across its tiers. Update any deployment that references the previous name.
***
### Known issues
**`gemma-4-31B-it` tool calling fails under constrained decoding.** Requests to `gemma-4-31B-it` that force a tool call will fail with `adapter_mapping_failed`. The model's tool-call output is parsed with a generic JSON parser instead of the Gemma 4 parser, so the call cannot be decoded.
* **Affected** – Forced `tool_choice` (`required` or a named tool), parallel tool calls, nested argument schemas, and tools that take no arguments. All of `/v1/chat/completions`, `/v1/responses`, and `/v1/messages` are affected, streaming and non-streaming.
* **Not affected** – `tool_choice: auto` requests that produce no tool call, and `tool_choice: none`. Every other model is unaffected, including tool calling on models that do not use constrained decoding.
* **What to do** – If you need tool calling on `gemma-4-31B-it` with constrained decoding, hold off on this release until an upcoming hotfix. If you do not, v2.0.2 is safe to deploy.
* **`gemma-4-31B-it` `prompt_tokens` mismatch under load** – Some responses report a `prompt_tokens` value belonging to a different in-flight request, so usage accounting may be inaccurate. Completions are unaffected. Observed at batch sizes 4 and 8, not at batch size 2. A fix is targeted for the next release.
* **`gemma-4-31B-it` HTTP 500 on images near the 32K boundary** – Sequence-length routing counts text tokens only and excludes image tokens, roughly 258 per image. A request whose image-inclusive total exceeds 32,768 is routed to the 32K slot and fails with HTTP 500 `unexpected_error`. Reduce the image count or text length to stay under 32K.
* **`swappable` applied per expert, not per checkpoint** – When two experts share a PEF but set different `swappable` values, both use the value from whichever expert was configured first. Set `swappable` consistently across experts that share a PEF.
***
## SambaStack v1.2.0 release
**Release Date:** July 7, 2026
This release introduces the following key updates:
* **Text-to-Speech / Audio API** – New `/v1/audio/speech` endpoint
* **Prompt Caching** – Reduced cost and latency for requests sharing a common prefix
* **Mistral Large 3 675B** – Function calling support, context length extended to 32K
* **MiniMax M2.7** – New prompt-caching bundles
* **Gemma 4 31B** – Video understanding up to 128K context, or text-only up to 256K context (separate bundles, not combined)
* A substantial set of security, authentication, and API improvements
For full deployment details, bundle configurations, and context length options for all models and bundles mentioned below, see [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
### New features and enhancements
**Text-to-Speech / Audio API**
SambaStack now supports audio speech generation via a new `/v1/audio/speech` endpoint, backed by the Qwen3-TTS model family (talker + vocoder).
* Generate speech from text following the OpenAI audio speech API shape.
* Optional `language` parameter with validation – unsupported languages return a 400 with the list of supported options.
* Optional `sampling_params` object (`temperature`, `top_p`, `top_k`, `seed`) to control generation style.
***
**Prompt Caching**
Prompt caching reduces cost and latency for requests that share a common prefix (long system prompts, document Q\&A, multi-turn conversations).
* The `pricing` object in `/v1/models` now includes `input_cache_read` and `input_cache_write` fields showing per-token cache pricing.
* The usage object in responses reports `cached_tokens` separately from `input_tokens`.
* **MiniMax M2.7 prompt-caching bundles** are now available in SambaStack.
***
**Model updates**
Added support for the following models in SambaStack 1.2.0:
* **Mistral Large 3 675B** (`Mistral-Large-3-675B-Instruct-2512`) – Function calling support added and context length support extended to 32K.
* **MiniMax M2.7** (`MiniMax-M2.7`) – New prompt-caching bundles with default sampling enabled.
* **Gemma 4 31B** (`gemma-4-31B-it`) – Now supports video understanding up to 128K context via the `gemma-4-31b-32-128k` bundle, or text-only context extended to 256K via the separate `cd-gemma-4-31b-32-128-256k` bundle. The two are not combined – the 256K bundle is text-only and does not support image or video input.
* **Qwen3-TTS** (`qwen3-tts`) – Talker and vocoder models for the new `/v1/audio/speech` endpoint.
***
**Model CRs now public**
All Model Custom Resources are now rendered with `public: true` in the SambaStack chart, making them visible via API discovery without requiring operator intervention.
***
**Model-level fields**
Added support for model-level fields defined in Model Custom Resources. These fields apply automatically to every deployment of that model, so you no longer need to set them in bundles.
***
**Standalone authentication service**
The authentication service can now be deployed independently from the main backend, enabling finer-grained scaling and upgrade control.
***
### API improvements and fixes
**Streaming fix: tool-call content**
Tool-call requests were fully buffered instead of streamed, affecting all deployed model families. This release restores true incremental streaming for tool-call requests.
***
**Messages API (`/v1/messages`) fixes**
* **`stop_reason`** – Now correctly returns `"tool_use"` when the model invokes a tool.
* **`disable_parallel_tool_use`** – Now correctly enforced.
* **`role: "system"`** – The `messages[]` array now accepts `role: "system"` for MCP client compatibility.
* **Auth error shape** – Authentication errors now return the Anthropic-standard error envelope.
***
**`ignore_eos` parameter**
Requests can now pass `ignore_eos: true` to continue generation past the model's end-of-sequence token. That feature is disabled by default for security reasons and needs to be enabled in the bundle deployment.
***
**`response_format` fix for reasoning models**
Structured output parsing now uses reasoning-stripped completion text, preventing `` tokens from corrupting JSON extraction on Gemma 4 and other reasoning models.
***
### Bug fixes
* **Service-tier defaults** – Fixed an issue where some models weren't included in any service tier by default, requiring manual setup before they could serve requests. All models are now included by default.
* **Vision model path substitution** – Fixed an issue with path substitutions for vision models.
* **Gemma 4 tool calling in custom bundles** – Fixed an issue causing Gemma 4 to return malformed tool calling responses in custom bundles.
* **Multimodal pipeline** – Fixed the `reasoning` field being dropped during multimodal message preprocessing.
* **Thinking prefix** – Fixed spurious thinking prefixes being emitted on responses that don't require them.
* **Liveness probe** – Liveness probes are now kept alive during long single-request generation runs, preventing false-positive restarts when the model is actively generating a response.
* **Qwen3-TTS audio artifacts** – Upgraded PyTorch 2.4 → 2.7 to eliminate blast noise on generated audio.
* **Invalid bundle resources** – Fixed default bundle resource specs where `hugepages` were specified without `cpu`/`memory` limits.
* **Tiktoken cache for airgap** – Tiktoken vocabulary files are now cached for airgapped GPT-OSS deployments.
***
### Known issues
* **Constrained decoding on Gemma 4 31B** – Only available with text-only input.
* **Qwen3-TTS mid-stream audio artifacts** – Intermittent audio glitches occur in clips longer than approximately 60 seconds. For lengthy synthesis jobs, artifact frequency increases with clip length. Clips under 60 seconds are generally unaffected.
* **Qwen3-TTS long-input audio degradation** – For text inputs of approximately 1,000 words or more, audio quality may degrade mid-generation, in some cases collapsing to near-silence or garbled output. Keep individual TTS requests under 800 words for best results; split longer content into multiple requests.
* **Qwen3-TTS missing voice validation** – Requests with an invalid or missing `voice` field return HTTP 200 with audio output instead of a 4xx error.
* **DeepSeek-V3.2 `response_format`** – Structured output requests (`response_format: json_object` or `json_schema`) may fail for DeepSeek-V3.2. This is an ongoing regression from the previous release.
* **`70b-3dot3-ss-full-whisper` fails to deploy** – This bundle fails to start due to a recurring startup probe timeout. Redeploy attempts do not resolve the issue.
***
## SambaStack v1.1.1 release
**Release Date:** May 27, 2026
This release introduces the Anthropic Messages API (`/v1/messages`), four new SambaStack models (`Mistral-Large-3-675B-Instruct-2512`, `gpt-oss-20b`, `gemma-4-31B-it`, `MiniMax-M2.7`), migrates bundle configuration into PEF Custom Resources, and remediates several outstanding CVEs.
For full deployment details, bundle configurations, and context length options for all models and bundles mentioned below, see [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
**Backwards-incompatible changes.**
This release removes the `resubmit` bundle mechanism and renames model `Qwen3-235B` to its full name `Qwen3-235B-A22B-Instruct-2507`. See the inline warnings in **New models** and **Bundle configuration changes** before upgrading.
### New features and enhancements
**Anthropic Messages API (`/v1/messages`)**
A new Anthropic-compatible Messages endpoint is now available. SambaStack operators can now serve Anthropic-formatted traffic directly, without requiring translation to the OpenAI API format.
***
**New models**
Added support for the following models in SambaStack 1.1.1:
* **Mistral Large 3 675B** (`Mistral-Large-3-675B-Instruct-2512`)
* **gpt-oss 20B** (`gpt-oss-20b`)
* **Gemma 4 31B** (`gemma-4-31B-it`) - with function calling and reasoning-parser support
* **MiniMax M2.7** (`MiniMax-M2.7`)
**`Qwen3-235B` renamed to `Qwen3-235B-A22B-Instruct-2507`.** Update client model names from `Qwen3-235B*` to `Qwen3-235B-A22B-Instruct-2507`.
**1B draft model experts marked `private: true`.** Confirm no client code addresses these experts as public models.
***
**New PEFs**
* **PEF-1914 (gpt-oss-fp8-ss131072-bs8-dyt-1-cd:2)** for the existing `gpt-oss-120b` model - more performant version of gpt-oss-fp8-ss131072-bs8-dyt-1-cd with added support for `logit_bias`.
* **PEF-1910 (minimax-m2p5-ss196608-bs2-dyt-1-cd:1)** provides up to 192k context length with structured output support for both `MiniMax-M2.5` and `MiniMax-M2.7`.
***
**Bundle configuration changes**
This release continues moving deployment configuration from bundles into PEF Custom Resources.
* **Checkpoint sharing is now configured automatically based on PEF CRs.** `checkpoint_sharing_uuid` is always sourced from the corresponding PEF CR; prebuilt bundles no longer override it, and user-defined bundles should not override it either.
* `continuous_batching` and `constrained_decoding` are now configured on PEF CRs rather than bundles.
* `apply_default_sampling_params` added as an expert-level field in `BundleTemplates`. When set to `True`, sampling-parameter values are populated from the model's `generation_config.json`.
**Resubmit mechanism removed from bundles.** Remove `resubmit_to`, `resubmit_tool`, and `enableResubmit` from custom bundles.
***
### API improvements and fixes
**`logit_bias` parameter**
The `logit_bias` parameter is now supported in chat completions.
* Accepts a JSON object mapping token IDs to bias values in the range -100 to 100, matching the [OpenAI specification](https://platform.openai.com/docs/api-reference/chat/create#chat-create-logit_bias).
* Applies to text-generation models only - not supported for high throughput models or multimodal models.
***
**`response_format` accepts `null`**
The API-server validator now accepts `null` for `response_format`, matching OpenAI's looser shape.
***
**Invalid request error formats improved**
Error messages on bad input are clearer, and error responses now align with the OpenAI standard. `http error` was renamed to `api http error` for cross-platform consistency.
Example response for an out-of-range `logit_bias` token ID:
```json theme={}
{
"error": {
"message": "Token id 99999999 is out of the vocabulary range.",
"type": "invalid_request_error",
"param": "logit_bias",
"code": "invalid_value"
},
"request_id": "d7tfhubndjoecjv2ao50"
}
```
***
### Bug fixes
**`modelSpecs` overrides handling**
`modelSpecs` overrides now correctly apply zero-value fields (empty lists, scalar `false`, empty strings, zero ints). Previously these were silently skipped during the helm merge. For example:
```yaml theme={}
configs:
modelSpecs:
- "*"
- name: example-model
metadata:
category: [] # empty list now applied
provider: "" # empty string now applied
price:
input_tokens: 0 # zero int now applied
output_tokens: 0
public: false # scalar bool now applied
```
***
**`mm_token_type_ids` handling**
`mm_token_type_ids` added by transformers 5.5.3 is now handled correctly.
***
**CVE remediation**
This release includes fixes for multiple critical CVEs and security hardening improvements. Specific CVE references will be added after vulnerability-scan review.
***
### Known issues
* Mistral Large 3 675B (`Mistral-Large-3-675B-Instruct-2512`) currently has issues with function calling accuracy.
* gpt-oss-20b and gpt-oss-120b will fail to deploy in airgapped environments. Please contact your SambaNova admin for assistance with deploying these models in airgapped environments.
***
## SambaStack v1.0.57 release
**Release Date:** April 30, 2026
This release introduces new model support (Gemma 3 12B, Gemma 3 27B, DeepSeek V3.2), high-throughput configurations for DeepSeek models, constrained decoding support for GPT-OSS, more accurate and informative bundle legalizer responses, and major API additions including the OpenAI Responses API, the `n` and `seed` parameters, improved TTFT measurement, and OpenAI-conformant error responses.
For full deployment details, bundle configurations, and context length options for all models and bundles mentioned below, see [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
**Disruptive release - downtime required to upgrade.**
Allow approximately 1 hour per node when planning your maintenance window.
* Single SambaRack SN40L-16: Full system downtime required for the duration of the upgrade.
* Multiple SambaRack SN40L-16 nodes running the same bundle: Rolling upgrade supported - apply nodes sequentially to maintain service at reduced capacity.
### New features and enhancements
**New models**
Added support for the following models in SambaStack 1.0.57:
* **Gemma 3 12B** (`gemma-3-12b-it`) - image understanding
* `gemma3-v3`: 128K context, BS 2/4/6/8
* **Gemma 3 27B** (`gemma-3-27b-it`) - image understanding
* `gemma3-27b-32-128k`: 32K and 128K context, BS 2/4/6/8
* **Qwen3 235B** (Qwen3-235B-A22B-Instruct-2507) - the legacy `Qwen3-235B` model name is preserved for backward compatibility
* `dyt-qwen3-235b-32-128k`: 32K context (BS 2/4/6/8), 128K context (BS 2)
* `qwen3-235b-16-32-64k`: 16K, 32K, 64K context
* `qwen3-235b-128k`: 128K context
* **DeepSeek V3.2** - available in high-interactivity configurations (up to 128k context) and high-throughput configurations (up to 32k context)
* **GPT-OSS 120B** - adds constrained decoding capability; the previous standard DYT bundle is replaced by two bundles with constrained decoding support enabled
* `cd-dyt-gpt-oss-120b-32-64-128k`: 32K, 64K, 128K context
* `cd-dyt-gpt-oss-120b-8-32-64-128k`: 8K, 32K, 64K, 128K context
***
**High-throughput configurations**
SambaStack 1.0.57 introduces high-throughput configuration options for running on SambaRack nodes.
* Optimized for large-scale serving of a single model, prioritizing total system throughput over per-user latency to support high volumes of concurrent users
* Suited for use cases that do not require low end-to-end latency or interactivity
* Requires a minimum of 4 SambaRack nodes dedicated to this configuration
* Cannot be bundled with other models
* Transparent to users - no API or client code changes required
* Uses a disaggregated prefill-decode architecture with a configurable node ratio - for example, 3 nodes running prefill and 1 running decode
Supported models: DeepSeek-R1, DeepSeek-V3-0324, DeepSeek-V3.1, DeepSeek-V3.1-Terminus, and DeepSeek-V3.2. See [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
Use the following log metrics to monitor high-throughput deployments:
* `decode_queue_time` - time requests wait in the decode queue before processing begins (new in 1.0.57)
* `time_to_first_token` - latency from request receipt to first output token
* `completion_tokens_per_sec` - decode throughput
***
**Constrained decoding (structured output)**
Added constrained decode mask sampling on a per-token schedule. Models that declare `supports_constrained_decoding` in their PEF CRs can now use structured output generation with JSON schema enforcement.
Set `constrained_decoding: true` in the BundleTemplate to enable this feature for the bundle.
***
**TTFT measurement improvement**
`process_request()` processing time is now included in TTFT and end-to-end latency measurements for more accurate reporting.
***
**Bundle legalizer: accuracy and validation improvements**
Improved legalizer accuracy for memory accounting; bundles that exceed available memory or host segment size are now rejected at validation time rather than failing at runtime.
***
**Bundle memory utilization in bundle CR status**
The Bundle CR status now includes a `legalizerInfo` block with memory utilization data from the bundle legalizer. Use `kubectl get bundle -o yaml` to inspect the block.
| Field | Description |
| - | - |
| `status` | Legalizer validation result: `passed`, `failed`, or `skipped` |
| `errors` | List of validation errors (present when `status: failed`) |
| `warnings` | Non-fatal warnings (present when `status: passed` with warnings) |
| `ddr` | DDR memory utilization ratio |
| `hbm_resident` | HBM resident memory utilization; can exceed 1.0 under over-allocation |
| `host` | Host memory utilization ratio |
The `status` field is absent if the legalizer output could not be processed. `skipped` only appears when `skip_legalizer: true` is set on a PEF CR.
***
**PEF and checkpoint lifecycle status**
PEF CRs now include a `pef_status` field, and checkpoint CR version entries include a `checkpoint_status` field, giving operators visibility into artifact lifecycle state without needing to inspect pod logs.
Status values for both fields:
* `preview` - newly available configurations which may not be fully tested and / or may not have full feature support; not recommended for production use cases
* `stable` - well-tested, production-ready configurations
* `deprecated` - still functional; scheduled for removal in a future release
* `removed` - no longer available; must be replaced before deployment
Check `pef_status` when reviewing PEF CR versions before deploying a bundle to confirm the version is `stable`.
***
### API improvements and fixes
Enhancements to improve OpenAI compatibility and new API capabilities across the `chat/completions` endpoint and a new `responses` endpoint.
**Responses API**
The OpenAI-compatible Responses API (`POST /v1/responses`) is now supported.
* Currently supported model: `gpt-oss-120b`
* Supported capabilities: text generation (streaming and non-streaming), function calling (2-step), structured output (JSON schema), multi-turn conversations (client-managed state), and reasoning output
* Uses stateless request semantics - conversation history is passed by the client on each request
***
**`reasoning_tokens` in usage response**
Reasoning models now include a `reasoning_tokens` field in the `usage` object, reporting the number of tokens consumed by the model's internal reasoning step.
***
**`tool_choice` support for GPT-OSS 120B**
`tool_choice` is now supported for `gpt-oss-120b`. Accepted values: `auto`, `none`, `required`, and `{"type": "function", "function": {"name": "..."}}`.
* Only `gpt-oss-120b` supports `tool_choice` in Release 1.0.57
* The forced function call format follows the Chat Completions API structure - the inner `"function"` key is required; this differs from the Responses API format which omits it
* `allowed_tools` is not supported
`tool_choice` is supported only for `gpt-oss-120b` bundles with constrained decoding enabled: `cd-dyt-gpt-oss-120b-32-64-128k` and `cd-dyt-gpt-oss-120b-8-32-64-128k`.
***
**`n` parameter - multiple completions**
The `n` parameter is now supported in chat completions.
* Valid range: 1–16 (default: 1)
* Implemented via API-level decomposition - `n` parallel single-completion requests are issued and combined before returning to the client
* Not supported when `tools` or `functions` are present in the request
***
**`seed` parameter**
The `seed` parameter is now supported for reproducible outputs.
* Accepts any integer, including negative values
* Applies to text generation models only - not supported for multi-modal or continuous batching models
* `system_fingerprint` is not returned (unlike OpenAI)
***
**OpenAI-conformant error responses**
All API error responses now use the OpenAI-standard error format. A new top-level `request_id` field is included in every error response.
```json theme={}
{
"request_id": "abc-123",
"error": {
"message": "...",
"type": "...",
"param": null,
"code": "..."
}
}
```
Provide `request_id` to SambaNova support when reporting an issue.
***
**Structured output parser fixes**
Fixed bugs in structured output that produced incorrect output for certain schema patterns.
***
**Better error message for tool call truncation**
Improved the error message when a tool call exceeds the maximum token length.
***
**Empty streaming chunk removed**
Removed a spurious empty chunk emitted during streaming responses, improving conformance with OpenAI streaming behavior.
***
**Whisper rate limit error code fix**
Fixed the Whisper transcription endpoint to return the correct 429 status code when the rate limit is exceeded.
***
### Bug fixes
**PEF cache fix**
Fixed a bug with PEF cache when migrating Bundles to use PEF CRs
***
**Auth provider validation fix**
Fixed a Helm auth provider validation issue that incorrectly rejected custom secrets when Keycloak was also enabled.
***
**CVE dependency updates**
Updated multiple dependency versions across the inference operator, global model router, and supporting libraries to address known CVEs.
***
### Known issues
**Gemma 3: function calling not supported**
Gemma 3 12B and 27B do not support native function calling. The `toolSupport: true` flag in bundles using Gemma models indicates support for JSON output schema, rather than general native function calling support.
* **Impact:** Requests using the `tools` parameter with `gemma-3-12b-it` will not produce function call outputs.
* **Workaround:** Function calling behavior can be approximated by implementing tool-use logic via user prompts.
**Gemma 3 27B: vision mode instability at 104k–112k context length**
Vision requests to `gemma-3-27b-it` may experience intermittent errors.
* **Impact:** Vision requests to `gemma-3-27b-it` at 104k–112k token context lengths may experience intermittent HTTP 524 timeout errors with elevated latency. Text and function calling modes are not affected.
* **Workaround:** Avoid vision requests in the 104k–112k context window range for now. We are looking in to a fix.
***
## SambaStack v0.5.17 release
**Release Date:** April 8, 2026
This release introduces support for SambaRack SN40L-16 configuration with 4TB of DDR memory.
### New features and enhancements
**Cluster-level memory management**
Adds support for declaring cluster-wide DDR memory limits for SambaRack SN40L-16 via an environment variable in `sambastack.yaml`, enforced by the bundle validation tool.
* The default memory limit is 12TB. Update this value to support SambaRack SN40L-16 with 4TB of DDR.
* Set the `DDR_PER_RDU_GB` environment variable: default is `768` (12TB per-node), set to `256` for 4TB per-node. For more details, see the [**SambaStack.yaml reference**](/docs/en/sambastack/resources/sambastack-yaml)
* The memory limit applies to all SambaRack nodes in the cluster.
* The bundle validation tool enforces the limit at runtime. Configurations exceeding the limit fail with an informative error and must be refactored by removing model configurations.
### Known issues
**Inventory check shows degraded status for 4TB memory configurations**
Running `snfadm inventory` shows "degraded" status for RDUs that are in nodes with 4TB memory configurations.
* **Impact:** Cosmetic only. Does not affect operation.
* **Resolution:** Expected to be resolved in a future release.
***
## SambaStack v0.5.14 release
**Release Date:** April 1, 2026
This release introduces simplified checkpoint discovery, new model support (MiniMax-M2.5, Agentic RAG bundle), enhanced installation verification tools, and multiple API enhancements for improved OpenAI compatibility.
For full deployment details, bundle configurations, and context length options for all models and bundles mentioned below, see [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles).
### New features and enhancements
**Checkpoint path discovery via model CRs**
Checkpoint paths are now discoverable through the Model Custom Resource (CR), eliminating the need for customers to manually locate checkpoint paths in configuration files. The following Kubernetes command can be used to view Model CRs, which now contain checkpoint paths:
```bash theme={}
kubectl -n describe model
```
For example:
```bash theme={}
kubectl describe model minimax-m2.5
```
* Model CRs now include checkpoint path information for all supported models.
* Supports multiple checkpoints for different model configurations.
* Backwards compatible with existing bundle configurations - checkpoint paths in bundle CRs override Model CR paths if specified.
* Works with on-prem and air-gapped deployments.
**Improved discovery of available models and bundles**
All available models and bundles are now applied automatically by the SambaStack helm chart. No additional configuration is required.
* Models can be discovered using `kubectl -n get models`.
* Bundles can be discovered using `kubectl -n get bundles`.
**Agentic RAG bundle**
Added the `us-agentic-rag-1-1` bundle, a multi-model bundle optimized for retrieval-augmented generation (RAG) workflows. It contains the following model configs:
* `gpt-oss-120b`
* Seq Length: 32K, BS: 4
* Seq Length: 64K, BS: 2
* Seq Length: 128K, BS: 2
* `Llama-4-Maverick-17B-128E-Instruct`
* Seq Length: 8K, BS: 1
* Seq Length: 16K, BS: 1
* `Meta-Llama-3.3-70B` (Target) / `Meta-Llama-3.2-1B` (Draft)
* Seq Length: 4K, BS: 1, 4, 8, 16, 32
* Seq Length: 8K, BS: 1, 4, 8
* Seq Length: 16K, BS: 1, 4
* Seq Length: 32K, BS: 1, 4
* Seq Length: 64K, BS: 1
* Seq Length: 128K, BS: 1
* `Meta-Llama-3.1-8B-Instruct`
* Seq Length: 4K, BS: 1, 4, 16, 32
* Seq Length: 8K, BS: 1, 4, 16, 32
* Seq Length: 16K, BS: 1, 4, 8
* `E5-Mistral-7B-Instruct`
* Seq Length: 4K, BS: 1, 4, 8, 16, 32
**MiniMax-M2.5 model support**
Added support for MiniMax-M2.5 model on SambaStack.
* Checkpoint accessible via your artifact reader service account.
* Customers can include MiniMax-M2.5 in bundles that pass bundle validation and deploy successfully.
* Includes reasoning support.
**Pre/post-install verification scripts**
New verification scripts help customers validate their SambaStack environment before and after installation.
* **Pre-install script:** Validates all hardware, connectivity, and software prerequisites.
* **Post-install script:** Confirms all SambaStack components are installed and running correctly.
* Clear pass/fail reporting with actionable guidance on failures.
* Scripts maintained and validated against the current SambaStack release.
* Distributed via the [sambastack-tools](https://github.com/sambanova/sambastack-tools) public GitHub repository with README instructions.
**Per-model queue depth configuration**
Added support for configuring different queue depths based on context length, enabling optimized uptime for high-context-length requests.
* Queue depths can now be configured per context length group using `contextGroups` in the Service Tier configuration.
* Queue depth controls how many concurrent requests can be queued for a model configuration.
* Lower queue depths for higher context lengths help prevent memory exhaustion and improve overall service stability.
* SambaStack now validates queue depth configuration at request time. Misconfigured models with missing queue depth definitions will surface a clear error instead of failing silently.
* The empty string `""` in `contextLengths` matches requests to the base model name without a context length suffix (e.g., `DeepSeek-R1-0528`). Requests with explicit suffixes like `-8k` or `-128k` match their corresponding `contextLengths` values.
Context length suffixes (8k, 16k, 32k, etc.) are case-sensitive. Use lowercase `k` in all configurations. This applies to all models supported by SambaNova.
The `contextGroups` field is a sub-component of a model grouping within a service tier. Example configuration:
```yaml theme={}
free:
- models:
- DeepSeek-V3-0324
- DeepSeek-R1-0528
queueDepth: 10
qos: free
rates:
- allowedRequests: 100
periodSeconds: 60
example_service_tier:
inherits: free
overrides:
- models:
- DeepSeek-V3-0324
queueDepth: 10
qos: example
rates:
- allowedRequests: 100
periodSeconds: 60
- models:
- DeepSeek-R1-0528
queueDepth: 10
qos: example
contextGroups:
- contextLengths: ["", "8k"]
queueDepth: 2
- contextLengths: ["16k", "32k", "64k", "128k"]
queueDepth: 1
rates:
- allowedRequests: 100
periodSeconds: 60
```
**Helm chart configuration update**
The `substitutions` field has moved from `bundles` to `global` in the SambaStack Helm chart.
This is a breaking change that affects air-gapped and NFS customers. Update your Helm values file before upgrading.
**Before:**
```yaml theme={}
bundles:
substitutions:
gs://: nfs:///nfsdata
```
**After:**
```yaml theme={}
global:
substitutions:
gs://: nfs:///nfsdata
```
### API improvements and fixes
Enhancements to improve OpenAI compatibility across the `chat/completions` endpoint and a new, non-standard feature to track usage in streaming chunks.
**Text object support in user message content**
* Expanded support for text objects in content arrays, matching OpenAI `ChatCompletionsContentPartText` specification.
* Enabled for: DeepSeek-R1-0528, Llama-3.3-Swallow-70B-Instruct-v0.4, and MiniMax-M2.5.
**Log probabilities**
* Added `logprobs` field that, when set to `true`, returns the log probabilities for each generated token.
* Added `top_logprobs` field that, when set to an integer `n`, returns the top `n` log probabilities for each generation.
**Usage in streaming chunks**
* Added a non-standard feature to allow users to obtain partial usage statistics in chunks returned in streaming responses.
* This feature is enabled by setting `STREAM_USAGE_IN_CHUNKS: true` in the replica group section of your custom bundle deployment.
**Tool choices**
Supported only for models that support function calling.
* `tool_choice: none` ensures that the model will not see available tools.
**Invalid message role validation**
* The `chat/completions` endpoint now rejects invalid message roles. Only `user`, `assistant`, `system`, and `tool` are accepted.
**Whisper audio error response**
* The Whisper transcription endpoint now returns descriptive error messages when audio file processing fails, instead of a bare HTTP 400 status code.
### Bug fixes
**Function calling routing fix**
* Fixed an issue where function calling routing did not apply the model name prefix check correctly, causing some models to skip tool routing.
**Air-gap inventory fix**
* Fixed the air-gap inventory to include the correct `cloudnative-pg` image configuration, preventing missing image errors during offline installation.
### Known issues
**SambaRack Manager does not support 2 PDU configurations**
SambaRack Manager does not currently support configurations with 2 PDUs. Customers using 2 PDU setups should contact SambaNova Support for guidance on alternative configurations.
***
## SambaStack v0.4.8 release
**Release Date:** March 10, 2026
This release introduces air-gapped deployment support, custom checkpoint management with NFS storage, swappable model configurations, and multiple API enhancements for improved OpenAI compatibility.
### New features and enhancements
**SambaStack air-gapped support**
Added support for air-gapped mode of operation, enabling secure, isolated deployments.
* Install, upgrade, and setup for air-gapped configurations is performed in conjunction with SambaNova support.
* Ongoing administration (Auth, User Management, Custom DB) is designed for self-service and follows the same workflows as on-prem deployments.
Install, setup, port forwarding to access Keycloak UI, and upgrade steps are not documented for air-gapped deployments due to varying customer network configurations. Please work with SambaNova support for these workflows.
**Custom checkpoints with NFS storage**
Added the ability to reference custom checkpoints from customer-provided NFS storage in deployments.
**Swappable models in bundles**
Added configurable model swapping behavior to optimize high-bandwidth memory (HBM) utilization.
* By default, all models in bundles can be swapped out of HBM and replaced with other models in DDR memory.
* Use the `swappable: ` field in the bundle YAML definition to enable or disable this behavior.
* Default value is `true`. When set to `false`, the model remains in HBM and cannot be swapped out, ensuring zero switching time for requests to that model.
### API improvements and fixes
Enhancements to improve OpenAI compatibility across the `chat/completions` endpoint.
**Text object support in user message content**
* Added support for text objects in content arrays, matching OpenAI `ChatCompletionsContentPartText` specification.
* Enabled for: gpt-oss-120b, DeepSeek-V3.1, DeepSeek-V3.1-Terminus, DeepSeek-V3.2, DeepSeek-V3-0324, Qwen3-32B, Qwen3-235B.
**Response format text option**
* Fixed an issue where `response_format=text` would throw an error.
* The endpoint now supports all OpenAI formats: `text`, `json_object`, `json_schema`.
**Extended temperature range**
* Expanded `temperature` range from 0.0–1.0 to 0.0–2.0, matching OpenAI specification.
**Tool calling number type fix**
* Tools with number-type arguments were always returned as floats.
* Now integers are preserved as integers, matching [JSON Schema number specification](https://json-schema.org/understanding-json-schema/reference/numeric#number).
**Parallel tool calls support**
* Added `parallel_tool_calls` parameter support.
* When set to `false`, the model will make at most one tool call per response, matching [OpenAI specification](https://platform.openai.com/docs/api-reference/chat/create#chat-create-parallel_tool_calls).
**Streaming token usage reporting**
* Added support for token usage reporting in each chunk of stream.
### Known issues
* Parallel Tool Calls with Constrained Decoding.
* The following models return `null` for `logprobs` even when `logprobs=true` or `top_logprobs` is set. The parameters are accepted without error but have no effect:
* Llama-4-Maverick-17B-128E-Instruct
* Whisper-Large-v3
***
## SambaStack initial release
**Release Date:** September 19, 2025
This release introduces the comprehensive SambaStack documentation suite.
### New features and enhancements
**SambaStack guide**
Added the [SambaStack Guide](/docs/en/sambastack/getting-started/introduction), providing step-by-step instructions for deploying, configuring, and managing [SambaStack](https://sambanova.ai/products/sambastack).
* Setup, installation, and environment configuration.
* User and authentication management (Keycloak, OIDC).
* Monitoring, logging, and artifact management.
* Bundle and model deployment workflows.
* Common command reference.
**SambaStack models**
Added the [SambaStack models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles) page to help customers understand which models are available on SambaStack and how to configure them.
* Lists all supported models (e.g., Llama 3.3, Llama 4 Maverick, DeepSeek).
* Shows context length, batch size options, and supported features.
* Instructions for using the Model list API to check availability in your environment.