SambaStack release notes

Documentation Index

Fetch the complete documentation index at: https://sambanova-systems.mintlify.dev/docs/llms.txt

Use this file to discover all available pages before exploring further.

​

SambaStack v1.0.57 release

​

New features and enhancements

• Gemma 3 12B (gemma-3-12b-it) - image understanding
  • gemma3-v3: 128K context, BS 2/4/6/8
• Gemma 3 27B (gemma-3-27b-it) - image understanding
  • gemma3-27b-32-128k: 32K and 128K context, BS 2/4/6/8
• Qwen3 235B (Qwen3-235B-A22B-Instruct-2507) - the legacy Qwen3-235B model name is preserved for backward compatibility
  • dyt-qwen3-235b-32-128k: 32K context (BS 2/4/6/8), 128K context (BS 2)
  • qwen3-235b-16-32-64k: 16K, 32K, 64K context
  • qwen3-235b-128k: 128K context
• DeepSeek V3.2 - available in high-interactivity configurations (up to 128k context) and high-throughput configurations (up to 32k context)
• GPT-OSS 120B - adds constrained decoding capability; the previous standard DYT bundle is replaced by two bundles with constrained decoding support enabled
  • cd-dyt-gpt-oss-120b-32-64-128k: 32K, 64K, 128K context
  • cd-dyt-gpt-oss-120b-8-32-64-128k: 8K, 32K, 64K, 128K context

• Optimized for large-scale serving of a single model, prioritizing total system throughput over per-user latency to support high volumes of concurrent users
• Suited for use cases that do not require low end-to-end latency or interactivity
• Requires a minimum of 4 SambaRack nodes dedicated to this configuration
• Cannot be bundled with other models
• Transparent to users - no API or client code changes required
• Uses a disaggregated prefill-decode architecture with a configurable node ratio - for example, 3 nodes running prefill and 1 running decode

• decode_queue_time - time requests wait in the decode queue before processing begins (new in 1.0.57)
• time_to_first_token - latency from request receipt to first output token
• completion_tokens_per_sec - decode throughput

• preview - newly available configurations which may not be fully tested and / or may not have full feature support; not recommended for production use cases
• stable - well-tested, production-ready configurations
• deprecated - still functional; scheduled for removal in a future release
• removed - no longer available; must be replaced before deployment

​

API improvements and fixes

• Currently supported model: gpt-oss-120b
• Supported capabilities: text generation (streaming and non-streaming), function calling (2-step), structured output (JSON schema), multi-turn conversations (client-managed state), and reasoning output
• Uses stateless request semantics - conversation history is passed by the client on each request

• Only gpt-oss-120b supports tool_choice in Release 1.0.57
• The forced function call format follows the Chat Completions API structure - the inner "function" key is required; this differs from the Responses API format which omits it
• allowed_tools is not supported

• Valid range: 1–16 (default: 1)
• Implemented via API-level decomposition - n parallel single-completion requests are issued and combined before returning to the client
• Not supported when tools or functions are present in the request

• Accepts any integer, including negative values
• Applies to text generation models only - not supported for multi-modal or continuous batching models
• system_fingerprint is not returned (unlike OpenAI)

{
  "request_id": "abc-123",
  "error": {
    "message": "...",
    "type": "...",
    "param": null,
    "code": "..."
  }
}

​

Bug fixes

​

Known issues

• Impact: Requests using the tools parameter with gemma-3-12b-it will not produce function call outputs.
• Workaround: Function calling behavior can be approximated by implementing tool-use logic via user prompts.

• Impact: Vision requests to gemma-3-27b-it at 104k–112k token context lengths may experience intermittent HTTP 524 timeout errors with elevated latency. Text and function calling modes are not affected.
• Workaround: Avoid vision requests in the 104k–112k context window range for now. We are looking in to a fix.

​

SambaStack v0.5.17 release

​

New features and enhancements

• The default memory limit is 12TB. Update this value to support SambaRack SN40L-16 with 4TB of DDR.
• Set the DDR_PER_RDU_GB environment variable: default is 768 (12TB per-node), set to 256 for 4TB per-node. For more details, see the SambaStack.yaml reference
• The memory limit applies to all SambaRack nodes in the cluster.
• The bundle validation tool enforces the limit at runtime. Configurations exceeding the limit fail with an informative error and must be refactored by removing model configurations.

​

Known issues

• Impact: Cosmetic only. Does not affect operation.
• Resolution: Expected to be resolved in a future release.

​

SambaStack v0.5.14 release

​

New features and enhancements

kubectl -n <namespace> describe model <model-name>

kubectl describe model minimax-m2.5

• Model CRs now include checkpoint path information for all supported models.
• Supports multiple checkpoints for different model configurations.
• Backwards compatible with existing bundle configurations - checkpoint paths in bundle CRs override Model CR paths if specified.
• Works with on-prem and air-gapped deployments.

• Models can be discovered using kubectl -n <namespace> get models.
• Bundles can be discovered using kubectl -n <namespace> get bundles.

• gpt-oss-120b
  • Seq Length: 32K, BS: 4
  • Seq Length: 64K, BS: 2
  • Seq Length: 128K, BS: 2
• Llama-4-Maverick-17B-128E-Instruct
  • Seq Length: 8K, BS: 1
  • Seq Length: 16K, BS: 1
• Meta-Llama-3.3-70B (Target) / Meta-Llama-3.2-1B (Draft)
  • Seq Length: 4K, BS: 1, 4, 8, 16, 32
  • Seq Length: 8K, BS: 1, 4, 8
  • Seq Length: 16K, BS: 1, 4
  • Seq Length: 32K, BS: 1, 4
  • Seq Length: 64K, BS: 1
  • Seq Length: 128K, BS: 1
• Meta-Llama-3.1-8B-Instruct
  • Seq Length: 4K, BS: 1, 4, 16, 32
  • Seq Length: 8K, BS: 1, 4, 16, 32
  • Seq Length: 16K, BS: 1, 4, 8
• E5-Mistral-7B-Instruct
  • Seq Length: 4K, BS: 1, 4, 8, 16, 32

• Checkpoint accessible via your artifact reader service account.
• Customers can include MiniMax-M2.5 in bundles that pass bundle validation and deploy successfully.
• Includes reasoning support.

• Pre-install script: Validates all hardware, connectivity, and software prerequisites.
• Post-install script: Confirms all SambaStack components are installed and running correctly.
• Clear pass/fail reporting with actionable guidance on failures.
• Scripts maintained and validated against the current SambaStack release.
• Distributed via the sambastack-tools public GitHub repository with README instructions.

• Queue depths can now be configured per context length group using contextGroups in the Service Tier configuration.
• Queue depth controls how many concurrent requests can be queued for a model configuration.
• Lower queue depths for higher context lengths help prevent memory exhaustion and improve overall service stability.
• SambaStack now validates queue depth configuration at request time. Misconfigured models with missing queue depth definitions will surface a clear error instead of failing silently.
• The empty string "" in contextLengths matches requests to the base model name without a context length suffix (e.g., DeepSeek-R1-0528). Requests with explicit suffixes like -8k or -128k match their corresponding contextLengths values.

free:
  - models:
      - DeepSeek-V3-0324
      - DeepSeek-R1-0528
    queueDepth: 10
    qos: free
    rates:
      - allowedRequests: 100
        periodSeconds: 60
example_service_tier:
  inherits: free
  overrides:
    - models:
        - DeepSeek-V3-0324
      queueDepth: 10
      qos: example
      rates:
        - allowedRequests: 100
          periodSeconds: 60
    - models:
        - DeepSeek-R1-0528
      queueDepth: 10
      qos: example
      contextGroups:
        - contextLengths: ["", "8k"]
          queueDepth: 2
        - contextLengths: ["16k", "32k", "64k", "128k"]
          queueDepth: 1
      rates:
        - allowedRequests: 100
          periodSeconds: 60

bundles:
  substitutions:
    gs://<SAMBASTACK_ARTIFACTS_BUCKET>: nfs:///nfsdata

global:
  substitutions:
    gs://<SAMBASTACK_ARTIFACTS_BUCKET>: nfs:///nfsdata

​

API improvements and fixes

• Expanded support for text objects in content arrays, matching OpenAI ChatCompletionsContentPartText specification.
• Enabled for: DeepSeek-R1-0528, Llama-3.3-Swallow-70B-Instruct-v0.4, and MiniMax-M2.5.

• Added logprobs field that, when set to true, returns the log probabilities for each generated token.
• Added top_logprobs field that, when set to an integer n, returns the top n log probabilities for each generation.

• Added a non-standard feature to allow users to obtain partial usage statistics in chunks returned in streaming responses.
• This feature is enabled by setting STREAM_USAGE_IN_CHUNKS: true in the replica group section of your custom bundle deployment.

• tool_choice: none ensures that the model will not see available tools.

• The chat/completions endpoint now rejects invalid message roles. Only user, assistant, system, and tool are accepted.

• The Whisper transcription endpoint now returns descriptive error messages when audio file processing fails, instead of a bare HTTP 400 status code.

​

Bug fixes

• Fixed an issue where function calling routing did not apply the model name prefix check correctly, causing some models to skip tool routing.

• Fixed the air-gap inventory to include the correct cloudnative-pg image configuration, preventing missing image errors during offline installation.

​

Known issues

​

SambaStack v0.4.8 release

​

New features and enhancements

• Install, upgrade, and setup for air-gapped configurations is performed in conjunction with SambaNova support.
• Ongoing administration (Auth, User Management, Custom DB) is designed for self-service and follows the same workflows as on-prem deployments.

• By default, all models in bundles can be swapped out of HBM and replaced with other models in DDR memory.
• Use the swappable: <boolean> field in the bundle YAML definition to enable or disable this behavior.
• Default value is true. When set to false, the model remains in HBM and cannot be swapped out, ensuring zero switching time for requests to that model.

​

API improvements and fixes

• Added support for text objects in content arrays, matching OpenAI ChatCompletionsContentPartText specification.
• Enabled for: gpt-oss-120b, DeepSeek-V3.1, DeepSeek-V3.1-Terminus, DeepSeek-V3.2, DeepSeek-V3-0324, Qwen3-32B, Qwen3-235B.

• Fixed an issue where response_format=text would throw an error.
• The endpoint now supports all OpenAI formats: text, json_object, json_schema.

• Expanded temperature range from 0.0–1.0 to 0.0–2.0, matching OpenAI specification.

• Tools with number-type arguments were always returned as floats.
• Now integers are preserved as integers, matching JSON Schema number specification.

• Added parallel_tool_calls parameter support.
• When set to false, the model will make at most one tool call per response, matching OpenAI specification.

• Added support for token usage reporting in each chunk of stream.

​

Known issues

• Parallel Tool Calls with Constrained Decoding.
• The following models return null for logprobs even when logprobs=true or top_logprobs is set. The parameters are accepted without error but have no effect:
  • Llama-4-Maverick-17B-128E-Instruct
  • Whisper-Large-v3

​

SambaStack initial release

​

New features and enhancements

• Setup, installation, and environment configuration.
• User and authentication management (Keycloak, OIDC).
• Monitoring, logging, and artifact management.
• Bundle and model deployment workflows.
• Common command reference.

• Lists all supported models (e.g., Llama 3.3, Llama 4 Maverick, DeepSeek).
• Shows context length, batch size options, and supported features.
• Instructions for using the Model list API to check availability in your environment.