SambaStack Models Chart - SambaNova Documentation

SambaStack keeps model definitions separate from platform infrastructure. The models chart (sambastack-models) is the Helm release that deploys the models, bundles, and PEFs your cluster serves, and it is installed alongside the infrastructure charts rather than bundled into them. Because the model chart is in a separate release from the platform infrastructure charts, you can update which models you serve without touching the platform, and pin the two to different compatible versions when you need to.

When you pin the models chart and the infra chart to different versions, make sure the two are compatible. SambaNova publishes the compatible version pairs.

Chart layout

A SambaStack deployment is three Helm releases, installed in this order:

Order	Release	What it provides
1	`sambastack-base`	Cluster prerequisites
2	`sambastack`	The platform: operators, routing, gateway, and UI
3	`sambastack-models`	The model catalog: `Model`, `ModelProfile`, `ModelBundle`, and `Pef` resources

sambastack-base → sambastack → sambastack-models is the recommended install order.

For the commands, see Quickstart - on-prem. On hosted deployments the sambastack-installer deploys all three releases for you, in the same order — see Quickstart - hosted.

Where its values live

The models chart takes its own values, separate from sambastack.yaml:

Deployment type	Location
On-prem	A values file passed to `helm upgrade --install sambastack-models`, with the keys at the top level
Hosted	The `models.yaml` section of the installer ConfigMap, which the installer passes verbatim to the release

Selecting and deploying bundles

Two keys control what the chart serves, and they do different jobs:

modelBundleSpecs selects which of the ModelBundle resources provided with SambaStack to render. Omit it, or leave it empty, to render all of them. Rendering a bundle makes it available for deployment, but does not deploy it.
modelDeploymentSpecs deploys bundles, with replica-group configuration.

modelBundleSpecs:
  - name: cd-dyt-gpt-oss-120b-8-32-64-128k
modelDeploymentSpecs:
  - name: cd-dyt-gpt-oss-120b-8-32-64-128k
    groups:
      - name: default
        minReplicas: 1
        qosList: [web, free]

modelDeploymentSpecs[].name is used as both the ModelDeployment name and its spec.bundle, so it must match a ModelBundle name. Duplicate names fail the Helm render.

For the bundles SambaNova provides, see Supported models and bundles. For the QoS tags in qosList, see Service tiers.

Configuration reference

These are the models chart’s own values. Hosted, they go in the models.yaml section of the installer ConfigMap. On-prem, they go at the top level of the values file passed to the release.

Parameter	Type	Description
`modelBundleSpecs[]`	list	Selects which of the `ModelBundle` resources provided with SambaStack to render. Omit it, or leave it empty, to render all of them. Rendering a `ModelBundle` does not deploy it.
`modelDeploymentSpecs[]`	list	Deploys bundles, with replica-group configuration.
`modelDeploymentSpecs[].name`	string	Bundle name to deploy. Used as both the `ModelDeployment` name and its `spec.bundle`, so it must match a `ModelBundle` name. Duplicate names fail the Helm render.
`modelDeploymentSpecs[].groups[].name`	string	Routing/capacity group name
`modelDeploymentSpecs[].groups[].minReplicas`	integer	Minimum engines for the group
`modelDeploymentSpecs[].groups[].qosList[]`	list	QoS tags (e.g., `web`, `free`, `pro`)
`global.substitutions`	object	String substitutions applied to the rendered model resources. Only needed for NFS or air-gapped deployments.

See Service tiers for details on QoS tags.

Artifact locations

The models chart applies string substitutions to the resources it renders. NFS and air-gapped deployments set global.substitutions in the models chart values:

global:
  substitutions:
    gs://<SAMBASTACK_ARTIFACTS_BUCKET>: nfs:///nfsdata

SambaStack.yaml reference

Every configuration key, and which chart each one belongs to

Migrating bundle configuration

The resource model behind the bundles this chart deploys, and how to migrate to it

Deploying models and bundles

Authoring the custom resources directly

Deploying custom checkpoints

The Model resource reference, and serving your own checkpoints

Getting Started

Service Administration

Hardware Administration

Reference Architecture

Resources

Models chart

Chart layout

Where its values live

Selecting and deploying bundles

Configuration reference

Artifact locations

SambaStack.yaml reference

Migrating bundle configuration

Deploying models and bundles

Deploying custom checkpoints

​Chart layout

​Where its values live

​Selecting and deploying bundles

​Configuration reference

​Artifact locations

​Related resources

SambaStack.yaml reference

Migrating bundle configuration

Deploying models and bundles

Deploying custom checkpoints

Chart layout

Where its values live

Selecting and deploying bundles

Configuration reference

Artifact locations

Related resources