SambaStack artifacts are the fundamental components used to package, deploy, and run large language models. They include several key elements:
  • Bundle template: A bundle template defines what can be deployed together on a single node. It contains one or more model templates, each listing supported configurations such as sequence length and batch size.
    • Example - Model templates:
      1. DeepSeek-R1-0528-Template — supports sequence lengths 8k and 4k, with batch sizes 1, 2, 4, and 8 for DeepSeek-R1-0528.
      2. DeepSeek-V3-0324-Template — supports sequence lengths 8k and 4k, with batch sizes 1, 2, 4, and 8 for DeepSeek-V3-0324.
    • Example - Bundle template:
      1. deepseek-r1-v3-fp8-32k-Template — the bundle template includes both DeepSeek-R1-0528-Template and DeepSeek-V3-0324-Template.
  • Bundle: A bundle is created by binding actual checkpoints to the model templates defined in a bundle template.
    • Example - Bundle:
    1. deepseek-r1-v3-fp8-32k — a Bundle that binds the R1-0528 checkpoint and the V3-0324 checkpoint to their respective Model Templates.
  • Checkpoints: Saved states of a machine learning model at a particular point during training.
  • PEFs: PEFs are compiler runtime files optimized for inference with specific sequence lengths, batch sizes, and model configurations.
Artifacts are stored in organized folders, typically in Google Cloud Storage (GCS) for hosted environments. Metadata points to the latest versions to ensure traceability and simplify updates.

​
Version management and private assets

Before reviewing specific update procedures, it is important to understand how SambaStack organizes and manages artifact versions. This ensures your deployments remain stable and compatible as new models and software revisions are introduced.

​
Model and software updates

This section explains how SambaStack handles updates for models and software, helping keep your deployments current and compatible with minimal disruption.
  • Model updates are delivered by updating folders in GCS. Existing deployments remain unchanged until updated by customers.
  • SambaStack software releases include all relevant bundle and model updates. Details of each release are provided in the SambaStack release notes.
  • The update procedure involves:
    1. Reinstalling the SambaStack tarball from the Google Artifact Registry (GAR).
    2. Using the included values.yaml to point to the correct Docker artifact paths and deployment YAMLs (details provided with each release).
    3. Applying changes using: 
      kubectl apply -f values.yaml
  • New model and artifact versions are designed to maintain compatibility and avoid breaking existing deployments or APIs. Any exceptions will be communicated in release notes.

​
Private artifacts

Private artifacts are custom models and checkpoints managed separately to protect sensitive or proprietary data. This section explains how access to these assets is protected and controlled.
  • Custom models and checkpoints are managed in separate storage buckets.
  • Access to private artifacts is restricted based on defined rules to ensure security and compliance.
  • Access to private artifacts is restricted through segregation and access control policies to ensure their security and compliance.