Skip to main content
This section guides Sambastack administrators in managing user permissions, configuring admin access, and setting up service tiers and Quality of Service (QoS) levels. It explains how administrators control which users can access specific models, how much they can use, and the priority assigned to their requests.

​
Add an administrator

Users can be granted elevated access by adding their email to the Sambastack configuration. To add a user as an administrator:
  1. Add their email address under the db-admin section in the sambastack.yaml file. For example: 
    db-admin:
  admins:
	- abc@example.com
    Only add email addresses of authorized admins to maintain security. Must be at the same YAML level as bundles (root-level key in sambastack.yaml).
  2. After updating the .yaml file, apply the following configuration: 
    helm upgrade \
  --install \
  --namespace sambastack \
  --version 0.3.528 \
  sambastack \
  -f sambastack.yaml \
  oci://<REGISTRY_URL>/sambastack/sambastack
Open a browser (Chrome is recommended) and navigate to the URL below or simply select the “Administration” tab on the left panel:
<UI Domain>/admin
The admin page becomes accessible once the administrator logs in.Admin Page

​
Service tier configuration

Service tiers (known as Usage Plans in the UI) define what models users can access, their usage limits, and permissions.

​
Sample service tier configuration

For service tiers, you can add a section must be at the same YAML level as bundles (root-level key in sambastack.yaml) with proper indentation. See the example.
serviceTiers:                           # [Optional] Configure serviceTiers
  <Tier1>:                              # Name a custom service tier
  - models:                             # List models allowed in the service tier
    - Llama-4-Maverick-17B-128E-Instruct
    queueDepth: 25                      # Queries to queue before returning busy
    qos: "free"                         # Quality-of-service, usually same as service tier name
    rates:                              # Rate limit list
    - allowedRequests: 0
      periodSeconds: 30
  <Tier2>:                              # Name a custom service tier that inherits another tier
    inherits: <Tier1>
    overrides:
    - models:
      - Llama-4-Maverick-17B-128E-Instruct
      queueDepth: 25
      qos: "free"
      rates:
      - allowedRequests: 2
        periodSeconds: 30
Apply changes by running:
helm upgrade \
  --install \
  --namespace sambastack \
  --version 0.3.528 \
  sambastack \
  -f sambastack.yaml \
  oci://<REGISTRY_URL>/sambastack/sambastack
The tier named example will appear as a Usage Plan on the Admin page and can be assigned to users.User Service Tier Admin Page

​
Key fields in service tiers

The following table outlines the key fields used to define service tiers, along with descriptions and example values for each.
FieldDescriptionExample value
qosQuality of Service level assigned to requests from this tier. Usually matches the service tier name.enterprise-group-1, customer-demo
modelsList of models accessible to users within the tier. A model must be included in at least one tier for users to access it.[Llama-3.3-Swallow-70B-Instruct-v0.4]
queueDepthMaximum number of queries to queue before returning a busy response.100
ratesDefines rate limits (allowed requests and period in seconds){ allowedRequests: 10, periodSeconds: 60 }
inheritsAllows a tier to inherit settings from a base tier and override specific fields.inherits: previously defined tier name, overrides: mentions which properties to override

​
Service tier features

Service tiers offer powerful controls to tailor user access, usage limits, and permissions, ensuring flexible and secure management of AI model resources.
  • Control access: Decide which models each user or group can use.
  • Set usage limits: Define how many requests or tokens a user can make in a set period.

​
Service tier functionality

Service tiers are structured lists of model-group objects that define access controls and operational limits.Each model-group block sets parameters such as queue depth and per-user rate limits to control resource usage and request handling.The inherits attribute allows a tier to extend another base tier’s configuration, promoting reuse.When inheriting, only specified fields in the overrides section are modified, enabling precise and maintainable customization.

​
Service tier management

​
Creating and editing service tiers

You can define base tiers and create derived tiers by using inheritance to promote reuse and consistency.Example - Base tier
serviceTiers:
  free:
  - models:
    - Llama-3.1-Swallow-8B-Instruct-v0.3
    - Meta-Llama-3.1-8B-Instruct
    queueDepth: 25
    qos: "free"
    rates:
    - allowedRequests: 20
      periodSeconds: 60
Example - Derived tier with inheritance
premium:
  inherits: free
  overrides:
  - models:
    - Meta-Llama-3.1-70B-Instruct
    batchSize: 2
    queueDepth: 50
    qos: "premium"
    rates:
    - allowedRequests: 100
      periodSeconds: 60
After updating the configuration, apply the changes by running:
helm upgrade \
  --install \
  --namespace sambastack \
  --version 0.3.528 \
  sambastack \
  -f sambastack.yaml \
  oci://<REGISTRY_URL>/sambastack/sambastack

​
Service tier recommendations

When creating and managing service tiers, consider the following best practices to ensure stability, security, and flexibility:1. Preserve system-managed and default tiers
Some tiers are pre-configured and system-managed and should not be removed or disabled. These tiers provide baseline access and enforce model lifecycle and access controls. Removing or misconfiguring them can interrupt critical workflows.This includes:
  • free / web – Default baseline access tiers.
  • deprecated – Models permanently removed (HTTP 410).
  • maintenance – Models temporarily unavailable (HTTP 503).
  • restricted – Models with limited access (HTTP 403).
These tiers are managed by the system and should be accounted for in service tier planning, not customized.2. Modify with caution
You may adjust rate limits or model lists in default tiers where supported, but changes should align with user needs to avoid unintended access restrictions. Overwriting a tier replaces its full configuration. If a tier is removed from sambastack.yaml, it reverts to SambaNova defaults.3. Use custom tiers for flexibility
Create custom tiers by inheriting from base tiers (such as free or web) to tailor access, rate limits, and models while preserving the underlying structure.These tiers are managed by the system and should be factored into service tier planning.Example of System-managed, Required, and Custom Tiers in sambastack.yaml. Must be at the same YAML level as bundles (root-level key in sambastack.yaml).
serviceTiers:
  ###################################
  #     SYSTEM-MANAGED TIERS        #
  ###################################
  deprecated:
  - models:
    - Old-Model-v1
    queueDepth: 0
    rates:
    - allowedRequests: 0
      periodSeconds: 3600

  restricted:
  - models:
    - Restricted-Model
    queueDepth: 1
    rates:
    - allowedRequests: 0
      periodSeconds: 60

  ###################################
  #        REQUIRED TIERS           #
  ###################################
  free:
  - models:
    - Meta-Llama-3.1-8B-Instruct
    queueDepth: 100
    qos: "free"
    rates:
    - allowedRequests: 20
      periodSeconds: 60

  web:
  - models:
    - Meta-Llama-3.1-8B-Instruct
    queueDepth: 100
    qos: "web"
    rates:
    - allowedRequests: 20
      periodSeconds: 60

  ###################################
  #        CUSTOM TIERS             #
  ###################################
  developer:
    inherits: free
    overrides:
    - models:
      - Meta-Llama-3.1-8B-Instruct
      queueDepth: 100
      qos: "developer"
      rates:
      - allowedRequests: 60
        periodSeconds: 60

  enterprise:
    inherits: developer
    overrides:
    - models:
      - Meta-Llama-3.3-70B-Instruct
      queueDepth: 100
      qos: "enterprise"
      rates:
      - allowedRequests: 200
        periodSeconds: 60

​
Quality of Service (QoS)

Quality of Service (QoS) defines priority levels that determine how requests are processed across deployments when competing for resources. It ensures that higher-priority traffic receives precedence over lower-priority traffic, optimizing resource allocation during periods of contention.

​
QoS configuration

  • QoS levels are specified within service tier configurations by assigning a qos label to each tier.
  • Deployments define the priority order of QoS levels they support using the qosList in their deployment specifications.
Example configuration snippet from sambastack.yaml:
serviceTiers:
  - name: Standard
    qos: "free"

bundleDeploymentSpecs:
  - name: <Model Name>
    groups:
      - name: <Group Name>
        minReplicas: <number>
        qosList:
          - "web"
          - "free"
This example means the deployment prioritizes requests from the web tier first, then the free tier if no web requests are pending.

​
Purpose of QoS

QoS prioritizes requests so that higher-tier traffic is served before lower-tier traffic, ensuring predictable and fair resource sharing.
  • Example: A deployment listing qosList: [“free”, “web”] serves free tier requests first, falling back to web tier requests only when no free traffic is queued.

​
Important considerations

  • The Free tier is a default, hardcoded tier assigned to all new users automatically.
  • Deployments can be configured to support multiple QoS levels to handle different traffic types concurrently.

​
Difference between QoS and service tiers

ConceptControlsDefined In
Service TierAssigns users to models, rate limits, queue depth, and QoS labels. Determines access and usage restrictions.sambastack.yaml (serviceTiers section) or Admin UI (Usage Plans)
QoSPrioritizes real-time request dispatch among traffic from various tiers based on deployment capabilities.Deployment specifications (qosList in bundleDeploymentSpecs)
This means that:
  • Service tiers define who can access what and how much.
  • QoS defines when requests are processed based on priority.

​
User request handling workflow

The following outlines the step-by-step processing of a user request, illustrating how service tiers and QoS priorities interact to manage and route traffic efficiently.
  1. A user sends an API request using their credentials.
  2. SambaStack identifies the user’s assigned service tier (usage plan).
  3. The request is checked against that tier’s allowed models, batch size, rate limits, and associated QoS.
  4. The deployment selects requests to process according to its qosList priority.
  5. If the request exceeds the user’s rate limit, it is rejected with a 429 Too Many Requests response.
  6. If the QoS queue for the request’s priority level is full, the system returns a busy response.
  7. Otherwise, the request is placed in the QoS queue awaiting processing.
Administrators can adjust service tiers or quotas via the admin UI. Any changes to tiers, rate limits, or QoS settings apply cluster-wide and should be made by editing and deploying the YAML configuration, preferably under version control.