Customize SambaStack Deployment - SambaNova Documentation

SambaStack on-prem
SambaStack hosted

1. Custom OIDC Configuration

To integrate SambaStack with your organization’s identity provider, configure custom OIDC authentication.

Step 1: Create Kubernetes Secret

Create a secret with your OIDC provider details:

apiVersion: v1
kind: Secret
metadata:
  name: oidc-auth
  namespace: sambastack
stringData:
  OIDC_CLIENT_ID: "<client-id>"
  OIDC_CLIENT_SECRET: "<client-secret>"
  OIDC_ISSUER_URL: "https://auth.example.com/"
  OIDC_REDIRECT_URI: "https://ui.<yourdomain>/web/auth/callback"
  JWT_SECRET_KEY: "<random-string>"

Replace the placeholders with your actual values (all base64 encoded):

<client-id>: Your OIDC client ID
<client-secret>: Your OIDC client secret
<yourdomain>: Your organization’s domain
<random-string>: A securely generated random string for JWT signing

Step 2: Apply the Secret

kubectl apply -f oidc-auth.yaml

Step 3: Update sambastack.yaml

Add the OIDC configuration to your sambastack.yaml:

gateway:
  replicas: 3
  auth:
    enabled: true
    secretName: oidc-auth     # Must match the secret name created above

Step 4: Apply the Configuration

Update your Helm deployment:

helm upgrade sambastack \
  -f sambastack.yaml \
  --namespace sambastack \
  oci://<REGISTRY_URL>/sambastack/sambastack

SambaNova provides the full registry URL and version number during handover. Contact your SambaNova representative for access credentials.

Ensure the OIDC_REDIRECT_URI matches your UI domain’s callback endpoint: https://ui.<yourdomain>/web/auth/callback

2. Custom PostgreSQL Database

To use an external PostgreSQL database instead of the in-cluster deployment, you can create the secret using either base64-encoded values or literal values.

Step 1: Create the Secret

Option A: Base64-encoded Values

First, encode your database credentials:

echo -n "<db-host>" | base64
echo -n "<db-name>" | base64
echo -n "<db-user>" | base64
echo -n "<db-password>" | base64

Create a Kubernetes secret with base64-encoded values:

apiVersion: v1
kind: Secret
metadata:
  name: pg-credentials
  namespace: sambastack
type: Opaque
data:
  DB_HOST: <Base64 encoded DB_HOST>
  DB_DATABASE: <Base64 encoded DB_DATABASE>
  DB_USER: <Base64 encoded DB_USER>
  DB_PASSWD: <Base64 encoded DB_PASSWD>

Apply the secret:

kubectl apply -f pg-credentials.yaml

Option B: Using kubectl with Literal Values

Instead of manually encoding, you can create the secret with literal values:

kubectl create secret generic pg-credentials \
  --from-literal=DB_HOST=<host> \
  --from-literal=DB_DATABASE=<database> \
  --from-literal=DB_USER=<user> \
  --from-literal=DB_PASSWD=<password> \
  --namespace sambastack

Step 2: Update sambastack.yaml

Configure your sambastack.yaml to use the external database:

auth-and-billing:
  pgSecretName: pg-credentials

cloudnative-pg:
  enabled: false

auth-and-billing.pgSecretName: References the Kubernetes secret containing database credentials
cloudnative-pg.enabled: false: Disables the in-cluster PostgreSQL deployment

Step 3: Apply the Configuration

Update your Helm deployment:

helm upgrade sambastack \
  -f sambastack.yaml \
  --namespace sambastack \
  oci://<REGISTRY_URL>/sambastack/sambastack

SambaNova provides the full registry URL and version number during handover. Contact your SambaNova representative for access credentials.

Ensure your external PostgreSQL database is accessible from the Kubernetes cluster and that network policies allow the connection.

3. Rate Limiting

Rate limiting controls the number of API or UI requests accepted per minute to protect the cluster from overload and abuse.In SambaStack on-prem, rate limiting can be applied at two layers:

Layer	Description	Configuration Source	Status
Ingress Layer (NGINX)	Enforces limits at the Kubernetes ingress controller using NGINX annotations	NGINX ingress controller configuration	Optional (Recommended)
Application Layer (Service Tiers)	Enforces limits at the gateway level using Helm chart parameters	`sambastack.yaml` (`serviceTiers` section)	Required. See Service tier management

Application layer rate limits are mandatory and configured via service tiers. See Service tier management for details.

Overview

Ingress-layer rate limits are optional but recommended for customers using the NGINX ingress controller.
The Helm chart does not ship with default rate-limit annotations; customers can enable them manually.
Application layer rate limits remain mandatory and are configured in the serviceTiers block of sambastack.yaml.
If you bring your own Kubernetes cluster and ingress controller, you can use equivalent annotations or omit ingress-level limits entirely.

Ingress-Layer Rate Limiting (Optional, Recommended)

If your cluster uses RKE2’s built-in NGINX ingress controller, you can define global rate-limit zones at the cluster level and reference them in the API and UI ingress annotations.

Step 1: Define Global Rate-Limit Zones

Apply the following HelmChartConfig to configure global rate-limit zones for the ingress controller.Save the below as rke2-ingress-nginx.yaml:

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-ingress-nginx
  namespace: kube-system
spec:
  valuesContent: |-
    controller:
      config:
        limit-req-status-code: "429"
        allow-snippet-annotations: "true"
        http-snippet: |
          limit_req_zone $binary_remote_addr zone=240_req_min_ip:50m rate=240r/m;
          limit_req_zone $http_authorization zone=240_req_min_header:50m rate=240r/m;

Apply it:

kubectl apply -f rke2-ingress-nginx.yaml

Zone Definitions:

_req_min_ip zones are used for the UI (IP-based limits)
_req_min_header zones are used for the API/Gateway (Authorization header-based limits)

Rate limit values should scale with cluster size. For larger deployments (e.g., 10-node clusters), consider higher limits such as 5000 or 8000 requests per minute.

Step 2: Apply Ingress Annotations in sambastack.yaml

Once zones are defined, reference them in your sambastack.yaml under the cloud-ui.ingress and gateway.ingress sections:

cloud-ui:
  ingress:
    annotations:
      nginx.ingress.kubernetes.io/backend-protocol: HTTP
      nginx.ingress.kubernetes.io/force-ssl-redirect: 'true'
      nginx.ingress.kubernetes.io/proxy-body-size: 21m
      nginx.ingress.kubernetes.io/configuration-snippet: |
        proxy_set_header Authorization $http_authorization;
        limit_req zone=120_req_min_ip burst=240 nodelay;
        limit_req_status 429;

gateway:
  ingress:
    annotations:
      nginx.ingress.kubernetes.io/backend-protocol: HTTP
      nginx.ingress.kubernetes.io/force-ssl-redirect: 'true'
      nginx.ingress.kubernetes.io/proxy-read-timeout: '600'
      nginx.ingress.kubernetes.io/proxy-body-size: 25m
      nginx.ingress.kubernetes.io/enable-cors: 'true'
      nginx.ingress.kubernetes.io/configuration-snippet: |
        proxy_set_header Authorization $http_authorization;
        limit_req zone=240_req_min_header burst=360 nodelay;
        limit_req_status 429;

Configuration Details:

These snippets tell NGINX which rate-limit zone to apply
You can adjust the burst and rate values as needed
Ingress-based rate limiting is per ingress-controller pod, not cluster-wide
Total effective rate = configured rate × number of ingress pods

Step 3: Apply the Updated Configuration

helm upgrade sambastack \
  -f sambastack.yaml \
  --namespace sambastack \
  oci://<REGISTRY_URL>/sambastack/sambastack

SambaNova provides the full registry URL and version number during handover. Contact your SambaNova representative for access credentials.

Key Recommendations

Use ingress-layer limits for infrastructure-level protection (e.g., DDoS or abusive burst prevention)
Use service-tier limits for application-level throttling and user-specific rate control
Ingress configuration is optional, but SambaNova recommends enabling it when using RKE2’s built-in NGINX ingress controller
For customers using other ingress controllers (e.g., Istio, AWS ALB, or NGINX Plus), equivalent configurations can be applied at their discretion
Default Helm chart values will not include any rate-limit annotations; these can be overridden by customers at deployment time

Always configure application-layer rate limits via service tiers. Ingress-layer rate limiting alone is insufficient for proper request management.

This section describes optional configurations for SambaStack hosted deployments. You can customize your installation by using custom domains with TLS certificates or connecting to an external PostgreSQL database instead of the default in-cluster database.

Custom Domain and TLS Certificate Configuration

To use a custom domain with TLS on your SambaStack deployment, follow these steps:

1. Obtain TLS Certificates

Acquire the TLS certificate and private key for your domain from a trusted certificate authority.

2. Create Kubernetes TLS Secret

Follow the official Kubernetes documentation to create a Kubernetes secret for your TLS certificate and key.Example command:

kubectl create secret tls <name-for-tls> --cert=path/to/cert.crt --key=path/to/key.key

3. Configure DNS Records

Create DNS records to map your custom domains to the SambaStack servers as follows:

Domain	Record Type	Value
`api.example.com`	CNAME	`api.sambanova.ai`
`api.example.com`	TXT	`<From installer log>`
`ui.example.com`	CNAME	`cloud.sambanova.ai`
`ui.example.com`	TXT	`<From installer log>`

Contact SambaNova support to confirm your regional endpoint, if needed.

4. Update sambastack.yaml

Edit the sambastack.yaml configuration file under data → sambastack.yaml to include your domain and TLS secret information with correct indentation:

data:
  sambastack.yaml: |
    gateway:
      ingress:
        hosts:
          - host: api.example.com
            tlsSecretName: <name-for-tls>
    cloud-ui:
      ingress:
        hosts:
        - host: ui.example.com
          tlsSecretName: <name-for-tls>

5. Apply Configuration

Apply your changes:

kubectl apply -f sambastack.yaml

6. Sanity Check

Monitor the installer logs to verify TLS certificate processing and domain verification:

kubectl -n sambastack-installer logs -l sambanova.ai/app=sambastack-installer -f

After applying these changes, your SambaStack deployment will use the specified custom domains and TLS certificates.

Using External PostgreSQL Database

By default, SambaNova provisions a PostgreSQL instance within your cluster. To use an external PostgreSQL database, complete the following steps:

1. Provision External Database

Set up an external PostgreSQL instance and collect these credentials:

DB_HOST
DB_DATABASE
DB_USER
DB_PASSWD

2. Create Kubernetes Secret for PostgreSQL Credentials

Create a secret named pg-credentials with your database credentials:

cat <<EOPGSECRET | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: pg-credentials
  namespace: default
type: Opaque
data:
  DB_HOST: <Base64 encoded DB_HOST>
  DB_DATABASE: <Base64 encoded DB_DATABASE>
  DB_USER: <Base64 encoded DB_USER>
  DB_PASSWD: <Base64 encoded DB_PASSWD>
EOPGSECRET

Ensure all credential values are base64 encoded.

3. Update sambastack.yaml

Modify sambastack.yaml (data → sambastack.yaml) to reference your external PostgreSQL secret and disable the in-cluster database:

data:
  sambastack.yaml: |
    # Auth and Billing Configuration
    auth-and-billing:
      pgSecretName: pg-credentials # Kubernetes secret with PostgreSQL credentials
    # Database Configuration
    cloudnative-pg:
      enabled: false # Disable in-cluster PostgreSQL

4. Apply Updated Configuration

Apply the changes:

kubectl apply -f sambastack.yaml

After applying these changes, your deployment will connect to the configured external PostgreSQL database.

Overview

Installation

Service Administration

Resources

Optional Configuration

1. Custom OIDC Configuration

Step 1: Create Kubernetes Secret

Step 2: Apply the Secret

Step 3: Update sambastack.yaml

Step 4: Apply the Configuration

2. Custom PostgreSQL Database

Step 1: Create the Secret

Option A: Base64-encoded Values

Option B: Using kubectl with Literal Values

Step 2: Update sambastack.yaml

Step 3: Apply the Configuration

3. Rate Limiting

Overview

Ingress-Layer Rate Limiting (Optional, Recommended)

Step 1: Define Global Rate-Limit Zones

Step 2: Apply Ingress Annotations in sambastack.yaml

Step 3: Apply the Updated Configuration

Key Recommendations

Custom Domain and TLS Certificate Configuration

1. Obtain TLS Certificates

2. Create Kubernetes TLS Secret

3. Configure DNS Records

4. Update sambastack.yaml

5. Apply Configuration

6. Sanity Check

Using External PostgreSQL Database

1. Provision External Database

2. Create Kubernetes Secret for PostgreSQL Credentials

3. Update sambastack.yaml

4. Apply Updated Configuration

Overview

Installation

Service Administration

Resources

​1. Custom OIDC Configuration

​Step 1: Create Kubernetes Secret

​Step 2: Apply the Secret

​Step 3: Update sambastack.yaml

​Step 4: Apply the Configuration

​2. Custom PostgreSQL Database

​Step 1: Create the Secret

​Option A: Base64-encoded Values

​Option B: Using kubectl with Literal Values

​Step 2: Update sambastack.yaml

​Step 3: Apply the Configuration

​3. Rate Limiting

​Overview

​Ingress-Layer Rate Limiting (Optional, Recommended)

​Step 1: Define Global Rate-Limit Zones

​Step 2: Apply Ingress Annotations in sambastack.yaml

​Step 3: Apply the Updated Configuration

​Key Recommendations

​Custom Domain and TLS Certificate Configuration

​1. Obtain TLS Certificates

​2. Create Kubernetes TLS Secret

​3. Configure DNS Records

​4. Update sambastack.yaml

​5. Apply Configuration

​6. Sanity Check

​Using External PostgreSQL Database

​1. Provision External Database

​2. Create Kubernetes Secret for PostgreSQL Credentials

​3. Update sambastack.yaml

​4. Apply Updated Configuration

1. Custom OIDC Configuration

Step 1: Create Kubernetes Secret

Step 2: Apply the Secret

Step 3: Update sambastack.yaml

Step 4: Apply the Configuration

2. Custom PostgreSQL Database

Step 1: Create the Secret

Option A: Base64-encoded Values

Option B: Using kubectl with Literal Values

Step 2: Update sambastack.yaml

Step 3: Apply the Configuration

3. Rate Limiting

Overview

Ingress-Layer Rate Limiting (Optional, Recommended)

Step 1: Define Global Rate-Limit Zones

Step 2: Apply Ingress Annotations in sambastack.yaml

Step 3: Apply the Updated Configuration

Key Recommendations

Custom Domain and TLS Certificate Configuration

1. Obtain TLS Certificates

2. Create Kubernetes TLS Secret

3. Configure DNS Records

4. Update sambastack.yaml

5. Apply Configuration

6. Sanity Check

Using External PostgreSQL Database

1. Provision External Database

2. Create Kubernetes Secret for PostgreSQL Credentials

3. Update sambastack.yaml

4. Apply Updated Configuration