> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Shared nodes

> [!NOTE]
>
> vCluster and Envoy Gateway are third-party products. SambaNova doesn't maintain, endorse, or audit them. This reference architecture is provided for informational purposes and may become outdated as they change.

In the shared-node model, a tenant's RDU nodes stay members of the host cluster. You label each node for one tenant, and vCluster shows the tenant only those nodes and places its pods only there. The tenant control plane runs as pods on a CPU node pool of the host. Nothing is installed on the RDU nodes.

Use this model for trusted tenants inside your organization. For external or untrusted tenants, use [private nodes](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/reference-architecture/multi-tenancy/private-nodes.md). See the [overview](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/reference-architecture/multi-tenancy/overview.md) for the comparison.

> [!NOTE]
>
> **Tested with:** SambaStack 2.3.2, RKE2 v1.35.5, vCluster Helm chart 0.35.4 (open-source image), tenant Kubernetes v1.35.0, Envoy Gateway v1.9.1.

## Architecture

```mermaid theme={}
graph TB
    accTitle: Shared-node tenant on a SambaStack host cluster
    accDescr: An operator reaches the tenant API through a TLS passthrough gateway on the host cluster. The tenant control plane runs as pods in a host namespace on the CPU node pool. The syncer copies tenant pods into that namespace and the host scheduler places them on the RDU node labeled for the tenant. Other RDU nodes are hidden from the tenant.

    OP["Operator<br/><small>kubectl · tenant1-api.example.com:30843</small>"]

    subgraph HOST["Host cluster"]
        GW["TLS passthrough gateway<br/><small>installed once · routes by SNI hostname</small>"]
        subgraph NS["Namespace vcluster-tenant1 · CPU node pool"]
            CP["Tenant control plane<br/><small>API server · syncer · controller manager · etcd</small>"]
        end
        subgraph RDU["RDU node · label vcluster=tenant1"]
            PODS["Tenant pods<br/><small>copied by the syncer · placed by the host scheduler</small>"]
        end
        OTHER["Other RDU nodes<br/><small>not visible to the tenant</small>"]
    end

    OP -->|"HTTPS"| GW
    GW -->|"TLSRoute for the tenant hostname"| CP
    CP -->|"pods synced to the host namespace"| PODS

    classDef host fill:#eef2f7,stroke:#8aa0bd,color:#1a2b45;
    classDef tenant fill:#cfe2f3,stroke:#2f6fb0,color:#0d2c4d,stroke-width:2px;
    class GW,OTHER,RDU,HOST host;
    class CP,PODS,NS tenant;
```

How the pieces work together:

1. **Claim.** You label a host RDU node `vcluster=tenant1`. The tenant's node sync selector matches that label, so the tenant sees only its nodes, and the syncer sets the same node selector on every pod it copies to the host.
2. **Control traffic.** `kubectl` connects to the tenant hostname on the gateway's NodePort. The gateway reads the SNI hostname and passes the connection, still encrypted, to the tenant's API server service. The tenant's API certificate includes the hostname.
3. **Pods.** The tenant creates pods in its own API server. The syncer copies them into the host namespace `vcluster-tenant1` under rewritten names, and the host scheduler places them, since the tenant's own scheduler is off.
4. **Logs and exec.** The vCluster syncer forwards these to the host API server for the synced pod. No node agent is needed.
5. **Pod network.** Tenant pods use the host CNI and the host's pod and service CIDRs.
6. **App traffic.** Tenant Ingress objects sync to the host, and the host's ingress controller serves them.

## Prerequisites

Host cluster, set up once:

* A host cluster that meets the SambaStack [on-prem requirements](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/getting-started/on-prem.md#prerequisites), with the RDU nodes already labeled `snRduArch=sn40-16` as that guide describes.
* A CPU node pool labeled `control-plane=shared` for tenant control planes. Without it the tenant's API server and etcd pods stay `Pending`.
* A storage class for tenant control plane state. This reference uses `openebs-hostpath`.
* A gateway or load balancer that routes TLS by SNI hostname without terminating it. This reference uses Envoy Gateway. NGINX or HAProxy in passthrough mode also work.
* A DNS name per tenant that resolves to the gateway, for example `tenant1-api.example.com`.

Per tenant:

* One or more RDU worker nodes that are `Ready` and not labeled for another tenant.
* Unique hostnames and TLS certificates for the tenant's SambaStack API and UI, as in the on-prem requirements.

On the operator's machine: `kubectl`, `helm` v3.19+, and the [vCluster CLI](https://www.vcluster.com/docs/vcluster/deploy/control-plane/kubernetes-pod/basics).

## Components

| Layer | Component | Provided by | Runs on |
| :- | :- | :- | :- |
| Kubelet, container runtime, CNI, kube-proxy | RKE2 | Host cluster | RDU node |
| Scheduler for tenant pods | kube-scheduler | Host cluster | Host control plane |
| RDU node software and labels | SambaStack host install | Host cluster | RDU node |
| Tenant control plane | API server, controller manager, syncer, etcd | vCluster chart | Host namespace `vcluster-tenant1`, CPU node pool |
| Tenant DNS | CoreDNS | vCluster chart | Synced pod on the tenant's RDU node |
| Tenant API entry point | Envoy Gateway, `Gateway`, `TLSRoute` | You, once per host | Host |
| Tenant app ingress | Host ingress controller, through Ingress sync | Host cluster | Host |
| SambaStack | `sambastack-base`, `sambastack`, `sambastack-models` | You, per tenant | Tenant, synced to the RDU node |

## Set up a tenant

Commands run against the host cluster unless noted. Replace `tenant1`, `vcluster-tenant1`, `tenant1-api.example.com`, and `<rdu-node>` with your tenant's values.

> [!NOTE]
>
> Envoy Gateway is the gateway this reference was tested with, not a requirement. Any gateway or load balancer that routes TLS by SNI hostname without terminating it works in its place. Steps 1 and 2 are specific to Envoy Gateway, and `controlPlane.tlsRoute` in step 4 creates a Gateway API `TLSRoute`. With a gateway that doesn't implement the Gateway API, leave `tlsRoute` out and route the tenant hostname to the `tenant1` service on port 443 in that gateway's own configuration.

### 1. Install the gateway (once per host)

```shellscript theme={}
helm upgrade --install envoy-gateway oci://docker.io/envoyproxy/gateway-helm \
  --version v1.9.1 \
  --namespace envoy-gateway-system --create-namespace
kubectl wait -n envoy-gateway-system deploy/envoy-gateway --for=condition=Available --timeout=5m
```

### 2. Create the TLS passthrough listener (once per host)

One listener on port 8443 in passthrough mode, exposed as NodePort 30843. Every tenant API shares it, routed by hostname.

```yaml gateway.yaml theme={}
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: envoy
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: envoy-gateway
  namespace: envoy-gateway-system
spec:
  gatewayClassName: envoy
  infrastructure:
    parametersRef:
      group: gateway.envoyproxy.io
      kind: EnvoyProxy
      name: envoy-gateway-proxy-config
  listeners:
    - name: vcluster-api
      protocol: TLS
      port: 8443
      tls:
        mode: Passthrough
      allowedRoutes:
        namespaces:
          from: All
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-gateway-proxy-config
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        type: NodePort
        externalTrafficPolicy: Cluster
        patch:
          type: StrategicMerge
          value:
            spec:
              ports:
                - name: vcluster-api
                  port: 8443
                  targetPort: 8443
                  nodePort: 30843
                  protocol: TCP
```

```shellscript theme={}
kubectl apply -f gateway.yaml
```

### 3. Label the tenant's RDU nodes

```shellscript theme={}
# Confirm the node is Ready, a worker, and not labeled for another tenant
kubectl get node <rdu-node> --show-labels | tr ',' '\n' | grep -E 'node-role|vcluster|snRduArch'

# Claim it for the tenant
kubectl label node <rdu-node> vcluster=tenant1 --overwrite

# Verify the tenant's nodes and the control plane pool
kubectl get nodes -l vcluster=tenant1
kubectl get nodes -l control-plane=shared
```

### 4. Write the tenant's vCluster values

The settings that carry the design:

| Setting | Value | Why |
| :- | :- | :- |
| `controlPlane.proxy.extraSANs` | `tenant1-api.example.com` | The API certificate is valid for the tenant hostname |
| `controlPlane.tlsRoute` | host `tenant1-api.example.com`, parent `envoy-gateway` | The gateway routes this hostname to this tenant |
| `controlPlane.statefulSet.scheduling.nodeSelector` | `control-plane: shared` | API server and etcd run on the CPU pool, not on RDU nodes |
| `controlPlane.distro.k8s.scheduler.enabled` | `false` | The host scheduler places synced pods |
| `controlPlane.coredns.deployment.nodeSelector` | `vcluster: tenant1` | Tenant DNS runs on the tenant's node |
| `sync.fromHost.nodes.selector.labels` | `vcluster: tenant1` | The tenant sees only the nodes with this label, and synced pods get the same node selector |
| `sync.fromHost.nodes.syncBackChanges` | `false` | Tenants can't edit host node labels or taints, including their own `vcluster` label. See the warning below |
| `sync.toHost.ingresses` | `true` | Tenant Ingresses are served by the host ingress controller |

```yaml vcluster.yaml theme={}
controlPlane:
  proxy:
    extraSANs:
      - tenant1-api.example.com
      - tenant1.vcluster-tenant1.svc.cluster.local
  tlsRoute:
    enabled: true
    host: tenant1-api.example.com
    parentRefs:
      - name: envoy-gateway
        namespace: envoy-gateway-system
  statefulSet:
    highAvailability:
      replicas: 1
    image:
      registry: ghcr.io
      repository: loft-sh/vcluster-oss
    persistence:
      volumeClaim:
        enabled: true
        retentionPolicy: Delete
        storageClass: openebs-hostpath
    scheduling:
      nodeSelector:
        control-plane: shared
      podManagementPolicy: Parallel
      tolerations:
        - operator: Exists
  backingStore:
    etcd:
      deploy:
        enabled: true
        statefulSet:
          highAvailability:
            replicas: 1
          persistence:
            volumeClaim:
              accessModes: [ReadWriteOnce]
              enabled: true
              retentionPolicy: Delete
              size: 10Gi
              storageClass: openebs-hostpath
          scheduling:
            nodeSelector:
              control-plane: shared
            podManagementPolicy: Parallel
            tolerations:
              - operator: Exists
  coredns:
    enabled: true
    deployment:
      nodeSelector:
        vcluster: tenant1
      replicas: 1
  distro:
    k8s:
      enabled: true
      image:
        registry: ghcr.io
        repository: loft-sh/kubernetes
        tag: v1.35.0
      apiServer:
        enabled: true
        extraArgs:
          - --enable-admission-plugins=PodNodeSelector,PodTolerationRestriction
      controllerManager:
        enabled: true
      scheduler:
        enabled: false
networking:
  advanced:
    fallbackHostCluster: true
sync:
  fromHost:
    csiDrivers: {enabled: auto}
    csiNodes: {enabled: auto}
    csiStorageCapacities: {enabled: auto}
    events: {enabled: true}
    ingressClasses: {enabled: false}
    storageClasses: {enabled: auto}
    nodes:
      enabled: true
      clearImageStatus: false
      syncBackChanges: false
      selector:
        all: false
        labels:
          vcluster: tenant1
  toHost:
    configMaps: {all: false, enabled: true}
    endpoints: {enabled: true}
    ingresses: {enabled: true}
    networkPolicies: {enabled: false}
    persistentVolumeClaims: {enabled: true}
    persistentVolumes: {enabled: false}
    podDisruptionBudgets: {enabled: false}
    priorityClasses: {enabled: false}
    secrets: {all: false, enabled: true}
    serviceAccounts: {enabled: false}
    services: {enabled: true}
    storageClasses: {enabled: true}
    pods:
      enabled: true
      rewriteHosts:
        enabled: true
```

> [!NOTE]
>
> `PodNodeSelector` and `PodTolerationRestriction` let you pin a tenant namespace to specific nodes or tolerations through namespace annotations. They are optional; the node sync selector already confines tenant pods to labeled nodes.

> [!WARNING]
>
> Tooling that labels nodes from inside the tenant, for example to drive a rolling runtime upgrade node by node, needs `syncBackChanges: true`. That setting also lets a tenant administrator change or remove any label or taint on its host nodes, including the `vcluster` label that confines it. Enable it only for tenants you trust to manage their own nodes, and keep the tenant's cluster-admin kubeconfig with the platform team.

### 5. Install the tenant

```shellscript theme={}
helm repo add loft-sh https://charts.loft.sh && helm repo update

helm upgrade --install tenant1 loft-sh/vcluster \
  --version 0.35.4 \
  --namespace vcluster-tenant1 --create-namespace \
  -f vcluster.yaml

kubectl -n vcluster-tenant1 get pods -w
```

`tenant1-etcd-0` comes up first. `tenant1-0`, which runs the API server and syncer, is `Running` about a minute after the install.

### 6. Create DNS and connect

Create a DNS record for `tenant1-api.example.com` that resolves to a host node IP or to a VIP in front of the host nodes. Then write a tenant kubeconfig that points at the gateway's NodePort:

```shellscript theme={}
vcluster connect tenant1 -n vcluster-tenant1 --print \
  --server https://tenant1-api.example.com:30843 > tenant1.kubeconfig

export KUBECONFIG=tenant1.kubeconfig
kubectl get nodes
```

`kubectl get nodes` returns only the nodes labeled `vcluster=tenant1`.

> [!WARNING]
>
> The kubeconfig holds a cluster-admin client certificate for the tenant. Treat it like a password. Anyone with access to the host cluster can generate one with the same command.

## Verify

From the tenant:

```shellscript theme={}
export KUBECONFIG=tenant1.kubeconfig
kubectl get nodes -o wide          # only the tenant's labeled nodes
kubectl -n kube-system get pods     # CoreDNS, scheduled on the tenant's node
```

From the host:

```shellscript theme={}
unset KUBECONFIG
kubectl -n vcluster-tenant1 get pods -o wide
```

The host namespace shows the control plane pods on the CPU pool and the synced tenant pods, under rewritten names, on the labeled RDU node.

## Install SambaStack in the tenant

Install SambaStack with the standard [on-prem quickstart](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/getting-started/on-prem.md), using the tenant kubeconfig (`export KUBECONFIG=tenant1.kubeconfig`). Only these points differ from a normal install:

* **Skip the quickstart's node labeling step.** The `snRduArch=sn40-16` label is set on the host and reaches the tenant through node sync.
* **Ingress and DNS.** Tenant Ingress objects sync to the host and the host's ingress controller serves them. Point the tenant's API and UI hostnames at the host ingress, and keep them unique across tenants. The hostnames must also resolve from inside the tenant, and the TLS certificate must come from a trusted CA, as the quickstart requires.

From the host, the tenant's pods appear in `vcluster-tenant1` under rewritten names, all on the tenant's labeled nodes: `kubectl -n vcluster-tenant1 get pods -o wide`.

## Limitations

* The `vcluster=tenant1` label is a scheduling rule, not a security boundary. A tenant with cluster-admin access to its virtual cluster shares the host's kernel, CNI, and kubelets. Use [private nodes](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/reference-architecture/multi-tenancy/private-nodes.md) for those tenants.
* The SambaStack platform services run on the tenant's RDU nodes, so maintenance on those nodes interrupts the tenant's API, UI, and login. To keep them apart, use [private nodes](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/reference-architecture/multi-tenancy/private-nodes.md) with dedicated CPU nodes.
* Host administrators see the tenant's pods in the host namespace, and host DaemonSets such as monitoring agents run on the tenant's nodes.
* With `syncBackChanges: true`, a tenant administrator can edit its host nodes' labels and taints. This reference sets it to `false`; see the warning in the values step.
* Tenant pods share the host pod network. To restrict traffic between tenants, apply `NetworkPolicy` resources in each tenant's host namespace, or enable `sync.toHost.networkPolicies` so tenants manage their own.
* This reference runs one replica each of the API server and etcd on host-path storage. For production, raise `highAvailability.replicas` and back up the etcd volume.
* The tenant API is reached through one NodePort on one host node IP. Put a VIP or load balancer in front of the host nodes for production.

## Cleanup

```shellscript theme={}
# Tenant: remove SambaStack first
export KUBECONFIG=tenant1.kubeconfig
helm uninstall sambastack-models sambastack sambastack-base -n sambastack

# Host: remove the tenant and release the node
unset KUBECONFIG
helm uninstall tenant1 -n vcluster-tenant1
kubectl delete namespace vcluster-tenant1
kubectl label node <rdu-node> vcluster-
```

With `retentionPolicy: Delete`, the control plane volumes and all tenant state are removed with the Helm release. The gateway stays for other tenants.

## Additional resources

* [vCluster: sync nodes from the host](https://www.vcluster.com/docs/vcluster/configure/vcluster-yaml/sync/from-host/nodes)
* [vCluster production guide: choose a worker node model](https://www.vcluster.com/docs/vcluster/production-guide/choose-worker-node-model)
* [Envoy Gateway: TLS passthrough](https://gateway.envoyproxy.io/docs/tasks/security/tls-passthrough/)