The reference architecture described here is SambaNova’s suggested implementation, but is completely optional. SambaNova provides an example of a default monitoring stack based on widely used open-source tools.
Many customers already have mature monitoring solutions. The SambaStack monitoring architecture is modular, so you can:
- Adopt the full stack as provided, or
- Swap individual components with equivalents from your existing observability platform (Splunk, Datadog, Elasticsearch, New Relic, etc.)
Reference architectures use third-party products. There is no guarantee that they will be updated in sync with version or command syntax changes. Address any issues not specific to SambaStack to the respective vendor.
Components
SambaStack’s reference monitoring stack uses four primary components:
| Component | Tool | Description |
|---|
| Log Forwarder | Fluent Bit | Collects logs from Kubernetes pods, nodes, and system services. Parses, enriches, and forwards logs to OpenSearch. |
| Log Storage | OpenSearch | Stores logs and audit trails at scale. Provides search, filtering, and aggregation for log data. |
| Metrics Collection | Prometheus | Scrapes metrics from SambaStack services, Kubernetes components, and node exporters. Stores time-series data for monitoring and alerting. |
| Visualization | Grafana | Connects to Prometheus and OpenSearch. Provides dashboards for metrics and log exploration. |
Architecture
VISUALIZATION
+--------------------------------------------------------------------+
| Grafana |
| (Dashboards, Alerts, Log Exploration) |
+--------------------------------------------------------------------+
| |
v v
+-------------------------------+ +-----------------------------+
| Prometheus | | OpenSearch |
| (Metrics Storage) | | (Log Storage) |
+-------------------------------+ +-----------------------------+
^ ^
| |
+-------------------------------+ +-----------------------------+
| Node Exporter | | Fluent Bit |
| (Host Metrics) | | (Log Forwarding) |
+-------------------------------+ +-----------------------------+
^ ^
| |
+--------------------------------------------------------------------+
| SAMBASTACK |
+--------------------------------------------------------------------+
| Inference Router | Model Deployments | Kubernetes Pods/Nodes |
+--------------------------------------------------------------------+
Design principles
- Modular integration — Each component exposes well-defined interfaces (Fluent Bit outputs, Prometheus remote_write, Grafana data sources). You can replace any component with an equivalent.
- Kubernetes-native — All components run on and integrate with the Kubernetes cluster where SambaStack workloads are deployed.
- Bring-your-own stack — Integrate with your existing log platform, metrics system, or visualization layer.
- Security and compliance ready — Logging, metrics, and audit data can integrate with your existing SIEM and compliance tooling.
Component substitution
You can replace any component with an equivalent from your existing observability platform:
| Component | Replaceable with | Requirements |
|---|
| OpenSearch | Elasticsearch, Splunk, Loki, Datadog Logs | Must accept logs via HTTP, gRPC, or Kafka |
| Fluent Bit | Fluentd, Vector, Logstash, Datadog Agent | Must support Kubernetes log collection |
| Prometheus | Managed Prometheus, Datadog, New Relic | Must scrape /metrics endpoints or accept remote_write |
| Grafana | Datadog dashboards, Kibana, custom tooling | Must integrate with both metrics and log sources |
Prerequisites
Before deploying the monitoring stack, ensure you have:
| Requirement | Description |
|---|
| Kubernetes cluster | A running cluster with SambaStack installed |
| kubectl | Configured with access to your target Kubernetes cluster |
| Helm (latest version) | For deploying all Helm charts |
| jq | For parsing JSON output during verification |
| Storage class | A valid storage class for persistent volumes (OpenSearch and Prometheus) |
Directory structure
Create this directory structure before starting:
mkdir -p ~/.sambastack-observability/{opensearch,fluentbit,monitoring}
~/.sambastack-observability/
|-- monitoring-namespace.yaml
|-- opensearch/
| |-- opensearch-initial-admin-password-secret.yaml
| +-- opensearch-values.yaml
|-- fluentbit/
| |-- append_tags.lua
| |-- fluentbit-conf.conf
| +-- fluentbit-values.yml
+-- monitoring/
|-- grafana-initial-admin-credentials-secret.yaml
|-- inference-router-sm.yaml
+-- prometheus-grafana-values.yaml
Deployment order
Deploy components in this order:
| Step | Component | Guide |
|---|
| 1 | OpenSearch | Log Storage |
| 2 | Fluent Bit | Log Forwarding |
| 3 | Prometheus and Grafana | Monitoring |
Resource requirements
| Component | CPU | Memory | Storage |
|---|
| OpenSearch | 2-4 cores | 4-8 GB | 50-100 GB |
| Fluent Bit (per node) | 100m | 128Mi | — |
| Prometheus | 500m | 2Gi | 30Gi |
| Grafana | 250m | 512Mi | — |
| Node Exporter (per node) | 100m | 128Mi | — |
