It is recommended to configure Prometheus to scrape these exporters every 60 seconds. A 30–60 second interval gives you timely visibility into hardware health without excessive storage overhead.
Metric labels
Every metric carries labels that let you filter and group by hardware topology. Labels come from:- Kubernetes metadata –
namespace,job(ServiceMonitor name) - Your
values.yaml–node,xrdu,rack,group, and any additional custom fields you add to target entries (Manual discovery mode only)
SambaNova Exporter
Installed on each host via the SambaRack runtime RPM. The DaemonSet is a thin wrapper that makes the host-side exporter discoverable by Prometheus.Already running on your nodes if SambaStack is installed.
Metrics
| Metric | Type | Labels | Description |
|---|---|---|---|
sambanova_tile_aload_percent | Gauge | instance, rdu, tile, xrdu | Arithmetic load percentage per RDU tile |
sambanova_tile_exec_percent | Gauge | instance, rdu, tile, xrdu | Execution time percentage – how much of the time the tile is actively computing |
sambanova_tile_frequency_mhz | Gauge | instance, rdu, tile, xrdu | Current operating frequency of each tile |
sambanova_platform_fault_count | Gauge | instance, namespace | Cumulative fault count for the SambaNova platform |
sambanova_platform_error_count | Gauge | instance, namespace | Cumulative error count |
sambanova_ddrch_rdB_bytes | Gauge | instance, rdu | Cumulative DDR channel bytes read – use rate(sambanova_ddrch_rdB_bytes[5m]) to get read bandwidth |
sambanova_hbmch_grp_rdB_bytes | Gauge | instance, rdu | Cumulative HBM channel group bytes read – use rate(sambanova_hbmch_grp_rdB_bytes[5m]) to get read bandwidth |
Host BMC IPMI Exporter
Runs on every SambaRack node and reads IPMI data from the host BMC via the/dev/ipmi0 device.
Requires the
ipmi_devintf kernel module to be loaded. Connects to the host BMC on each node – no additional IP configuration needed.Metrics
| Metric | Type | Labels | Description |
|---|---|---|---|
ipmi_up | Gauge | instance, namespace | 1 if the exporter is successfully reaching the BMC, 0 otherwise |
ipmi_power_watts | Gauge | instance, namespace | Instantaneous host server power draw |
ipmi_dcmi_power_consumption_watts | Gauge | instance, namespace | Platform-level power via DCMI (more reliable single value per host) |
ipmi_temperature_celsius | Gauge | instance, namespace, name | Temperature sensor reading; name identifies the sensor (e.g. Inlet Temp, CPU Temp) |
ipmi_fan_speed_rpm | Gauge | instance, namespace, name | Fan speed in RPM; name identifies the fan |
ipmi_voltage_volts | Gauge | instance, namespace, name | Voltage sensor reading |
ipmi_sensor_state | Gauge | instance, namespace, name, type | Sensor health state (0 = ok, 1 = warning, 2 = critical) |
XRDU BMC Exporter
Connects out from each cluster node to the XRDU BMC management IPs configured invalues.yaml. Each pod scrapes only the XRDUs assigned to its own node.
Verify XRDU BMC IPs with
snctl node show <node-name>.Metrics
| Metric | Type | Labels | Description |
|---|---|---|---|
xrdu_bmc_power_watts | Gauge | instance, target, xrdu, rack, group, sensor | XRDU power reading per sensor; sensor identifies the power supply (e.g. ps1_input) |
xrdu_bmc_temp_celsisus | Gauge | instance, target, xrdu, rack, sensor | XRDU component temperature |
xrdu_bmc_fan_rpms | Gauge | instance, target, xrdu, rack, sensor | XRDU fan speed in RPM |
xrdu_bmc_vddc_volts | Gauge | instance, target, xrdu, rack, sensor | XRDU voltage |
xrdu_bmc_vddc_amps | Gauge | instance, target, xrdu, rack, sensor | XRDU current in amps |
xrdu_bmc_cpu_utilization | Gauge | instance, target, xrdu, rack | XRDU BMC CPU utilization |
xrdu_bmc_memory_utilization | Gauge | instance, target, xrdu, rack | XRDU BMC memory utilization |
xrdu_bmc_storage_utilization | Gauge | instance, target, xrdu, rack | XRDU BMC storage utilization |
What’s next
Building dashboards in Grafana
Use these metrics to build a complete hardware monitoring dashboard with topology drill-down