SambaRack Manager Device Administration

This guide covers checking and updating firmware on devices within SambaRack. Devices are individual components attached to racks and nodes, including BMCs, switches, PDUs, and serial terminals.

Commands

Check firmware version

Check firmware versions on devices:

snctl device firmware-check \
  (--rack-id <rack_id> | --group <group_name>) \
  --type <device_type> \
  [--index <index>]

Target selection: One of --rack-id or --group is required (mutually exclusive):

Parameter	Description
`--rack-id <rack_id>`	Check devices in a specific rack
`--group <group_name>`	Check devices in all racks within a group

Device types:

Type	Description
`access_switch`	Access switch
`data_switch`	Data switch
`scs`	System Control Switch
`bios`	BIOS
`bmc`	Baseboard Management Controller
`card`	Card/Adapter
`pdu`	Power Distribution Unit
`xrdu`	XRDU
`all`	All device types

Single rack

Example – check BMC firmware on a single rack:

snctl device firmware-check --rack-id 001 --type bmc

Example output:

             Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component ┃ Status ┃ CURR     ┃ POR      ┃
┡━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ BMC       │ PASS   │ 12.61.21 │ 12.61.21 │
└──────────┴───────────┴────────┴──────────┴──────────┘

Example – check all firmware on a rack:

snctl device firmware-check --rack-id 001 --type all

Example output:

                                        Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component     ┃ Status ┃ CURR                        ┃ POR         ┃ OBMC_CURR ┃ OBMC_POR ┃ RDUC_CURR ┃ RDUC_POR ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ ACCESS_SWITCH │ FAIL   │                             │             │           │          │           │          │
│ Rack-001 │ BIOS          │ PASS   │ R24_P1                      │ R24_P1      │           │          │           │          │
│ Rack-001 │ BMC           │ PASS   │ 12.61.21                    │ 12.61.21    │           │          │           │          │
│ Rack-001 │ CARD          │ PASS   │ snhni0 -> 228.1.111.0  PASS │ 228.1.111.0 │           │          │           │          │
│          │               │        │ snni0 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni2 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni4 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni6 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni8 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni10 -> 228.1.111.0  PASS │             │           │          │           │          │
│          │               │        │ snni12 -> 228.1.111.0  PASS │             │           │          │           │          │
│          │               │        │ snni14 -> 228.1.111.0  PASS │             │           │          │           │          │
│ Rack-001 │ DATA_SWITCH   │ FAIL   │                             │             │           │          │           │          │
│ Rack-001 │ PDU_PDU_1     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_2     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_3     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_4     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ SCS           │ FAIL   │ 9.4.0.1R1                   │ 9.5.0.0R15  │           │          │           │          │
│ Rack-001 │ XRDU_XRDU_0   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_1   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_2   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_3   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_4   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_5   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_6   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_7   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
└──────────┴───────────────┴────────┴─────────────────────────────┴─────────────┴───────────┴──────────┴───────────┴──────────┘

Group

Example – check BMC firmware across all racks in a group:

snctl device firmware-check --group g1 --type bmc

Example output (one row per rack):

             Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component ┃ Status ┃ CURR     ┃ POR      ┃
┡━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ BMC       │ PASS   │ 12.61.21 │ 12.61.21 │
│ Rack-002 │ BMC       │ FAIL   │ 12.85.05 │ 12.61.21 │
└──────────┴───────────┴────────┴──────────┴──────────┘

If devices are unreachable or configured incorrectly, the firmware check fails for those devices with a FAIL status, as shown in the example output above. For card devices, the CURR column lists each network interface with its firmware version and a per-interface PASS/FAIL comparison. The overall CARD status is PASS only when every interface matches the POR version.

Update firmware

Update firmware on devices within a rack or group:

snctl device firmware-update \
  (--rack-id <rack_id> | --group <group_name>) \
  --type <device_type> \
  [--index <index_or_list>] \
  [--file <path> ...] \
  [--force]

Target selection: One of --rack-id or --group is required (mutually exclusive):

Parameter	Description
`--rack-id <rack_id>`	Update devices in a specific rack
`--group <group_name>`	Update devices in all racks within a group

Device types:

Type	Description
`bios`	BIOS
`bmc`	Baseboard Management Controller
`card`	Card/Adapter
`xrdu`	XRDU

Other device types (access_switch, data_switch, pdu, scs) are not currently supported for firmware update and will be rejected by the CLI.

Optional parameters:

Parameter	Description
`--index <index_or_list>`	Device index to update. For XRDU: omit to target all XRDUs, pass a single index (0–7), or a comma-separated list (e.g. `0,2,5`). Multiple indices run in parallel within the rack.
`--file <path>`	Local firmware file. For XRDU, pass one combined archive whose name starts with `xrdu` (e.g. `xrdu-combined-5.1.4.tgz`), or two split archives whose names start with `obmc` and `rduc` respectively. Repeat the flag for two files: `--file obmc-5.1.4.tgz --file rduc-5.1.4.tgz`. If omitted, the firmware is downloaded from the configured artifact store.
`--force`	Skip checksum validation

Single rack

Example – update BMC firmware:

snctl device firmware-update --rack-id 001 --type bmc

Example – update all XRDUs:

snctl device firmware-update --rack-id 001 --type xrdu

Example – update specific XRDUs:

snctl device firmware-update --rack-id 001 --type xrdu --index 0,2,5

Example – update XRDU firmware from local files:

snctl device firmware-update --rack-id 001 --type xrdu --index 3 \
  --file /path/to/obmc-5.1.4.tgz --file /path/to/rduc-5.1.4.tgz --force

Group

Example – update firmware across all racks in a group:

snctl device firmware-update --group g1 --type bmc

When you use --group, snctl processes racks sequentially (one rack at a time) while running individual device updates in parallel within each rack. This bounds concurrency – for example, a group with 10 racks and 8 XRDUs each won’t launch 80 simultaneous jobs. During the update, snctl prints:

Per-component PASS/FAIL lines as each job completes
Per-rack checkpoints after each rack finishes, including PASS/FAIL counts and log file paths for any failures
A final Firmware Update Results table summarizing the entire run

If one or more devices fail, the command exits non-zero and prints the stdout, stderr, and instruction output log paths for each failed job.

Order of execution

When updating multiple components, there is no strict dependency between them. However, if upgrading more than one type in the same maintenance window, follow this order: THOR CARD → BIOS → BMC → XRDU

Device	Pre-condition	Post-upgrade operations	Expected duration
THOR CARD	Host must be powered on	Power cycle, firmware check	~25 min (~5 min upgrade + ~20 min power cycle)
BIOS	Host must be powered on	Host power off/on, SR-IOV verify, Node Powercycle, Firmware Check	1 hour (~5 mins upgrade + ~30 mins SR-IOV disable & verify + ~20 mins powercycle)
BMC	None	Power cycle, firmware check	~25 min (~5 min upgrade + ~20 min power cycle)
XRDU	Host must be powered off	Power on, firmware check	~30 min (~20 min upgrade + ~10 min power on)

1. THOR CARD

Pre-condition: Host must be powered on.

# Verify host is powered on
snctl node powerstate --node <node_id> --type host

# Power on host if needed
snctl node poweron --node <node_id> --type host

Update:

snctl device firmware-update --rack-id <rack_id> --type card

Post-upgrade:

snctl node powercycle --node <node_id>
snctl device firmware-check --rack-id <rack_id> --type card

2. BIOS

Pre-condition: Host must be powered on.

# Verify host is powered on
snctl node powerstate --node <node_id> --type host

# Power on host if needed
snctl node poweron --node <node_id> --type host

Update:

snctl device firmware-update --rack-id <rack_id> --type bios

Post-upgrade: Applying the BIOS update and then immediately power cycling the whole node can leave the host powered on but stuck before POST (it never reaches the OS). To apply the new BIOS reliably, cycle the host on its own first, confirm the SR-IOV BIOS setting, and only then power cycle the entire node.

Power off the node (host and XRDUs). Omit --type so both the host and the XRDUs are powered off.

snctl node poweroff --node <node_id>

Power the host back on. The new BIOS is applied during POST (the console shows the BIOS settings being applied, and the host may reboot once on its own) before it boots to the OS.

snctl node poweron --node <node_id> --type host

Note: Wait ~15 minutes after power-on for the host to finish POST and boot to the OS before running the SR-IOV check below.

Ensure SR-IOV is disabled. SR-IOV must be disabled in the BIOS (PCIS007 is the SR-IOV attribute). Work through the checks below over Redfish.

3a. Gather the BMC connection details:
The curl commands below authenticate to the BMC’s Redfish API and need its IP address, username, and password. BMC IP and username come from the node’s inventory. Run snctl node show <node_id> and read them from the BMC row (IP and Username columns):

snctl node show <node_id>

BMC password is stored in the snctl secret store. Print the current value with:

snctl secret get --device-type bmc --show-value

—show-value prints the password in clear text. Only use it in a secure environment and never share the output

If the curl commands below time out or return an authentication error, re-verify the BMC IP, username, and password with snctl node show <node_id> and snctl secret get --device-type bmc --show-value, and confirm the BMC’s management interface is reachable from this host.

3b. Check the current value:

curl -s -k -u <bmc_user>:<bmc_password> \
  https://<bmc_ip>/redfish/v1/Systems/Self/Bios \
  | jq '.Attributes.PCIS007'

#Example Output
"Enabled"

If it reads "Disabled", you are done with this step, continue to step 4.
If it reads "Enabled", continue to step 3c.

3c. Disable SR-IOV (a successful request returns no output):

curl -s -k -u <bmc_user>:<bmc_password> \
  -X POST https://<bmc_ip>/redfish/v1/Systems/Self/Bios/SD \
  -H 'Content-Type: application/json' \
  -d '{"Attributes": {"PCIS007": "Disabled"}}'

3d. Apply the change by rebooting just the host again:

snctl node poweroff --node <node_id> --type host
snctl node poweron  --node <node_id> --type host

Wait ~15 minutes for the host to boot, then repeat step 3a to confirm PCIS007 now reads "Disabled" before continuing. If it still reads "Enabled", do not proceed to step 4 — repeat step 3c once, and if the value still hasn’t changed, stop and contact SambaNova support rather than power cycling with SR-IOV still enabled.

Power cycle the entire node once SR-IOV is confirmed disabled.

#Power Cycle the Node
snctl node powercycle --node <node_id> --type node

#Check the Firmware BIOS version
snctl device firmware-check --rack-id <rack_id> --type bios

3. BMC

Update:

snctl device firmware-update --rack-id <rack_id> --type bmc

Post-upgrade:

snctl node powercycle --node <node_id> --type node
snctl device firmware-check --rack-id <rack_id> --type bmc

4. XRDU

Pre-condition: Host must be powered off before updating XRDU firmware.

# Verify host is powered off
snctl node powerstate --node <node_id> --type host

# Power off host if needed
snctl node poweroff --node <node_id> --type host

Update:

snctl device firmware-update --rack-id <rack_id> --type xrdu

Post-upgrade:

snctl node poweron --node <node_id> --type node
snctl device firmware-check --rack-id <rack_id> --type xrdu

Output columns

The firmware check output includes the following columns:

Column	Description
Rack	Rack identifier
Component	Device type and index
Status	`PASS` if firmware matches expected version, `FAIL` otherwise
CURR	Current firmware version. For card devices, lists each network interface with its version and per-interface `PASS`/`FAIL` result.
POR	Plan of Record (expected) firmware version
OBMC_CURR	Current OpenBMC firmware version (XRDU only)
OBMC_POR	Expected OpenBMC firmware version (XRDU only)
RDUC_CURR	Current RDUC firmware version (XRDU only)
RDUC_POR	Expected RDUC firmware version (XRDU only)

Getting Started

Service Administration

Hardware Administration

Reference Architecture

Resources

Device administration

Commands

Check firmware version

Single rack

Group

Update firmware

Single rack

Group

Order of execution

1. THOR CARD

2. BIOS

3. BMC

4. XRDU

Output columns

​Commands

​Check firmware version

​Single rack

​Group

​Update firmware

​Single rack

​Group

​Order of execution

​1. THOR CARD

​2. BIOS

​3. BMC

​4. XRDU

​Output columns

Commands

Check firmware version

Single rack

Group

Update firmware

Single rack

Group

Order of execution

1. THOR CARD

2. BIOS

3. BMC

4. XRDU

Output columns