> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Device administration

This guide covers checking and updating firmware on devices within SambaRack. Devices are individual components attached to racks and nodes, including BMCs, switches, PDUs, and serial terminals.

## Commands

### Check firmware version

Check firmware versions on devices:

```bash theme={}
snctl device firmware-check \
  (--rack-id <rack_id> | --group <group_name>) \
  --type <device_type> \
  [--index <index>]
```

**Target selection:**

One of `--rack-id` or `--group` is required (mutually exclusive):

| Parameter | Description |
| - | - |
| `--rack-id <rack_id>` | Check devices in a specific rack |
| `--group <group_name>` | Check devices in all racks within a group |

**Device types:**

| Type | Description |
| - | - |
| `access_switch` | Access switch |
| `data_switch` | Data switch |
| `scs` | System Control Switch |
| `bios` | BIOS |
| `bmc` | Baseboard Management Controller |
| `card` | Card/Adapter |
| `pdu` | Power Distribution Unit |
| `xrdu` | XRDU |
| `all` | All device types |

#### Single rack

**Example – check BMC firmware on a single rack:**

```bash theme={}
snctl device firmware-check --rack-id 001 --type bmc
```

**Example output:**

```text theme={}
             Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component ┃ Status ┃ CURR     ┃ POR      ┃
┡━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ BMC       │ PASS   │ 12.61.21 │ 12.61.21 │
└──────────┴───────────┴────────┴──────────┴──────────┘
```

**Example – check all firmware on a rack:**

```bash theme={}
snctl device firmware-check --rack-id 001 --type all
```

**Example output:**

```text theme={}
                                        Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component     ┃ Status ┃ CURR                        ┃ POR         ┃ OBMC_CURR ┃ OBMC_POR ┃ RDUC_CURR ┃ RDUC_POR ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ ACCESS_SWITCH │ FAIL   │                             │             │           │          │           │          │
│ Rack-001 │ BIOS          │ PASS   │ R24_P1                      │ R24_P1      │           │          │           │          │
│ Rack-001 │ BMC           │ PASS   │ 12.61.21                    │ 12.61.21    │           │          │           │          │
│ Rack-001 │ CARD          │ PASS   │ snhni0 -> 228.1.111.0  PASS │ 228.1.111.0 │           │          │           │          │
│          │               │        │ snni0 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni2 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni4 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni6 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni8 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni10 -> 228.1.111.0  PASS │             │           │          │           │          │
│          │               │        │ snni12 -> 228.1.111.0  PASS │             │           │          │           │          │
│          │               │        │ snni14 -> 228.1.111.0  PASS │             │           │          │           │          │
│ Rack-001 │ DATA_SWITCH   │ FAIL   │                             │             │           │          │           │          │
│ Rack-001 │ PDU_PDU_1     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_2     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_3     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_4     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ SCS           │ FAIL   │ 9.4.0.1R1                   │ 9.5.0.0R15  │           │          │           │          │
│ Rack-001 │ XRDU_XRDU_0   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_1   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_2   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_3   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_4   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_5   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_6   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_7   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
└──────────┴───────────────┴────────┴─────────────────────────────┴─────────────┴───────────┴──────────┴───────────┴──────────┘
```

#### Group

**Example – check BMC firmware across all racks in a group:**

```bash theme={}
snctl device firmware-check --group g1 --type bmc
```

**Example output** (one row per rack):

```text theme={}
             Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component ┃ Status ┃ CURR     ┃ POR      ┃
┡━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ BMC       │ PASS   │ 12.61.21 │ 12.61.21 │
│ Rack-002 │ BMC       │ FAIL   │ 12.85.05 │ 12.61.21 │
└──────────┴───────────┴────────┴──────────┴──────────┘
```

> [!NOTE]
>
> If devices are unreachable or configured incorrectly, the firmware check fails for those devices with a `FAIL` status, as shown in the example output above. For card devices, the `CURR` column lists each network interface with its firmware version and a per-interface `PASS`/`FAIL` comparison. The overall `CARD` status is `PASS` only when every interface matches the POR version.

### Update firmware

Update firmware on devices within a rack or group:

```bash theme={}
snctl device firmware-update \
  (--rack-id <rack_id> | --group <group_name>) \
  --type <device_type> \
  [--index <index_or_list>] \
  [--file <path> ...] \
  [--force]
```

**Target selection:**

One of `--rack-id` or `--group` is required (mutually exclusive):

| Parameter | Description |
| - | - |
| `--rack-id <rack_id>` | Update devices in a specific rack |
| `--group <group_name>` | Update devices in all racks within a group |

**Device types:**

| Type | Description |
| - | - |
| `bios` | BIOS |
| `bmc` | Baseboard Management Controller |
| `card` | Card/Adapter |
| `xrdu` | XRDU |

> [!NOTE]
>
> Other device types (`access_switch`, `data_switch`, `pdu`, `scs`) are not currently supported for firmware update and will be rejected by the CLI.

**Optional parameters:**

| Parameter | Description |
| - | - |
| `--index <index_or_list>` | Device index to update. For XRDU: omit to target all XRDUs, pass a single index (0–7), or a comma-separated list (e.g. `0,2,5`). Multiple indices run in parallel within the rack. |
| `--file <path>` | Local firmware file. For XRDU, pass one combined archive whose name starts with `xrdu` (e.g. `xrdu-combined-5.1.4.tgz`), or two split archives whose names start with `obmc` and `rduc` respectively. Repeat the flag for two files: `--file obmc-5.1.4.tgz --file rduc-5.1.4.tgz`. If omitted, the firmware is downloaded from the configured artifact store. |
| `--force` | Skip checksum validation |

#### Single rack

**Example – update BMC firmware:**

```bash theme={}
snctl device firmware-update --rack-id 001 --type bmc
```

**Example – update all XRDUs:**

```bash theme={}
snctl device firmware-update --rack-id 001 --type xrdu
```

**Example – update specific XRDUs:**

```bash theme={}
snctl device firmware-update --rack-id 001 --type xrdu --index 0,2,5
```

**Example – update XRDU firmware from local files:**

```bash theme={}
snctl device firmware-update --rack-id 001 --type xrdu --index 3 \
  --file /path/to/obmc-5.1.4.tgz --file /path/to/rduc-5.1.4.tgz --force
```

#### Group

**Example – update firmware across all racks in a group:**

```bash theme={}
snctl device firmware-update --group g1 --type bmc
```

When you use `--group`, `snctl` processes racks sequentially (one rack at a time) while running individual device updates in parallel within each rack. This bounds concurrency – for example, a group with 10 racks and 8 XRDUs each won't launch 80 simultaneous jobs.

During the update, `snctl` prints:

* Per-component `PASS`/`FAIL` lines as each job completes
* Per-rack checkpoints after each rack finishes, including `PASS`/`FAIL` counts and log file paths for any failures
* A final **Firmware Update Results** table summarizing the entire run

If one or more devices fail, the command exits non-zero and prints the stdout, stderr, and instruction output log paths for each failed job.

## Order of execution

When updating multiple components, there is no strict dependency between them. However, if upgrading more than one type in the same maintenance window, follow this order:

**THOR CARD → BIOS → BMC → XRDU**

| Device | Pre-condition | Post-upgrade operations | Expected duration |
| - | - | - | - |
| THOR CARD | Host must be powered on | Power cycle, firmware check | \~25 min (\~5 min upgrade + \~20 min power cycle) |
| BIOS | Host must be powered on | Power cycle, firmware check | \~25 min (\~5 min upgrade + \~20 min power cycle) |
| BMC | None | Power cycle, firmware check | \~25 min (\~5 min upgrade + \~20 min power cycle) |
| XRDU | Host must be powered off | Power on, firmware check | \~30 min (\~20 min upgrade + \~10 min power on) |

### 1. THOR CARD

**Pre-condition:** Host must be powered on.

```bash theme={}
# Verify host is powered on
snctl node powerstate --node <node_id> --type host

# Power on host if needed
snctl node poweron --node <node_id> --type host
```

**Update:**

```bash theme={}
snctl device firmware-update --rack-id <rack_id> --type card
```

**Post-upgrade:**

```bash theme={}
snctl node powercycle --node <node_id>
snctl device firmware-check --rack-id <rack_id> --type card
```

### 2. BIOS

**Pre-condition:** Host must be powered on.

```bash theme={}
# Verify host is powered on
snctl node powerstate --node <node_id> --type host

# Power on host if needed
snctl node poweron --node <node_id> --type host
```

**Update:**

```bash theme={}
snctl device firmware-update --rack-id <rack_id> --type bios
```

**Post-upgrade:**

```bash theme={}
snctl node powercycle --node <node_id> --type node
snctl device firmware-check --rack-id <rack_id> --type bios
```

### 3. BMC

**Update:**

```bash theme={}
snctl device firmware-update --rack-id <rack_id> --type bmc
```

**Post-upgrade:**

```bash theme={}
snctl node powercycle --node <node_id> --type node
snctl device firmware-check --rack-id <rack_id> --type bmc
```

### 4. XRDU

**Pre-condition:** Host must be powered off before updating XRDU firmware.

```bash theme={}
# Verify host is powered off
snctl node powerstate --node <node_id> --type host

# Power off host if needed
snctl node poweroff --node <node_id> --type host
```

**Update:**

```bash theme={}
snctl device firmware-update --rack-id <rack_id> --type xrdu
```

**Post-upgrade:**

```bash theme={}
snctl node poweron --node <node_id> --type node
snctl device firmware-check --rack-id <rack_id> --type xrdu
```

## Output columns

The firmware check output includes the following columns:

| Column | Description |
| - | - |
| Rack | Rack identifier |
| Component | Device type and index |
| Status | `PASS` if firmware matches expected version, `FAIL` otherwise |
| CURR | Current firmware version. For card devices, lists each network interface with its version and per-interface `PASS`/`FAIL` result. |
| POR | Plan of Record (expected) firmware version |
| OBMC\_CURR | Current OpenBMC firmware version (XRDU only) |
| OBMC\_POR | Expected OpenBMC firmware version (XRDU only) |
| RDUC\_CURR | Current RDUC firmware version (XRDU only) |
| RDUC\_POR | Expected RDUC firmware version (XRDU only) |