Baseboard management controller (BMC) administration

When security patches are available or when BMC firmware updates are required for other reasons, you can perform the tasks in this section. Updating the BIOS is included with this BMC administration topic because the two tasks are usually performed at the same time. The tasks include:

  • Updating the DataScale SN10-H host module BMC firmware

  • Updating the DataScale SN10-H host module BIOS

  • Recovering the DataScale SN10-H BMC

1. General notes and warnings

Do not power off or reboot the DataScale SN10-8R components during firmware updates. Interrupting a firmware update may damage the DataScale SN10-8R components, and the damaged component might not be recoverable. Perform a shutdown or reboot only after a firmware update has been completed successfully.
Settings on the BMCs do not need modification and remain static unless you are updating the BMCs, collecting diagnostic material, or changing the login credentials. Do not make configuration changes to the BMC unless you are otherwise instructed.
Do not remove the admin user account or change this account’s password. This account is needed for password recovery of the DataScale SN10-H host module’s BMC.

2. Updating the DataScale SN10-H host module BMC firmware

If you start the update firmware process and you decide to cancel the process, you need to reset the BMC. To do that, close the web browser that was logged in to the BMC Web UI, and then log in to the BMC Web UI again before you attempt any administrative operations for the BMC.

Before you update the firmware, back up the existing configuration of the DataScale SN10-H host module. In some cases, having a backup helps with recovering the BMC.

2.1. Back up the existing configuration

Before you update the firmware, back up the existing configuration of the DataScale SN10-H host module. Having a backup might help with recovering the BMC.

To back up the existing configuration, your system must meet the following requirements:

  • Access to the DataScale SN10-H host module’s BMC via the access network

  • One of the following supported web browsers:

    • Chrome (latest version)

    • Firefox (latest version)

Follow these steps to back up the existing configuration:

  1. Open a web browser.

  2. In the browser’s address bar, enter the IP address of the DataScale SN10-H host module’s BMC, and log in to the management console with your user credentials, and click Sign me in.

    Login screen

  3. In the dashboard, select Maintenance.

    Dashboard

  4. On the Maintenance screen, select Backup Configuration.

    Maintenance screen

  5. On the Backup Configuration screen, select Check All to back up all the BMC configuration details.

    Backup Configuration screen

  6. Click Download to save this configuration to the local system (which is accessing the BMC WebUI).

  7. Click OK to download the bmc-config.bak backup configuration file. You can use that file later if a restore is required.

    Download screen

2.2. Updating the host module BMC host module firmware

Now that you have backed up the BMC configuration, you can update the SN10-H host module’s BMC firmware while preserving the configuration. Follow these steps:

  1. Download the DataScale SN10-H host module’s BMC patch update from the SambaNova Support portal to the local system that is accessing the BMC Web UI.

  2. Unzip the SambaNova patch update to a directory on the local system.

  3. From the Backup Configuration screen, select Maintenance from the left pane.

    Backup Configuration screen

  4. From the Maintenance screen, select Preserve Configuration.

    Maintenance screen

  5. Select Check All at the top of the list to preserve the configuration of everything.

    Preserve Configuration screen

  6. The following message appears if the configuration preservation was successful.

    Success message

After configuration preservation is successful, follow these steps to update the firmware:

  1. In the left pane, click Maintenance.

    Left pane

  2. In the Maintenance screen, select Firmware Update.

    Maintenance screen

  3. In the Firmware Update screen, click Browse.

    Firmware Update screen

  4. Navigate to the .bin file that you downloaded and unzipped erlier. This file is in the /SN10-8R/<version>/HostBMC_FW/ directory of the unzipped patch bundle. Select the rom.ima_enc file and click Open.

    rom.ima_enc file

  5. Back in the Firmware Update screen, click Start firmware update.

    Firmware Update screen

  6. The screen is expanded below the button that you just clicked. Select the Preserve all Configuration checkbox to use the preserved configuration you saved earlier.

    Preserve all Configuration

  7. Scroll to the bottom of the screen and click Proceed to Flash.

    Proceed to Flash

  8. Click OK in the BMC update confirmation screen.

    BMC update confirmation screen

    After the BMC update process has started, the BMC is not reachable for 5 to 10 minutes while the update is being applied. The DataScale SN10-H host module OS continues to run normally during the BMC update.

    After 10 minutes, log in to the BMC Web UI and confirm that the update was successful by checking the information in the upper left side of the dashboard. The BMC firmware version is identified as <XX.XX.X>.

    BMC firmware version

2.3. Updating the DataScale SN10-H host module BIOS

Ensure that the update process is not interrupted. After you enter update mode, the widgets and other web pages and services will not work. All the open widgets are closed automatically. If you cancel the upgrade in the middle of the process, the SN10-H host module will be reset only for the BMC BOOT and APP components of the firmware.
The SN10-H host module BIOS update requires a reboot of the system to apply the updated BIOS. Please plan accordingly.

To update the SN10-H host module BIOS, your system must meet the following requirements:

  • Access to the DataScale SN10-H host module’s BMC via the access network

  • One of the following supported web browsers:

    • Chrome (latest version)

    • Firefox (latest version)

You will first perform the BIOS update, and then update the OS. Follow these steps:

  1. Open a web browser.

  2. In the browser’s address bar, enter the IP address of the DataScale SN10-H host module’s BMC, enter your user credentials, and click Sign me in.

    Login screen

  3. In the dashboard, select Maintenance.

    Dashboard

  4. In the Maintenance screen, select Firmware Update.

    Maintenance screen

  5. In the Firmware Update screen, click Browse.

    Firmware Update screen

  6. Navigate to the image.RBU file that you downloaded and uncompressed earlier, select the file, and click Open. The file is located in the /Host_BIOS/RBU/ directory of the uncompressed infrastructure patch bundle.

    image.RBU file

  7. Back in the Firmware Update screen, click Start firmware update.

    Firmware Update screen

  8. The screen is expanded below the button you clicked. From the Update Type drop-down, select BIOS.

    Update Type drop-down list

    The Update Type drop-down shows that BIOS is selected.

    BIOS selected

  9. Click Proceed to Flash.

    Proceed to Flash button

  10. Click OK in the firmware upgrade confirmation screen.

    Firmware upgrade confirmation screen

    This initiates uploading the BIOS firmware update to the DataSale SN10-H host module, but it does not automatically apply the firmware update.

    Update initiated

  11. When the screen shows Uploading 100%, click the Flash BIOS button.

    Flash BIOS button

    This initiates the BIOS update process.

    Flash process screen

  12. When the flash process is complete, a firmware image has been updated successfully message appears. Click OK to continue.

    Success message

  13. Another message appears stating that the firmware reset has been called. Click OK to log out of the SN10-H BMC Web UI.

    Firmware reset message

2.4. Resetting the SN10-H OS

As a final step, you reset the SN10-H OS to complete the BIOS update.

  1. After you are logged out of the SN10-H BMC, log in to the SN10-H OS.

    $ ssh snuser1@<SN10-H_OS_IP_Address>
snuser1@<SN10-H_OS_IP_Address>’s password: <snuser1 Password>

  2. Reset the SN10-H OS to complete the BIOS update.

    $ sudo shutdown -r now
[sudo] password for snuser1: <snuser1 Password>

  3. After the SN10-H host module is back online, log in to the SN10-H BMC and confirm that the BIOS update has been applied. To do this:

    1. Select Maintenance from the left pane of the dashboard.

      Dashboard screen

    2. From the Maintenance screen, select Firmware Information.

      Maintenance screen

    3. On the BMCFirmware Information screen, check the firmware version under BIOSFirmware Information.

      BMCFirmware Information screen

3. Recovering the DataScale SN10-H BMC

If the DataScale SN10-H host module’s BMC is no longer responding or no longer accessible, or the DataScale SN10-H host module’s BMC password has been lost or forgotten, contact Customer Support.

4. Viewing SN10-H BMC diagnostic information and logs

You can quickly identify a system’s status and view diagnostic information and logs for the DataScale SN10-H BMC from the Web UI.

  1. Log in to the BMC’s Web UI and view the BMC dashboard.

  2. Click the More info link in each box for details on logs and pending events/deassertions.

    Diagnostic information

  3. Click Logs & Reports in the left pane and select a sub-item to find more logs and reports.

    Logs & Reports item

See KB article #1039, “Diagnostic Data Collection Tool(samba_diag),” in the SambaNova Support portal (https://support.sambanova.ai) for details on:

  • Diagnosing a DataScale SN10-H host module’s BMC specifically.

  • Diagnosing the DataScale SN10-H host module.

  • Collecting the required diagnostic and log material.