Troubleshooting SambaFlow Tutorials

This doc page helps with troubleshooting our tutorials. We expect to add more troubleshooting information over time.

All SambaFlow tutorials require that you have access to the console of a SambaNova DataScale system, usually through ssh.
You cannot run the SambaFlow tutorials in a SambaStudio environment.

Check the SambaNova Daemon status

Before running the example, make sure the SND (SambaNova Daemon) service is running.

  1. Run this command:

    $ systemctl status snd

  2. Look for Active: active (running) in the output.

  3. If you don’t see this line, the service is either not running or degraded. Ask your system administrator for help.

Errors and resolution

Error

You see an error about StatusCode.UNAVAILABLE or another strange error.

Resolution

  1. Check if SND is running. No special privileges are required for this check.

    $ systemctl status snd

  2. If SND is not running, ask your system administrator for help.

    One of the reasons for difficult to understand errors is a Userspace and Kernspace version mismach, that is, after an upgrade, the kernel modules were not reloaded.

  3. Your system admin should try the following commands in sequence:

    $ systemctl stop snd
$ modprobe -r rdu_peer_mem rdu_mem_map rdu_sn20
$ modprobe rdu_sn20
$ modprobe rdu_mem_map
$ modprobe rdu_peer_mem
$ systemctl start snd