Compare performance metrics

You can use SambaTune to compare runs with performance data. Comparing different runs might make sense if you’ve run the same model with different hyperparameters.

Prerequisites

  • Ensure that SambaTune is installed in your environment.

  • Install the SambaTune GUI client.

  • Run sambatune, passing in your custome YAML file for the model you want to tune. See Run SambaTune.

  • Run sambatune_ui and note down the URL, username and password that you see at the end of the run. A random password is generated for each run. See Run the SambaTune GUI.

Procedure

  1. From your web browser, connect to the SambaTune client.

  2. In the Dashboard, either use the selected directory or specify a different directory.

  3. Select two of the subdirectories by clicking the checkboxes and click Compare.

    Compare feature dashboard page usage

  4. With Configuration selected, click on Expand 5 lines for details about the 2 runs.

    Compare Configurations on INFO page

  5. In the top left, click Analytics

  6. First, explore analytics for the host.

    1. In the top right, select Hardware > Host to see the host comparison for the two runs.

    2. Scroll for more details or to export the result.

    3. Click Tensor to see tensor information.

  7. Click Overview and select Hardware > RDU to explore RDU analytics.

    1. Click Section Report for a detailed comparison of section-level information

      Compare feature section report

    2. From the Section menu in the top right, select a specific section to see the stage latency comparison between 2 runs.