SambaFlow Software Release Notes

Release 1.16 (2023-07-14)

New features and other improvements

  • Introduced new compiler modes -o0 and -o1 (Beta), which allow users to fine-tune compiler performance.

  • Change to compiler --help behavior. The --help command now returns a limited number of fully supported options. A call to compile with --help --debug returns a longer list of options, some of them experimental.

Performance improvements

  • Various optimizations in this release help improve model performance and reduce compile times especially for NLP models.

Supported versions

  • PyTorch: 1.10.2+cpu

  • Python: 3.7

Documentation improvements

  • Updated API Reference includes documentation for supported PyTorch operators

    API Reference documentation always opens in a new tab (or window). To return to the main doc set, click the previous tab (or window).

  • New SambaNova messages and logs doc page explains which messages you can safely ignore, where to find which logging information, and which errors you might be able to resolve yourself.

  • New SambaNova compilation overview doc page gives an overview of the compiler stack and discusses some compiler arguments, including the new o0, o1, etc. options.

  • New Arguments for compile doc page is a reference to frequently used compiler arguments and includes a discussion of the new arguments.

  • New SambaNova PyTorch operator support doc page lists which PyTorch options are fully supported and experimentally supported. This page will be updated with each release and includes links to the API Reference.

  • New Use sntilestat for performance analysis doc page explains how to use the sntilestat tool for performance analysis and includes examples of visualizing sntilestat CSV output in a spreadsheet.

Obsolete components and APIs

The grad_of_outputs parameter in samba.session.run, was deprecated in release 1.15 and has been removed. Use SambaTensor::sn_grad to set an output tensor’s gradients instead.

Release 1.15 (2023-03-30)

Deprecated components and APIs

The grad_of_outputs parameter in samba.session.run is deprecated and will be removed in release 1.16. Use SambaTensor::sn_grad to set an output tensor’s gradients instead.

Release 1.14 (2023-01-10)

Deprecated components and APIs

The following APIs have been renamed. The old names are deprecated.

  • Renamed samba.from_torch to samba.from_torch_tensor

  • Renamed samba.from_torch_ to samba.from_torch_model_

Release 1.13 (2022-11-03)

New features and other improvements

  • New features

    • Added option to sntilestat to skip idle tiles.

    • Enhanced multi-processing support for SambaNova Runtime APIs.

    • Enhanced host profiling information and detailed timeline view in SambaTune.

    • Enhanced snprof and added more robust fault reporting in snstat.

  • Performance improvements

    • Faster SambaFlow context creation.

    • More efficient CPU usage.

    • Better performance for scaleout operations.

  • Software

    • Updated PEF to version 2.5.0.

      Recompile all models with this release due to the PEF version change.

    • Version 2 of SambaFlow compiler scheduler, specified with option --mac-v2, is now the default. The --mac-v1 is still supported but requires using explicit option.

Deprecated components

  • venv: The venv shared generic package is deprecated and has been replaced by model-specific venv packages. The generic package will be removed from future releases.

  • UnoSecInf: The UnoSecInf inference performance test, which is based on section-by-section mapping, is deprecated starting in Release 1.13. Starting in Release 1.14, this performance test will no longer be available.

    The uno_full.py model is not deprecated.

Release 1.12.7 (2022-07-30)

New features

  • Added SambaTune: a tool that supports profiling application performance.

  • Improved Scale-out performance through parallel reduce.

  • Enhanced RDU reset support with VM.

Supported components and versions

Operating Systems

  • Red Hat Enterprise Linux 8.5

  • Ubuntu Linux 20.04 LTS

Software

  • Updated PEF to version 2.0.0. Models must be recompiled to be used with this release due to the PEF version change.

  • Version 2 of SambaFlow compiler scheduler, specified with option --mac-v2, is now the default. The --mac-v1 will continue to be supported but requires using explicit option.

Deprecated components

  • The global virtual environment under /opt/sambaflow/venv is deprecated and will be removed in version 1.13. It will be replaced by individual virtual environments for each model.