SambaFlow developer documentation

SambaFlow™ developer documentation includes release notes, quickstart, tutorials, and the Python API reference.

What’s new in 1.17

Release notes. Learn about new features, deprecated features, and bug fixes.
Tutorials on GitHub. New public repo for SambaNova tutorials , including 2 simple tutorials for new users.

Architecture and workflows. Learn how SambaFlow fits into the SambaNova hardware and software stack, and about the typical compile and run workflow.
Models on RDU hardware. Understand the big picture: requirements and workflow for running on RDU.
Compilation overview. Explore the different layers of the compiler stack and explains what happens at each layer.
Compiler optimization modes. Learn about compiler optimization modes, new in 1.17, that give you control over operator fusion.

SambaFlow learning map. Overview of all tutorials and where to find instructions, tutorial files, and code discussion.
Hello SambaFlow! Learn how to compile and run your first model (duplicate of the README on GitHub) and explore the code discussion to help you create your own models.
Intermediate tutorial. Build on Hello SambaFlow! and learn about data download and running inference in Compilation, training, and inference (duplicate of the README on GitHub). The code discussion is in Examine LeNet model code.
Model conversion 101. Learn what’s required to run a PyTorch model on RDU from the detailed discussion of code examples in Convert existing models to SambaFlow.
Transformers on RDU. Use a pretrained Hugging Face model on RDU. The tutorial discusses data preparation, compile and training run, and running inference in Compile, train, and perform inference with a Hugging Face GPT model and discusses the model code in Code elements of the training program and Code elements of the inference program.
Run language example applications. Learn how to run a simple language model that’s included in the SambaFlow installation.

SambaNova messages and logs. Understand which messages to stdout are useful and where different logs are kept.
Use sntilestat for performance analysis. Learn how to use the sntilestat utility interactively to learn where your model spends time, and how to use the .json and .csv files the tool generates.

SambaNova PyTorch operator support. Reference to supported PyTorch operators. Includes links to the API Reference.
Arguments for compile. Reference to commonly used compiler arguments. Includes descriptions and examples.
Operator fusion rule yaml syntax (Beta). Reference to the operator fusion rule yaml syntax. These yaml files are used in conjunction with o1 compiler mode.
Hyperparameter reference. Reference to supported hyperparameters.

Uncorrectable Error Replay (Beta). Learn how the new UE error replay feature works and how to use it.
Use Python virtual environments. Learn how to use the virtual environments that are included with SambaFlow example applications.
Data parallel applications in SambaFlow. Learn about compiling a data parallel model and best practices for data parallel.
Use LayerNorm instead of BatchNorm. Learn how to convert a model from BatchNorm to LayerNorm.
Control what tokens are back-propagated on during training. Learn how to use token ids to customize which tokens a model learns to generate, and which tokens a model attends to, but does not learn to generate.
Best practices for hardware transition. If you’re migrating to SN30 hardware, the best practices help you understand changes to your models you might want to make.

SambaFlow API Reference .
Data preparation scripts. We have a public GitHub repository with two scripts for pretraining data creation, pipeline.py and data_prep.py.
SambaNova Runtime documentation. Information on logs, fault management, and other lower-level procedures.