SN40L-16
SN40L-16 provides improved performance by increasing the number of Reconfigurable Dataflow Units™ (RDUs) available per node from 8 to 16.
Nodes and RDUs
A node describes the collection of SambaNova Systems' Reconfigurable Dataflow Units (RDUs) available. For example, with SambaNova’s SN10, SN30, and SN40L-8, a total of eight RDUs are available per node. SN40L-16 increases the available RDUs to 16 per node.
|
If a job or endpoint requires more than one RDU to start, the RDUs must be available on the same node. With 16 RDUs per node, SN40L-16 increases the availability of RDUs to start heavy RDU resource jobs or endpoints. |
Naming convention
-
Our existing SN40L, which includes eight RDUs per node, will be renamed to SN40L-8.
-
Our 16 RDUs per node is named SN40L-16.
|
The SambaStudio interface will include SN40L-16 in the following locations: |
Supported models
The table below describes the SambaNova provide CoEs and their expert models that are supported on SN40L-16. Search the CoE name or expert model name in the Model Hub to view SN40L-16 information in the corresponding model cards.
|
Batch inference is not supported for the expert models listed in the table. |
| CoE Name | Expert Model Name | Dynamic Batch Size Supported | Fine-tuned Checkpoints Inference Supported | Add Checkpoint From Storage Inference Supported |
|---|---|---|---|---|
Llama 3.1 405B with E5-mistral |
Meta-Llama-3.1-405B-Instruct e5-mistral-7b-instruct |
1, 4 1, 4, 8, 16, 32 |
No Yes |
No Yes |
Llama 3.1 70B with E5-mistral |
Meta-Llama-3.1-70B-Instruct e5-mistral-7b-instruct |
1, 4, 8, 16, 32 1, 4, 8, 16, 32 |
No Yes |
No Yes |
Llama 3.1 8B with E5-mistral |
Meta-Llama-3.1-8B-Instruct e5-mistral-7b-instruct |
1, 4, 8, 16, 32 1, 4, 8, 16, 32 |
Yes Yes |
Yes Yes |