SambaStudio release notes

Release 24.2.2 (2024-04-01)

SambaStudio release 24.2.2 is a patch update for 24.2.1 that includes the enhancements described below.

Enhancements

Release 24.2.1 (2024-03-19)

New features

  • With this release, SN40L External link users can utilize SambaStudio’s new Composition of Experts (CoE) architecture. The CoE architecture is a system of multiple experts, where each expert is a fully trained machine learning model.

  • SambaNova’s first Composition of Experts model, Samba-1, is available for SN40L users. Samba-1 combines the comprehensive power of trillion-parameter models with the precision and efficiency of specialized models accessed via a single endpoint.

  • Added new CoE specific model cards to the Model Hub. CoE model cards allow you to view the expert models used to create the CoE model.

  • SN40L users can create CoE endpoints to be deployed for use in the Playground.

  • Added the ability to select an expert for CoE endpoints in the Playground. This allows you to choose an expert model for your particular prompt.

Release 24.1.1 (2024-02-08)

New features

Known issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

  • When an organization administrator deletes a tenant, sometimes the GUI will not update that the tenant is deleted. As a workaround, please refresh the browser if the GUI update takes longer than 30 seconds.

  • We’ve noticed an issue where the CLI is parsing hyperparameter boolean values incorrectly when using snapi job create with the --hyperparams-file option. As a workaround, please use quotes for hyperparameter boolean values, as shown in the example hyperparameters file below.

    Example hyperparameters file
    max_seq_length : 2048
    precision: "true"
  • Cancelling the upload of a dataset can sometimes result in the cancelled dataset still displaying a status of Uploading in the Dataset Hub. The cancelled dataset may take up to 20 minutes for its status to change to Failed, at which point the dataset can be deleted.

Release 23.11.1 (2023-11-17)

New features

  • Added new endpoint API key features to both GUI and CLI endpoint workflows.

    • Added the ability to create multiple API keys for an endpoint.

    • Added the ability to revoke an endpoint API key to prevent further usage.

    • These features allow organizations to easily distribute and manage endpoints to different entities and users.

  • Added improved command options and performance for adding datasets using the CLI.

    • Previous versions of snapi dataset add commands and procedures are not compatible with release 23.11.1.

    • Adding a dataset using CLI commands results in faster uploads compared to using the GUI.

    • Using CLI commands to add datasets is recommended for datasets that are greater than 5GB or contain more than 1000 files.

  • Updated the installation requirements for the SambaNova SDK (snsdk) and SambaNova API (snapi) to Python 3.9.

Release 23.9.2 (2023-11-03)

New features

  • Organization administrators can now delete tenants using both the GUI and CLI workflows. Please be aware of the following considerations when deleting a tenant:

    • The default tenant cannot be deleted.

    • All jobs, endpoints, datasets, and models created in the tenant will be deleted.

    • Associated users will no longer have access to the tenant.

    • RDUs allocated to the tenant will be added to the default tenant.

    • Tenant deletion is irreversible.

  • You can now create a new training job directly from a checkpoint. This enables faster development and experimentation for your workflows.

  • Added insufficient storage alerts to the platform for storage related actions. You will still be able to proceed with your action, but you will need to clear storage space; otherwise, the action will eventually fail.

Bug fixes

  • Fixed the issue where if an endpoint did not successfully deploy and its status was Failed, the acquired Reconfigurable Dataflow Units™ (RDUs) were sometimes not released (described in the Known issues of Release 23.8.1).

  • Fixed an issue where a training job could be started using a model that was still in the state of Downloading. Now models will complete the downloading process before being available for training jobs.

Release 23.8.3 (2023-10-06)

SambaStudio release 23.8.3 is a patch update for 23.8.1 that includes the enhancements described below.

Enhancements

  • Improved general performance of the platform.

  • Improved the performance of the Metrics graph display used for evaluating a training job. The graph now downsamples job metrics that contain over 5K of data points, while still providing an accurate representation of the training job.

Release 23.8.1 (2023-09-29)

New features

  • Added new features and enhancements to the Playground.

    • The Playground now includes a chat mode experience. Chat mode provides a word-by-word response to your prompt with follow-on prompts kept within the context of your conversation. This allows the Playground to understand your prompts without the need to restate preceding information.

    • Improved and updated the Add from presets drop-down menu and input field of the Playground editor for both chat and completion mode.

    • Renamed the tuning parameter Return logits to Top logprobs to align with the updated industry standard.

      • Improved interacting with Top logprobs to only require hovering over the highlighted token to display the generated list of tokens and their probabilities.

  • Hyperparameters now include corresponding default values and constraints based on the model selected when creating a training job.

Bug fixes

  • Fixed an issue where clicking Resume from a stopped training job’s detail page would sometimes result in the job failing to resume (described in the Known issues of Release 23.7.1).

  • Fixed an issue where uploading a dataset from a local machine would sometimes result in an error (described in the Known issues of Release 23.7.1).

Known issues

If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • The Top logprobs tuning parameter currently does not function in chat mode of the Playground.

  • Hovering over a response in the Playground editor to view Top logprobs can sometimes cause the editor display to fluctuate.

  • If an endpoint does not successfully deploy and its status is Failed, the acquired Reconfigurable Dataflow Units™ (RDUs) are sometimes not released. As a workaround, please delete the failed endpoint to release the RDUs.

Release 23.7.1 (2023-08-11)

New features

  • Improved user interface that enables access to global features and information from anywhere within the platform.

  • New controlled sharing implemented with tenants and organization.

    • Provides users with flexible sharing options for datasets, models, and endpoints.

    • Organization datasets and models are now called Shared datasets and Shared models respectively.

  • The Dashboard now provides access to live endpoints in a tenant across all projects.

  • Datasets can now be deleted that are not being used in running jobs.

Bug fixes

  • Fixed an issue where the UI may not automatically refresh when creating or deleting projects or jobs.

  • Fixed an issue in the Playground where the Stop sequences side panel displayed an inaccurate value.

  • Fixed an issue where a model selected from the Select model drop-down may not remain selected while creating a training job.

Known issues

If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

Release 23.5.1 (2023-06-30)

New features

  • Added new features and enhancements to the Playground.

    • Added new Tuning parameters, including Stop sequences, Return logits, Repetition penalty, and Top k.

    • Implemented tooltips for Tuning parameters and detailed information cards for each parameter.

    • The Playground now displays a token count for each submission. The display shows the total number of tokens for the current submission relative to the maximum number of tokens supported by the model (max sequence length).

    • Improved the visibility of the generated responses for each submission. Responses are displayed in the input/output field with highlighted blue text.

    • Improved the notification error if max sequence length is exceeded.

    • Updated and improved User presets, which include examples for each model and provide better understanding on how to best use models.

  • Redesigned and improved the Model Hub

    • Updated design to include a more intuitive two-panel layout featuring a new model card design and filtering panel.

    • Improved the filtering and search functions to easily refine the display of corresponding models.

    • All available SambaNova provided models can easily be displayed by selecting the Available tab.

    • By default, new SambaNova models will not be automatically stored in an organization’s environment. This will allow organizations to save storage space.

    • Organization administrators (OrgAdmin) can download SambaNova models to the Model Hub that they wish to use for their organization.

    • Upcoming SambaNova models are visible to all users within the platform.

    • All users can view the model card of SambaNova Available models and contact their organization administrator (OrgAdmin) to download a model to the Model Hub. Downloaded models are available for use by the entire organization.

  • Improved the Train jobs and Create and use endpoints workflows by removing the concept of a Task.

    • You no longer need to specify a Task during job or endpoint creation. Instead, you select a model to use directly during the workflow or use the new ML App field to filter the list of models to use.

  • Improvements to Datasets.

    • Users can select multiple ML Apps while adding a dataset. This will help attach the same dataset with models under different ML Apps, for example Generative Tuning 1.5 and 13B models.

    • Improved dataset upload speed that will increase the upload speed of larger size datasets.

  • Improvements to Tenants and management.

    • Significant number of refresh improvements to the Tenant management window, resulting in the improvement of tenant and organization level RDUs information displayed in the GUI.

    • Simplified tenant creation process by removing Display Name. Only the tenant name is required. Requirements for tenant name are displayed in the GUI during creation.

  • Improved saving a checkpoint to the Model Hub by allowing users to create a model card during the process. All dataset and parameter information will automatically be taken from the respective job and included in the new model card.

  • Improved logs for batch inference jobs as well as data parallel train jobs.

Bug fixes

  • Fixed issue where jos failed due to dataset names that included spaces. Datasets with spaces in names will no longer be allowed.

  • Fixed issue where hyperparameters that were changed invalidated the initial model, for example num_label for ASR. Now those parameters will be grayed-out and not modifiable.

  • Fixed issue where in the case of certain recoverable errors, failed training jobs should get resumed automatically from the last good checkpoint.

  • Fixed issue where a job progress bar might not automatically refresh, requiring a manual refresh of the browser. Job progress bars will now automatically get refreshed during status updates.

  • Full loss plot will now be available for the entire run of a job.

  • Fixed issue where View code in the Playground could create an incorrect curl command.

Known issues

If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • In the input field on the Playground, an input with an exclamation point (!) that does not have a trailing space after it, can cause curl or snapi commands to fail when using the View code feature.

  • In the Playground, setting Return logits to more than 0 can cause invalid characters to display in the response.

  • In the Playground, when using an endpoint created from GPT_13B_Generic_Instruction_Tuning_v2, selecting the following User Presets can cause their corresponding issue:

    • Selecting Extractive QA (Natural Questions) from the User preset drop-down, can cause the response code copied (Copy code) from the CLI option of View code, to fail when the response code is inputted into the terminal.

    • Selecting Sentiment Analysis (IMDB) from the User preset drop-down, can cause the response code copied (Copy code) from the Python SDK option of View code, to fail when the response code is inputted into the terminal.

  • Loss values in the Checkpoints table (displayed in the Val_loss and Train_loss columns) of a training job can be displayed incorrectly when values are less or equal to 0.01. As a workaround, the loss values displayed in the Logs section for the training job are correct.

Release 23.4.2 (2023-06-12)

SambaStudio release 23.4.2 is a patch update for 23.4.1 that addresses the following issues.

Bug fixes

  • Fixed issue where special characters such as period (.), dash (-), and underscore (_) were unsupported in usernames. Special characters are now supported in usernames and will not result in an error message when adding users.

  • Fixed issue where the Diarization ASR Pipeline v2 model, Diarization_ASR_Pipeline_V2, would fail due to a checkpoint not being available.

  • Fixed issue where a model training job, that referenced an AWS dataset, could get stuck in a status of Retrying.

  • Fixed issue where data parallel is enabled in the SambaNova ViT model, Vit_B_Classification, even though it is not supported.

Release 23.4.1 (2023-05-25)

New features

  • Added the ability to create and manage multiple tenants within an organization.

    • A tenant is an allotment of users and resources within an organization. Organizations can leverage tenants for distinct purposes, such as development or production, and then share common resources across tenants. Each tenant has its own resources, users, and workloads. SambaStudio allows organizations to allocate resources across multiple tenants.

  • Added predefined user roles that can be assigned across multiple tenants.

    • A user is part of an organization and can be assigned a predefined role: organization administrator, tenant administrator, or user. A user can be part of and assigned to multiple tenants. All users within the same tenant can view or manage projects and workloads present in the tenant. SambaStudio allows organizations to assign users across multiple tenants via the organization administrator or tenant administrator role.

  • Improved search filters in the Model hub.

    • The Model hub contains new search filters to narrow search results of model cards. The filters can be displayed by clicking the arrow > next to the Search field in the model hub.

  • Added the ability to view and copy code blocks in the Playground.

    • The Playground now provides the ability to view and copy CURL, CLI, or Python SDK code blocks from the GUI.

  • Added new SambaNova API (snapi) CLI commands.

    • New CLI commands for tenant management including creating, updating, and renaming tenants.

    • New CLI commands for user management including adding users, assigning roles, and deleting users.

Bug fixes

  • Fixed an issue when uploading a dataset sometimes a validation error would display.

  • Fixed minor GUI issues.

Known issues

If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Email username, character limit, and special characters issue.

    • Multiple email domains having the same username (account name) will create a conflict in SambaStudio. For example, a user with the email ab@c.com and a user with the email ab@d.com will create a conflict and be treated the same in the platform.

    • The number of characters for a username is limited to 36. If an admin tries to add a username that exceeds the character limit, the platform will not allow the operation and will produce an error.

    • Special characters such as period (.), dash (-), and underscore (_) are unsupported in usernames. For example, a user with the email jane.doe@domain.com would have a username jane.doe and is not supported.

  • Base models do not support inference and cannot be deployed for endpoints. It is recommended to use Base models for training and not inference.

  • When an Organization administrator (OrgAdmin) creates a new tenant using the GUI, it cannot be deleted. As a workaround, we suggest finalizing tenant names before creating new tenants in the platform.

  • Tenant and resource information can take several seconds to update. This issue can affect both the User management GUI and the CLI command snapi tenant list.

  • Removed the Return logits tuning parameter from the Playground.

Release 23.3.1 (2023-04-17)

New features

  • Added model cards to the Model hub.

    • The Model hub now provides model information details via model cards. The updated GUI includes a two-panel interface for viewing the SambaStudio model cards. The side-by-side panels display the model card list in the left panel and the content of the selected model card in the right panel.

  • Added a Notifications section.

    • The new Notifications panel displays platform specific messages in a scrollable list. Each notification includes a detailed heading, a tracking id number, and a creation timestamp. Notifications are further defined by type: Success, Info, Warnings, Errors, and Critical.

  • Improved information presented in the Projects table.

    • Added a column displaying the number of checkpoints associated with a project. Projects now display the number of associated jobs and endpoints.

    • Added a column displaying the status of the project. Provides visual feedback on project statuses. For example, Deleting will display when the project is in the process of being deleted from the platform.

  • Added training job logs in GUI.

    • Training jobs now include a Logs section that allows you to preview and download logs of your training session in the GUI. Logs can help you track progress, identify errors, and determine the cause of potential errors.

  • Added the ability to edit an endpoint’s model.

    • Editing endpoints in the GUI now allows users to choose a different model. This allows you to change the endpoint’s model negating the need to create a new endpoint for a different model.

  • Added features to the Playground.

    • The Playground now allows you to download the results of the last response provided.

    • Added a Return logits tuning parameter, which adjusts how many log probabilities to return along with predictions.

Bug fixes

  • Fixed notifications not being sent for serve jobs.

  • Fixed upper limit value of Max tokens to generate tuning parameter in the Playground.

Known issues

The Return logits tuning parameter in the Playground can cause submission errors. As a workaround, do not change or set the value.

Release 23.2.1 (2023-03-17)

New features

  • Update the platform name.

    • Dataflow-as-a-Service is now SambaStudio! The new name aligns with SambaNova’s commitment to bring industry-leading AI performance to applications and services.

  • Added the ability to import models to the Model Hub.

    • SambaStudio allows users to add a model-checkpoint from a training job.

    • SambaStudio allows users to take a model from a different environment and add it as a new model to the current SambaStudio environment through NFS.

  • Added the ability to export models from the Model Hub.

    • SambaStudio allows users to export any checkpoint they have published to the Model Hub. You can only export models associated with the My Models and Organizational Models groups. SambaNova Models cannot be exported from the platform.

  • Added the ability to download the SambaStudio CLI utility packages.

    • Users can download the CLI and SDK packages directly from the Resources section of the SambaStudio platform.

  • Integrate re-ranker modelbox for semantic search.

    • This allows a deployed endpoint to be part of the semantic search pipeline.

  • Improved overall GUI and user experience of the platform.

    • Improved the user experience for restarting an endpoint.

    • Improved the user experience for adding datasets to the platform.

Bug fixes

  • Checkpoints properly being populated in the GUI for training jobs.

  • The on-start progress bar for a training job displays 0%.

  • Option to choose the number of instances aligns with the available system RDUs.