SambaStudio release notes

Release 24.9.1 (2024-10-17)

The SambaStudio 24.9.1 version release features are described below.

New features

SambaNova AI Starter Kits

SambaNova AI Starter Kit links.

Bug fixes

  • Fixed an issue where model cards sometimes did not display correctly.

  • Fixed an issue when adding a checkpoint to the Model Hub would sometimes fail.

  • Fixed an issue when switching tenants would sometimes cause an Access Forbidden. Please contact your administrator error to display with access to the tenant denied.

Known issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

  • Sometimes the number of nodes displayed in System Health will not be in sync with the number of nodes displayed in RDUs Available.

    • As a workaround, please refresh the browser after opening the System Health box.

  • If you have created your own Composition of Experts model prior to SambaStudio version 24.9.1 that uses one of the experts listed below, sometimes the corresponding deployed endpoint can stop functioning correctly. For example, updating the endpoint or starting/stopping the endpoint will fail and you will receive an error message.

    • As a workaround, please create a new Composition of Experts model using the same expert(s) in SambaStudio version 24.9.1 or later.

    • Expert list:

      • Meta-Llama-3-8B-Instruct

      • Meta-Llama-3-70B-Instruct

      • Meta-Llama-Guard-2-8B

Release 24.8.2 (2024-10-01)

SambaStudio release 24.8.2 is a patch update for 24.8.1 that includes performance improvements and minor bug fixes.

Release 24.8.1 (2024-09-16)

The SambaStudio 24.8.1 version release features are described below.

New features

  • You can now import a checkpoint from your local storage by using the new CLI import feature, which includes two modes.

    • The Interactive mode provides detailed feedback in the CLI on each step of the process and prompts you to input and verify your checkpoint’s configuration values.

    • The Non-interactive mode allows you to enter all of the required details for the checkpoint you wish to import, including configuration values in the CLI.

  • We also improved the workflow for adding a checkpoint using the GUI.

    • Step 1: Details includes enhanced information entry for your checkpoint.

      • This step now includes the option of editing the chat template before uploading your checkpoint. The chat template specifies the prompt template for a checkpoint to be used in Jinja format.

    • Step 2: Validation provides enhanced feedback on your checkpoint configurations.

  • Implemented a new DDR usage bar when creating your CoE model. The DDR usage bar displays the amount of Double Data Rate (DDR) memory used by the selected experts during the configuration process. SambaStudio tracks and displays the memory used as a percentage and displays it in real-time updates.

  • Added detailed logs for endpoints using OpenSearch.

  • Added the ability to update an endpoint while still keeping the endpoint active for use during the update process.

  • SambaStudio 24.8.1 provides the ability to run our latest release of high-performance inference models, Samba-1 Turbo v0.3.

SambaNova AI Starter Kits

SambaNova AI Starter Kit links.

Bug fixes

  • Fixed an issue where sometimes the sidebar in the Playground would not collapse correctly.

  • Fixed an issue in the Playground where new modal windows would sometimes open behind existing modal windows.

  • Fixed an issue in the Playground where Download Results button sometimes did not display correctly.

Known issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

  • Occasionally, when making consecutive API calls to different experts within the CoE endpoint, switching time to some experts can result in up to a 7 second delay.

Release 24.7.1 (2024-08-21)

The SambaStudio 24.7.1 version release features are described below.

New features

  • As we continue to update the SambaStudio GUI, we have implemented a new Playground interface with an improved workflow and new features.

    • The Playground now provides the ability to compare responses to your prompts using different models. You can add up to six model response panes to compare responses in the Playground.

    • The Playground supports both chat and single-turn interactions.

    • Models now include a chat icon next to their name to indicate that it supports chat functionality.

    • You can now search and filter the list of model experts to quickly locate and refine your choice(s).

  • Updated the RDUs Available information displayed.

    • You can now easily view RDU information based on a selected tenant, all tenants, as well as the hardware generations and nodes allocated for each.

    • The RDUs Available summary will now inform you if an RDU is not functioning correctly by identifying it as Unhealthy.

  • Added Tracking IDs for Batch inference and Training jobs.

    • Tracking IDs provide a current ID that can be used to identify and report issues to the SambaNova Support team.

    • A Tracking ID is created for each new job.

    • A new Tracking ID will be created if a stopped job is restarted.

    • Only jobs created with, or after, SambaStudio release 24.7.1 will display a Tracking ID.

SambaNova AI Starter Kits

SambaNova AI Starter Kit links:

Issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

Release 24.6.1 (2024-07-24)

The SambaStudio 24.6.1 version release features are described below.

New features

  • We are excited to announce SambaStudio’s new graphical user interface (GUI).

    • The new GUI is a light-themed interface that provides many new UX and UI updates, including improved workflows and intuitive navigation.

  • Implemented new model versioning.

    • You can choose the model version to use when creating jobs and endpoints.

    • You can update your endpoint’s model version directly from an Endpoint window.

  • Improved adding a checkpoint to the Model Hub from NFS to include a multi-step process with checks and validation.

  • Endpoint monitoring is now available to all user roles and can be access from an Endpoints table or the Endpoint window.

SambaNova AI Starter Kits

SambaNova AI Starter Kit links:

Known issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

  • Occasionally you will receive error that is not relative to your specific request, this is due to a known issue where all requests within the same batch will either succeed or fail together and result in an error. For example, in a batch of 4 requests with 3 requests that succeed and 1 request that fails because it exceeds the max_seq_length of its model, all 4 requests will fail and receive the same error message.

  • When cancelling a request of a Samba 1 Turbo App model, there is a known issue that does not prevent the model from processing the request. For example, if a user sends 30 concurrent requests and cancels all of them immediately, the model will still process those requests. This can result in slow time to first token (TTFT) for subsequent requests.

  • When using an E5 Mistral Embedding model in the Playground, there is a known issue where the endpoint will display an error, but will not crash.

  • The e5-mistral models only support the Predict API, not Stream API.

  • The e5-mistral models only support a batch size (BS) up to 8.

  • The Deepseek 33B models only support a batch size (BS) up to 4.

Release 24.5.1 (2024-06-14)

The SambaStudio 24.5.1 version release features are described below.

New features

  • Implemented new share roles of Collaborator and Viewer. Share roles allow greater control of allowing and assigning access to artifacts (projects and their jobs/endpoints, models, and datasets).

    • Share roles can be assigned for projects, models, and datasets by the owner/creator of the artifact (a User), tenant administrators (TenantAdmin), and organization administrators (OrgAdmin) user roles.

  • Integrated a new endpoint monitoring dashboard (Grafana) that displays metric information for all endpoints in a selected Tenant. This allows an organization administrator (OrgAdmin) or tenant administrator (TenantAdmin) to monitor the performance of SambaStudio endpoints.

  • Added integration with the OpenSearch Dashboard, which provides an interface to visualize and analyze logs.

  • Improved the available Playground modes by implementing a Prompt mode. Prompt mode replaces the Completion mode.

  • SambaStudio SN40L users can create endpoints that use dynamic batching of requests to the same model, which provides improved throughput and performance.

Bug fixes

  • Fixed an issue where the number of datasets in the Dataset Hub displayed incorrectly.

  • Fixed an issue that allowed a non-acceptable dataset name to be used when adding a new dataset, which would result in an upload failure. Now when attempting to add a dataset with a non-acceptable dataset name, an alert message will appear informing you to change the name before proceeding.

  • Fixed an issue where sometimes when creating a train job, selecting Clear from the ML App drop-down would not remove a selected ML App.

SambaNova AI Starter Kits

SambaNova AI Starter Kit links:

Known issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

  • Requests to SambaStudio have a 300-second connection timeout. Sometimes under high load, high volume curl requests will wait in the queue and surpass the SambaStudio connection timeout, resulting in a failed request. Please try one of the steps below as a workaround.

    • Reduce the load (requests per second) of the request.

    • Increase the number of instances used.

    • Contact SambaNova Support and open a support case to change the 300-second connection timeout configuration.

  • Please ensure that curl requests use valid JSON strings. If the input contains special characters, such as " or \n, they will need to be escaped by prepending them with a backslash (\). For example, “ would become \” and \n would become \\n.

    • If your "content": input contains the character \n, please prepend the character as shown below.

      Example input with escaped character
      {
  "conversation_id": "sambaverse-conversation-id",
  "messages": [
    {
      "message_id": 0,
      "role": "user",
      "content": "How can I print the completion field in this json data:\\n{\"result\":{\"status\":{\"complete\":true,\"exitCode\":0,\"elapsedTime\":29.62261724472046,\"message\":\"\",\"progress\":0,\"progressMessage\":\"\",\"reason\":\"\"},\"responses\":[{\"completion\":\"\\n\\nDark matter is a hypothet"
    }
  ]
}

  • Occasionally you will receive error that is not relative to your specific request, this is due to a known issue where all requests within the same batch will either succeed or fail together and result in an error. For example, in a batch of 4 requests with 3 requests that succeed and 1 request that fails because it exceeds the max_seq_length of its model, all 4 requests will fail and receive the same error message.

  • When cancelling a request of a Samba 1 Turbo App model, there is a known issue that does not prevent the model from processing the request. For example, if a user sends 30 concurrent requests and cancels all of them immediately, the model will still process those requests. This can result in slow time to first token (TTFT) for subsequent requests.

  • When using an E5 Mistral Embedding model in the Playground, there is a known issue where the endpoint will display an error, but will not crash.

  • The e5-mistral models only support the Predict API, not Stream API.

  • The e5-mistral models only support a batch size (BS) up to 8.

  • The Deepseek 33B models only support a batch size (BS) up to 4.

Release 24.4.1 (2024-05-10)

The SambaStudio 24.4.1 version release features are described below.

New features

  • We are very excited to announce that SambaStudio has added the ability to create your own Composition of Experts (CoE) models.

    • Now you can compose your own CoE model that is accessed via a single endpoint. This allows you to create a CoE model and define the expert models used in your composition.

  • Added new Samba-1 routers.

    • The Samba-1-Instruct-Router is a Composition of Experts router. It provides a single model experience to a subset of the Samba-1 experts that comprise the Enterprise Grade AI (EGAI) benchmark.

    • The Samba-1-Chat-Router is a composition of 7B parameter models that provides a lightweight, benchmark winning External link single model experience.

  • Improved the API features.

  • Improved the Endpoint window in the GUI to provide the Stream URL path as well as the Predict URL path.

SambaNova AI Starter Kits

  • Updates to the SambaNova AI Starter Kit example guides will now be included in the SambaStudio release notes. SambaNova AI Starter Kits External link are a collection of open-source examples and guides that facilitate the deployment of AI-driven use cases in the enterprise. You can use a deployed LLM endpoint in SambaStudio with one of the available AI Starter Kits.

    • Added a Samba-1 Routing Starter Kit.

      • Launched a Samba-1 CoE router starter kit showing customers how to use CoE models.

    • Added Sambaverse and Windows compatibility.

      • Updated Starter Kits for Sambaverse, enabling end-to-end customer self-service.

      • Launched a Windows-compatible starter kit deployment using Docker containers, which also includes a video.

    • Added web search and improved response quality in Starter Kits.

      • Launched a web search agent starter kit, allowing customers to use results as context to LLM prompt.

      • Improved response quality for document analysis Q&A.

    • Launched fine-tuning Starter Kits that includes information on synthetic Q&A data, low-resource languages, and embeddings models.

    • Added image and audio modalities.

      • Added post call analysis (including a video) and SambaNova embeddings for document analysis assistant.

    • Implemented SambaNova embeddings for EDGAR Q&A and document analysis assistant.

Known issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

  • Samba-1.0 Lite is currently not functioning as expected. As a workaround, please select a different CoE model for your needs.

  • An endpoint’s stream URL is not displaying in the SNSDK of Playground’s View Code and the CLI.

    • Please use the GUI to get an endpoint’s Stream URL path.

  • The SambaStudio GUI currently allows SN10 and SN30 hardware generations to start the Create your own CoE model process, even though CoE models will only run on SN40L hardware generations.

  • When creating a train job, the following models, including sequence size and vocabulary size in parenthesis, are currently not supported on SambaNova SN10 or SN30 RDU generations in both GUI and CLI workflows:

    • Llama-2-7b-16k-hf (Interpolate: True, from SS 4k)

    • Llama-2-7b-sambalingo-thai-base-hf (Llama2 7B Vocab size 57344)

  • When creating a non-CoE endpoint using the GUI or an endpoint using the CLI, the following models, including sequence size and vocabulary size in parenthesis, are currently not supported on SambaNova SN10 or SN30 RDU generations:

    • Llama-2-7b-80k-hf (Llama2 7B SS 64k)

    • Llama-2-7b-chat-16k-hf (Interpolate: True, from SS 4k)

    • Llama-2-7b-chat-80k-hf (Llama2 7B SS 64k)

  • When creating a batch inference job using the GUI or the CLI, the following models, including sequence size and vocabulary size in parenthesis, are currently not supported for batch inference on SambaNova SN10 or SN30 RDU generations:

    • Llama-2-7b-80k-hf (Llama2 7B SS 64k)

    • Llama-2-7b-chat-16k-hf (Interpolate: True, from SS 4k)

    • Llama-2-7b-chat-80k-hf (Llama2 7B SS 64k)

  • Requests to SambaStudio have a 300-second connection timeout. Sometimes under high load, high volume curl requests will wait in the queue and surpass the SambaStudio connection timeout, resulting in a failed request. Please try one of the steps below as a workaround.

    • Reduce the load (requests per second) of the request.

    • Increase the number of instances used.

    • Contact SambaNova Support and open a support case to change the 300-second connection timeout configuration.

  • Please ensure that curl requests use valid JSON strings. If the input contains special characters, such as " or \n, they will need to be escaped by prepending them with a backslash (\). For example, “ would become \” and \n would become \\n.

    • If your "content": input contains the character \n, please prepend the character as shown below.

      Example input with escaped character
      {
  "conversation_id": "sambaverse-conversation-id",
  "messages": [
    {
      "message_id": 0,
      "role": "user",
      "content": "How can I print the completion field in this json data:\\n{\"result\":{\"status\":{\"complete\":true,\"exitCode\":0,\"elapsedTime\":29.62261724472046,\"message\":\"\",\"progress\":0,\"progressMessage\":\"\",\"reason\":\"\"},\"responses\":[{\"completion\":\"\\n\\nDark matter is a hypothet"
    }
  ]
}

Release 24.2.2 (2024-04-01)

SambaStudio release 24.2.2 is a patch update for 24.2.1 that includes the features described below.

New features

Known issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

  • When creating a train job, the following models, including sequence size and vocabulary size in parenthesis, are currently not supported on SambaNova SN10 or SN30 RDU generations in both GUI and CLI workflows:

    • Llama-2-7b-16k-hf (Interpolate: True, from SS 4k)

    • Llama-2-7b-sambalingo-thai-base-hf (Llama2 7B Vocab size 57344)

  • When creating a non-CoE endpoint using the GUI or an endpoint using the CLI, the following models, including sequence size and vocabulary size in parenthesis, are currently not supported on SambaNova SN10 or SN30 RDU generations:

    • Llama-2-7b-80k-hf (Llama2 7B SS 64k)

    • Llama-2-7b-chat-16k-hf (Interpolate: True, from SS 4k)

    • Llama-2-7b-chat-80k-hf (Llama2 7B SS 64k)

  • When creating a batch inference job using the GUI or the CLI, the following models, including sequence size and vocabulary size in parenthesis, are currently not supported for batch inference on SambaNova SN10 or SN30 RDU generations:

    • Llama-2-7b-80k-hf (Llama2 7B SS 64k)

    • Llama-2-7b-chat-16k-hf (Interpolate: True, from SS 4k)

    • Llama-2-7b-chat-80k-hf (Llama2 7B SS 64k)

Release 24.2.1 (2024-03-19)

New features

  • With this release, SN40L External link users can utilize SambaStudio’s new Composition of Experts (CoE) architecture. The CoE architecture is a system of multiple experts, where each expert is a fully trained machine learning model.

  • SambaNova’s first Composition of Experts model, Samba-1, is available for SN40L users. Samba-1 combines the comprehensive power of trillion-parameter models with the precision and efficiency of specialized models accessed via a single endpoint.

  • Added new CoE specific model cards to the Model Hub. CoE model cards allow you to view the expert models used to create the CoE model.

  • SN40L users can create CoE endpoints to be deployed for use in the Playground.

  • Added the ability to select an expert for CoE endpoints in the Playground. This allows you to choose an expert model for your particular prompt.

Release 24.1.1 (2024-02-08)

New features

Known issues

  • If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Please ensure your browser is loading SambaStudio properly by performing a browser hard refresh to clear your browser cache. Press Command+Shift+R on a Mac or Control+Shift+R on Windows.

  • When an organization administrator deletes a tenant, sometimes the GUI will not update that the tenant is deleted. As a workaround, please refresh the browser if the GUI update takes longer than 30 seconds.

  • We’ve noticed an issue where the CLI is parsing hyperparameter boolean values incorrectly when using snapi job create with the --hyperparams-file option. As a workaround, please use quotes for hyperparameter boolean values, as shown in the example hyperparameters file below.

    Example hyperparameters file
    max_seq_length : 2048
precision: "true"

  • Cancelling the upload of a dataset can sometimes result in the cancelled dataset still displaying a status of Uploading in the Dataset Hub. The cancelled dataset may take up to 20 minutes for its status to change to Failed, at which point the dataset can be deleted.

Release 23.11.1 (2023-11-17)

New features

  • Added new endpoint API key features to both GUI and CLI endpoint workflows.

    • Added the ability to create multiple API keys for an endpoint.

    • Added the ability to revoke an endpoint API key to prevent further usage.

    • These features allow organizations to easily distribute and manage endpoints to different entities and users.

  • Added improved command options and performance for adding datasets using the CLI.

    • Previous versions of snapi dataset add commands and procedures are not compatible with release 23.11.1.

    • Adding a dataset using CLI commands results in faster uploads compared to using the GUI.

    • Using CLI commands to add datasets is recommended for datasets that are greater than 5GB or contain more than 1000 files.

  • Updated the installation requirements for the SambaNova SDK (snsdk) and SambaNova API (snapi) to Python 3.9.

Release 23.9.2 (2023-11-03)

New features

  • Organization administrators can now delete tenants using both the GUI and CLI workflows. Please be aware of the following considerations when deleting a tenant:

    • The default tenant cannot be deleted.

    • All jobs, endpoints, datasets, and models created in the tenant will be deleted.

    • Associated users will no longer have access to the tenant.

    • RDUs allocated to the tenant will be added to the default tenant.

    • Tenant deletion is irreversible.

  • You can now create a new training job directly from a checkpoint. This enables faster development and experimentation for your workflows.

  • Added insufficient storage alerts to the platform for storage related actions. You will still be able to proceed with your action, but you will need to clear storage space; otherwise, the action will eventually fail.

Bug fixes

  • Fixed the issue where if an endpoint did not successfully deploy and its status was Failed, the acquired Reconfigurable Dataflow Units™ (RDUs) were sometimes not released (described in the Known issues of Release 23.8.1).

  • Fixed an issue where a training job could be started using a model that was still in the state of Downloading. Now models will complete the downloading process before being available for training jobs.

Release 23.8.3 (2023-10-06)

SambaStudio release 23.8.3 is a patch update for 23.8.1 that includes the enhancements described below.

Enhancements

  • Improved general performance of the platform.

  • Improved the performance of the Metrics graph display used for evaluating a training job. The graph now downsamples job metrics that contain over 5K of data points, while still providing an accurate representation of the training job.

Release 23.8.1 (2023-09-29)

New features

  • Added new features and enhancements to the Playground.

    • The Playground now includes a chat mode experience. Chat mode provides a word-by-word response to your prompt with follow-on prompts kept within the context of your conversation. This allows the Playground to understand your prompts without the need to restate preceding information.

    • Improved and updated the Add from presets drop-down menu and input field of the Playground editor for both chat and completion mode.

    • Renamed the tuning parameter Return logits to Top logprobs to align with the updated industry standard.

      • Improved interacting with Top logprobs to only require hovering over the highlighted token to display the generated list of tokens and their probabilities.

  • Hyperparameters now include corresponding default values and constraints based on the model selected when creating a training job.

Bug fixes

  • Fixed an issue where clicking Resume from a stopped training job’s detail page would sometimes result in the job failing to resume (described in the Known issues of Release 23.7.1).

  • Fixed an issue where uploading a dataset from a local machine would sometimes result in an error (described in the Known issues of Release 23.7.1).

Known issues

If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • The Top logprobs tuning parameter currently does not function in chat mode of the Playground.

  • Hovering over a response in the Playground editor to view Top logprobs can sometimes cause the editor display to fluctuate.

  • If an endpoint does not successfully deploy and its status is Failed, the acquired Reconfigurable Dataflow Units™ (RDUs) are sometimes not released. As a workaround, please delete the failed endpoint to release the RDUs.

Release 23.7.1 (2023-08-11)

New features

  • Improved user interface that enables access to global features and information from anywhere within the platform.

  • New controlled sharing implemented with tenants and organization.

    • Provides users with flexible sharing options for datasets, models, and endpoints.

    • Organization datasets and models are now called Shared datasets and Shared models respectively.

  • The Dashboard now provides access to live endpoints in a tenant across all projects.

  • Datasets can now be deleted that are not being used in running jobs.

Bug fixes

  • Fixed an issue where the UI may not automatically refresh when creating or deleting projects or jobs.

  • Fixed an issue in the Playground where the Stop sequences side panel displayed an inaccurate value.

  • Fixed an issue where a model selected from the Select model drop-down may not remain selected while creating a training job.

Known issues

If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

Release 23.5.1 (2023-06-30)

New features

  • Added new features and enhancements to the Playground.

    • Added new Tuning parameters, including Stop sequences, Return logits, Repetition penalty, and Top k.

    • Implemented tooltips for Tuning parameters and detailed information cards for each parameter.

    • The Playground now displays a token count for each submission. The display shows the total number of tokens for the current submission relative to the maximum number of tokens supported by the model (max sequence length).

    • Improved the visibility of the generated responses for each submission. Responses are displayed in the input/output field with highlighted blue text.

    • Improved the notification error if max sequence length is exceeded.

    • Updated and improved User presets, which include examples for each model and provide better understanding on how to best use models.

  • Redesigned and improved the Model Hub

    • Updated design to include a more intuitive two-panel layout featuring a new model card design and filtering panel.

    • Improved the filtering and search functions to easily refine the display of corresponding models.

    • All available SambaNova provided models can easily be displayed by selecting the Available tab.

    • By default, new SambaNova models will not be automatically stored in an organization’s environment. This will allow organizations to save storage space.

    • Organization administrators (OrgAdmin) can download SambaNova models to the Model Hub that they wish to use for their organization.

    • Upcoming SambaNova models are visible to all users within the platform.

    • All users can view the model card of SambaNova Available models and contact their organization administrator (OrgAdmin) to download a model to the Model Hub. Downloaded models are available for use by the entire organization.

  • Improved the Train jobs and Create and use endpoints workflows by removing the concept of a Task.

    • You no longer need to specify a Task during job or endpoint creation. Instead, you select a model to use directly during the workflow or use the new ML App field to filter the list of models to use.

  • Improvements to Datasets.

    • Users can select multiple ML Apps while adding a dataset. This will help attach the same dataset with models under different ML Apps, for example Generative Tuning 1.5 and 13B models.

    • Improved dataset upload speed that will increase the upload speed of larger size datasets.

  • Improvements to Tenants and management.

    • Significant number of refresh improvements to the Tenant management window, resulting in the improvement of tenant and organization level RDUs information displayed in the GUI.

    • Simplified tenant creation process by removing Display Name. Only the tenant name is required. Requirements for tenant name are displayed in the GUI during creation.

  • Improved saving a checkpoint to the Model Hub by allowing users to create a model card during the process. All dataset and parameter information will automatically be taken from the respective job and included in the new model card.

  • Improved logs for batch inference jobs as well as data parallel train jobs.

Bug fixes

  • Fixed issue where jos failed due to dataset names that included spaces. Datasets with spaces in names will no longer be allowed.

  • Fixed issue where hyperparameters that were changed invalidated the initial model, for example num_label for ASR. Now those parameters will be grayed-out and not modifiable.

  • Fixed issue where in the case of certain recoverable errors, failed training jobs should get resumed automatically from the last good checkpoint.

  • Fixed issue where a job progress bar might not automatically refresh, requiring a manual refresh of the browser. Job progress bars will now automatically get refreshed during status updates.

  • Full loss plot will now be available for the entire run of a job.

  • Fixed issue where View code in the Playground could create an incorrect curl command.

Known issues

If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • In the input field on the Playground, an input with an exclamation point (!) that does not have a trailing space after it, can cause curl or snapi commands to fail when using the View code feature.

  • In the Playground, setting Return logits to more than 0 can cause invalid characters to display in the response.

  • In the Playground, when using an endpoint created from GPT_13B_Generic_Instruction_Tuning_v2, selecting the following User Presets can cause their corresponding issue:

    • Selecting Extractive QA (Natural Questions) from the User preset drop-down, can cause the response code copied (Copy code) from the CLI option of View code, to fail when the response code is inputted into the terminal.

    • Selecting Sentiment Analysis (IMDB) from the User preset drop-down, can cause the response code copied (Copy code) from the Python SDK option of View code, to fail when the response code is inputted into the terminal.

  • Loss values in the Checkpoints table (displayed in the Val_loss and Train_loss columns) of a training job can be displayed incorrectly when values are less or equal to 0.01. As a workaround, the loss values displayed in the Logs section for the training job are correct.

Release 23.4.2 (2023-06-12)

SambaStudio release 23.4.2 is a patch update for 23.4.1 that addresses the following issues.

Bug fixes

  • Fixed issue where special characters such as period (.), dash (-), and underscore (_) were unsupported in usernames. Special characters are now supported in usernames and will not result in an error message when adding users.

  • Fixed issue where the Diarization ASR Pipeline v2 model, Diarization_ASR_Pipeline_V2, would fail due to a checkpoint not being available.

  • Fixed issue where a model training job, that referenced an AWS dataset, could get stuck in a status of Retrying.

  • Fixed issue where data parallel is enabled in the SambaNova ViT model, Vit_B_Classification, even though it is not supported.

Release 23.4.1 (2023-05-25)

New features

  • Added the ability to create and manage multiple tenants within an organization.

    • A tenant is an allotment of users and resources within an organization. Organizations can leverage tenants for distinct purposes, such as development or production, and then share common resources across tenants. Each tenant has its own resources, users, and workloads. SambaStudio allows organizations to allocate resources across multiple tenants.

  • Added predefined user roles that can be assigned across multiple tenants.

    • A user is part of an organization and can be assigned a predefined role: organization administrator, tenant administrator, or user. A user can be part of and assigned to multiple tenants. All users within the same tenant can view or manage projects and workloads present in the tenant. SambaStudio allows organizations to assign users across multiple tenants via the organization administrator or tenant administrator role.

  • Improved search filters in the Model hub.

    • The Model hub contains new search filters to narrow search results of model cards. The filters can be displayed by clicking the arrow > next to the Search field in the model hub.

  • Added the ability to view and copy code blocks in the Playground.

    • The Playground now provides the ability to view and copy CURL, CLI, or Python SDK code blocks from the GUI.

  • Added new SambaNova API (snapi) CLI commands.

    • New CLI commands for tenant management including creating, updating, and renaming tenants.

    • New CLI commands for user management including adding users, assigning roles, and deleting users.

Bug fixes

  • Fixed an issue when uploading a dataset sometimes a validation error would display.

  • Fixed minor GUI issues.

Known issues

If you need assistance with any issue, please see the Help section in the SambaStudio introduction for information on contacting SambaNova Support and accessing the SambaNova Support Cloud.

  • Email username, character limit, and special characters issue.

    • Multiple email domains having the same username (account name) will create a conflict in SambaStudio. For example, a user with the email ab@c.com and a user with the email ab@d.com will create a conflict and be treated the same in the platform.

    • The number of characters for a username is limited to 36. If an admin tries to add a username that exceeds the character limit, the platform will not allow the operation and will produce an error.

    • Special characters such as period (.), dash (-), and underscore (_) are unsupported in usernames. For example, a user with the email jane.doe@domain.com would have a username jane.doe and is not supported.

  • Base models do not support inference and cannot be deployed for endpoints. It is recommended to use Base models for training and not inference.

  • When an Organization administrator (OrgAdmin) creates a new tenant using the GUI, it cannot be deleted. As a workaround, we suggest finalizing tenant names before creating new tenants in the platform.

  • Tenant and resource information can take several seconds to update. This issue can affect both the User management GUI and the CLI command snapi tenant list.

  • Removed the Return logits tuning parameter from the Playground.

Release 23.3.1 (2023-04-17)

New features

  • Added model cards to the Model hub.

    • The Model hub now provides model information details via model cards. The updated GUI includes a two-panel interface for viewing the SambaStudio model cards. The side-by-side panels display the model card list in the left panel and the content of the selected model card in the right panel.

  • Added a Notifications section.

    • The new Notifications panel displays platform specific messages in a scrollable list. Each notification includes a detailed heading, a tracking id number, and a creation timestamp. Notifications are further defined by type: Success, Info, Warnings, Errors, and Critical.

  • Improved information presented in the Projects table.

    • Added a column displaying the number of checkpoints associated with a project. Projects now display the number of associated jobs and endpoints.

    • Added a column displaying the status of the project. Provides visual feedback on project statuses. For example, Deleting will display when the project is in the process of being deleted from the platform.

  • Added training job logs in GUI.

    • Training jobs now include a Logs section that allows you to preview and download logs of your training session in the GUI. Logs can help you track progress, identify errors, and determine the cause of potential errors.

  • Added the ability to edit an endpoint’s model.

    • Editing endpoints in the GUI now allows users to choose a different model. This allows you to change the endpoint’s model negating the need to create a new endpoint for a different model.

  • Added features to the Playground.

    • The Playground now allows you to download the results of the last response provided.

    • Added a Return logits tuning parameter, which adjusts how many log probabilities to return along with predictions.

Bug fixes

  • Fixed notifications not being sent for serve jobs.

  • Fixed upper limit value of Max tokens to generate tuning parameter in the Playground.

Known issues

The Return logits tuning parameter in the Playground can cause submission errors. As a workaround, do not change or set the value.

Release 23.2.1 (2023-03-17)

New features

  • Update the platform name.

    • Dataflow-as-a-Service is now SambaStudio! The new name aligns with SambaNova’s commitment to bring industry-leading AI performance to applications and services.

  • Added the ability to import models to the Model Hub.

    • SambaStudio allows users to add a model-checkpoint from a training job.

    • SambaStudio allows users to take a model from a different environment and add it as a new model to the current SambaStudio environment through NFS.

  • Added the ability to export models from the Model Hub.

    • SambaStudio allows users to export any checkpoint they have published to the Model Hub. You can only export models associated with the My Models and Organizational Models groups. SambaNova Models cannot be exported from the platform.

  • Added the ability to download the SambaStudio CLI utility packages.

    • Users can download the CLI and SDK packages directly from the Resources section of the SambaStudio platform.

  • Integrate re-ranker modelbox for semantic search.

    • This allows a deployed endpoint to be part of the semantic search pipeline.

  • Improved overall GUI and user experience of the platform.

    • Improved the user experience for restarting an endpoint.

    • Improved the user experience for adding datasets to the platform.

Bug fixes

  • Checkpoints properly being populated in the GUI for training jobs.

  • The on-start progress bar for a training job displays 0%.

  • Option to choose the number of instances aligns with the available system RDUs.