Playground

The Playground provides an in-platform experience for generating predictions using deployed generative tuning endpoints. You can choose between a chat mode and completion mode experience.

Playground window
Figure 1. Playground interface

Requirements

A live endpoint is required to use the Playground. If no endpoint is available, the message below will display.

Try Now
Figure 2. No live endpoint exists

Access the Playground from the left menu

To access the Playground experience directly, click Playground from the left menu. The Playground window will open.

Playground menu
Figure 3. Playground menu icon

Access the Playground from an endpoint window

From an Endpoint window, click Try now. The Playground window will open.

Try Now
Figure 4. Generative tuning endpoint Try Now

Use the Playground editor

  1. Select your live endpoint from the Endpoint drop-down. Starting with SambaStudio release 24.2.1, SN40L External link users can select a Composition of Experts (CoE) endpoint to use. CoE endpoints are indicated by a CoE badge.

    1. When selecting a CoE endpoint, you will also be able to choose an expert for your particular task.

      Endpoint select
      Figure 5. Playground endpoint select

  2. Choose the type of interaction you wish to have from the Mode drop-down.

    1. Chat mode provides a word-by-word response to your prompt. Additionally, follow-on prompts are kept within the context of your conversation. This allows the Playground to understand your prompts without the need to restate preceding information.

    2. In Completion mode, the Playground will provide complete statement responses to your prompt. Providing specific instructions in your prompt will help produce the best generations.

      Some CoE experts are only supported in Completion mode. The Mode drop-down will not display if one of those experts is selected.
      Mode select
      Figure 6. Playground mode select

  3. If your endpoint supports presets, select one from the Add from presets drop-down to populate the editor and quickly experience generative tuning.

    Add from presets chat mode
    Figure 7. Add from presets Chat mode
    Add from presets completion mode
    Figure 8. Add from presets Completion mode

    1. Alternatively, you can input text directly into the editor, without selecting a preset.

  4. Click Submit to initiate a response by the platform.

    1. Chat mode is displayed in the editor with row striping, designating your prompt as a highlighted row.

      1. Click Stop generating to force the editor to halt and discontinue generating a response that is in progress.

      2. Click the copy icon Copy icon to copy the corresponding prompt or response to your clipboard.

    2. A Completion mode response is displayed in the editor with highlighted blue text.

      1. Click Download results to download the last response provided. The file will be downloaded to the location configured by your browser.

        Add from presets chat mode
        Figure 9. Chat mode response
        Add from presets completion mode
        Figure 10. Completion mode response

  5. Click the clear editor icon Trash icon to clear the Playground editor. This will remove all inputs and responses, as well as reset the tokens available.

    If your prompt returns an unexpected response or error, clear the editor before submitting additional prompts.

Select a CoE expert

CoE endpoints in the Playground provide a list of specialized experts to choose for your particular task.

  • Choose an expert to use with your prompt from the Select Experts drop-down.

  • Choosing a different expert will reset the Playground editor and remove your existing prompt.

CoE select expert
Figure 11. CoE select expert

Task-specific prompt templates

To get optimal responses from the CoE experts listed below, use a task-specific prompt template to format your input. Click the links below for each expert to navigate to the corresponding template.

System prompts

System prompts are unique instruction messages used to steer the behavior of models and their resulting outputs.

Edit the system prompt using the GUI

Click the edit icon Edit icon and select View/Edit System Prompt to open the System Prompt edit box. Input your system prompt and click Apply changes.

  • For CoE endpoints, the system prompt applies to the selected expert.

  • For non-CoE endpoints, the system prompt applies to the selected endpoint.

  • Edits to the system prompt remain in effect only for your current session. When you log out of the platform, the system prompt will revert to its unedited state.

  • An edited system prompt is denoted by a red dot in the edit icon Edited system prompt.

System prompt edit box
Figure 12. System prompt edit box

Prompt guidelines

We recommend using the following guidelines when submitting prompts to the Playground.

Prompt structure

End the prompts with either a colon (:), a question mark (?), or another way of letting the model know that it is time for it to start generating. For example, using Please summarize the previous article: (with a colon) is a better prompt than Please summarize the previous article (without a colon). Adding these annotations tends to lead to better generations as it indicates to the model that you’ve finished your question and are expecting an answer.

Resubmitting prompts

Please ensure that you do not submit an <|endoftext|> token in your prompt. This might happen if you hit submit twice after the model returns its generations.

Tokens

Tokens are basic units based on text that are used when processing a prompt to generate a language output, or prediction. They can be thought of as pieces of words. Tokens are not defined exactly on where a word begins or ends and can include trailing spaces (spaces after a word) and subwords. Before the processing of a prompt begins, the input is broken into tokens.

The SambaStudio Playground displays the available tokens, updated for each submission. Click Tokens available to expand and show Max seq length and Used tokens.

Max seq length

Represents the maximum number of tokens supported by the model used for the endpoint.

Used tokens

The number of tokens displayed in the editor, including inputs and responses.

Tokens available

Displays the number of tokens that are remaining to be used. The number is updated for each submission by subtracting the Used tokens value from Max seq length. Click the clear editor icon Trash icon to reset the tokens available and clear the editor.

Token available
Figure 13. Tokens available expanded

Tuning parameters

Tuning parameters provide additional flexibility and options for generative tuning. Adjusting these parameters allows you to search for the optimal values to maximize the performance and output of the response.

Tuning parameters
Figure 14. Tuning parameters panel

View tuning parameter information in the GUI

A tuning parameter’s definition and values are viewable within the SambaStudio GUI. Follow the steps below to view information for a tuning parameter.

  1. In the Tuning parameters panel, hover over the parameter name you wish to view. An overview parameter card will display.

  2. Click the > (right arrow) to display the complete parameter card that includes its definition and values.

  3. Click the X to close the complete parameter card.

Tuning parameter card
Figure 15. Tuning parameter information

View code

Follow the steps below to view and copy code generated from the current input.

  1. At the bottom of the editor, click View code to open the View code window.

  2. Click the CURL, CLI, or Python SDK tab to view the corresponding code block and make a request programmatically.

  3. Click Copy code to copy the selected code block to your clipboard.

  4. Click Close, or the upper right X, to close the window and return to the Playground.

View code window
Figure 16. View code