Skip to main content
Deepgram is a speech platform whose Voice Agent API runs speech-to-text, an LLM, and text-to-speech over a single WebSocket. The LLM step is swappable: the open_ai provider type accepts any endpoint that follows the OpenAI Chat Completions format. Pointing it at SambaCloud lets a Deepgram voice agent answer with SambaNova models. Because voice depends on how fast the first token arrives, this guide also measures that for each model.

Prerequisites

  • A SambaCloud account and API key
  • A Deepgram account and API key. Create the key in the console under API keys.
  • Python 3.9 or higher
macOS exposes its Python as python3 rather than python. Check your version with python3 --version. The stock macOS Python 3.9 works with this guide.

Installation and setup

1

Create a virtual environment

2

Install the WebSocket client

This guide was tested with websockets 17.4 on Python 3.12 and websockets 15.0.1 on Python 3.9.6.
3

Set your API keys

Connect a voice agent to SambaCloud

You configure the agent with a Settings message that you send as soon as the WebSocket opens. The part that changes for SambaNova is agent.think:
  • provider.type is open_ai, which means “OpenAI Chat Completions format”. The model name is a SambaCloud model ID.
  • endpoint.url is the full chat completions URL, not the base URL.
  • endpoint.headers.authorization carries your SambaCloud key.
The script below sends this Settings message, then types a question to the agent with an InjectUserMessage message so you can test without a microphone. It prints the transcript and the latency report that Deepgram sends after each turn, and saves the spoken reply as reply.wav.
voice_agent.py
Output
ttt_text_latency is the time to the first text token from the LLM, which is the number to watch for a voice agent. total_latency is the time from the end of the user’s turn to the first byte of audio. Your numbers depend on the model, the prompt, and your network. Because this test sends text, the figures do not include speech-to-text time.

Choose a model for voice

The script uses MiniMax-M3.1-Flash. Voice is where first-token latency matters most, so it is worth comparing models. Each row is ttt_text_latency measured with the script above, changing only MODEL:
MiniMax-M3.1-Flash is a reasoning model. It thinks before it answers, and that time counts toward the first text token, so the reply starts later than with a non-reasoning model and the delay varies from turn to turn. For a voice agent where the pause matters most, set MODEL to Meta-Llama-3.3-70B-Instruct, which answered in under half a second in most runs.
gemma-4-31B-it varied the most. Its first token took from about 1.1 seconds to over 10 seconds, with most runs between 3 and 6 seconds. Both Meta-Llama-3.3-70B-Instruct and gpt-oss-120b also had an occasional slow first token of several seconds in testing. Measure over several runs before you quote a latency number.

Troubleshooting

An environment variable is not set in the shell you run Python from. Set it as shown in Installation and setup, then run the script again from the same shell.
Deepgram rejected the connection, so DEEPGRAM_API_KEY is not valid. Create a key in the Deepgram console under API keys and export it again.
The warning comes from the LLM request to SambaCloud, so the key in the authorization header is not valid. Copy a key from the SambaCloud API keys page and keep the Bearer prefix.
The model value is not a SambaCloud model ID. See SambaCloud models for the current list.
The endpoint.url is the base URL. Use the full path, https://api.sambanova.ai/v1/chat/completions.
Deepgram prints this warning when the LLM has not started answering after 5 seconds, and again at 10 seconds. A slower model, such as the reasoning model MiniMax-M3.1-Flash or gemma-4-31B-it, can take more than 5 seconds to produce its first token. The reply still arrives. To avoid long pauses, switch to a faster model as described in Choose a model for voice.
Deepgram retried the LLM request three times and gave up. The Warning messages printed just before this error show the real cause, such as one of the two errors above.
Deepgram closes a session that receives no audio for a while. This error can appear when the model is slow to answer. In testing, it appeared in runs where the first text token took about 4 seconds or longer. The script prints the error, stops, and still saves reply.wav. If you keep a text-only session open longer, expect this error as well.

Additional resources