open_ai provider type accepts any endpoint that follows the OpenAI Chat Completions format. Pointing it at SambaCloud lets a Deepgram voice agent answer with SambaNova models. Because voice depends on how fast the first token arrives, this guide also measures that for each model.
Prerequisites
- A SambaCloud account and API key
- A Deepgram account and API key. Create the key in the console under API keys.
- Python 3.9 or higher
macOS exposes its Python as
python3 rather than python. Check your version with python3 --version. The stock macOS Python 3.9 works with this guide.Installation and setup
1
Create a virtual environment
2
Install the WebSocket client
websockets 17.4 on Python 3.12 and websockets 15.0.1 on Python 3.9.6.3
Set your API keys
Connect a voice agent to SambaCloud
You configure the agent with aSettings message that you send as soon as the WebSocket opens. The part that changes for SambaNova is agent.think:
provider.typeisopen_ai, which means “OpenAI Chat Completions format”. The model name is a SambaCloud model ID.endpoint.urlis the full chat completions URL, not the base URL.endpoint.headers.authorizationcarries your SambaCloud key.
Settings message, then types a question to the agent with an InjectUserMessage message so you can test without a microphone. It prints the transcript and the latency report that Deepgram sends after each turn, and saves the spoken reply as reply.wav.
voice_agent.py
Output
ttt_text_latency is the time to the first text token from the LLM, which is the number to watch for a voice agent. total_latency is the time from the end of the user’s turn to the first byte of audio. Your numbers depend on the model, the prompt, and your network. Because this test sends text, the figures do not include speech-to-text time.Choose a model for voice
The script usesMiniMax-M3.1-Flash. Voice is where first-token latency matters most, so it is worth comparing models. Each row is ttt_text_latency measured with the script above, changing only MODEL:
gemma-4-31B-it varied the most. Its first token took from about 1.1 seconds to over 10 seconds, with most runs between 3 and 6 seconds. Both Meta-Llama-3.3-70B-Instruct and gpt-oss-120b also had an occasional slow first token of several seconds in testing. Measure over several runs before you quote a latency number.Troubleshooting
KeyError: 'DEEPGRAM_API_KEY' or 'SAMBANOVA_API_KEY'
KeyError: 'DEEPGRAM_API_KEY' or 'SAMBANOVA_API_KEY'
An environment variable is not set in the shell you run Python from. Set it as shown in Installation and setup, then run the script again from the same shell.
server rejected WebSocket connection: HTTP 401
server rejected WebSocket connection: HTTP 401
Deepgram rejected the connection, so
DEEPGRAM_API_KEY is not valid. Create a key in the Deepgram console under API keys and export it again.Code: 404 Not Found ... model_not_found
Code: 404 Not Found ... model_not_found
The
model value is not a SambaCloud model ID. See SambaCloud models for the current list.Code: 404 Not Found. Body: 404 page not found
Code: 404 Not Found. Body: 404 page not found
The
endpoint.url is the base URL. Use the full path, https://api.sambanova.ai/v1/chat/completions.We have now waited 5 seconds for a think response
We have now waited 5 seconds for a think response
Deepgram prints this warning when the LLM has not started answering after 5 seconds, and again at 10 seconds. A slower model, such as the reasoning model
MiniMax-M3.1-Flash or gemma-4-31B-it, can take more than 5 seconds to produce its first token. The reply still arrives. To avoid long pauses, switch to a faster model as described in Choose a model for voice.FAILED_TO_THINK: Failed to think. Please check your agent.think settings.
FAILED_TO_THINK: Failed to think. Please check your agent.think settings.
Deepgram retried the LLM request three times and gave up. The
Warning messages printed just before this error show the real cause, such as one of the two errors above.CLIENT_MESSAGE_TIMEOUT: We did not receive audio within our timeout
CLIENT_MESSAGE_TIMEOUT: We did not receive audio within our timeout
Deepgram closes a session that receives no audio for a while. This error can appear when the model is slow to answer. In testing, it appeared in runs where the first text token took about 4 seconds or longer. The script prints the error, stops, and still saves
reply.wav. If you keep a text-only session open longer, expect this error as well.
