> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deepgram integration guide

[Deepgram](https://deepgram.com/) is a speech platform whose Voice Agent API runs speech-to-text, an LLM, and text-to-speech over a single WebSocket. The LLM step is swappable: the `open_ai` provider type accepts any endpoint that follows the OpenAI Chat Completions format. Pointing it at SambaCloud lets a Deepgram voice agent answer with SambaNova models. Because voice depends on how fast the first token arrives, this guide also measures that for each model.

## Prerequisites

* A [SambaCloud](https://cloud.sambanova.ai/?utm_source=deepgram\&utm_medium=external\&utm_campaign=cloud_signup) account and [API key](https://cloud.sambanova.ai/apis?utm_source=deepgram\&utm_medium=external\&utm_campaign=cloud_signup)
* A [Deepgram](https://console.deepgram.com/signup) account and API key. Create the key in the console under **API keys**.
* Python 3.9 or higher

> [!NOTE]
>
> macOS exposes its Python as `python3` rather than `python`. Check your version with `python3 --version`. The stock macOS Python 3.9 works with this guide.

## Installation and setup

**1. Create a virtual environment**

```bash macOS / Linux theme={}
python3 -m venv .venv
source .venv/bin/activate
```

```powershell Windows theme={}
python -m venv .venv
.venv\Scripts\Activate.ps1
```

**2. Install the WebSocket client**

```bash theme={}
pip install websockets
```

This guide was tested with `websockets` 17.4 on Python 3.12 and `websockets` 15.0.1 on Python 3.9.6.

**3. Set your API keys**

```bash macOS / Linux theme={}
export SAMBANOVA_API_KEY="your-sambanova-api-key"
export DEEPGRAM_API_KEY="your-deepgram-api-key"
```

```powershell Windows theme={}
$env:SAMBANOVA_API_KEY="your-sambanova-api-key"
$env:DEEPGRAM_API_KEY="your-deepgram-api-key"
```

## Connect a voice agent to SambaCloud

You configure the agent with a `Settings` message that you send as soon as the WebSocket opens. The part that changes for SambaNova is `agent.think`:

```json theme={}
"think": {
  "provider": {
    "type": "open_ai",
    "model": "MiniMax-M3.1-Flash"
  },
  "endpoint": {
    "url": "https://api.sambanova.ai/v1/chat/completions",
    "headers": { "authorization": "Bearer your-sambanova-api-key" }
  }
}
```

* `provider.type` is `open_ai`, which means "OpenAI Chat Completions format". The model name is a SambaCloud model ID.
* `endpoint.url` is the full chat completions URL, not the base URL.
* `endpoint.headers.authorization` carries your SambaCloud key.

The script below sends this `Settings` message, then types a question to the agent with an `InjectUserMessage` message so you can test without a microphone. It prints the transcript and the latency report that Deepgram sends after each turn, and saves the spoken reply as `reply.wav`.

```python voice_agent.py theme={}
import asyncio
import json
import os
import wave

import websockets

MODEL = "MiniMax-M3.1-Flash"
QUESTION = "What can you do?"

settings = {
    "type": "Settings",
    "audio": {
        "input": {"encoding": "linear16", "sample_rate": 24000},
        "output": {"encoding": "linear16", "sample_rate": 24000, "container": "none"},
    },
    "agent": {
        "listen": {"provider": {"type": "deepgram", "model": "nova-3"}},
        "think": {
            "provider": {"type": "open_ai", "model": MODEL, "temperature": 0.7},
            "endpoint": {
                "url": "https://api.sambanova.ai/v1/chat/completions",
                "headers": {"authorization": f"Bearer {os.environ['SAMBANOVA_API_KEY']}"},
            },
            "prompt": "You are a helpful voice assistant. Keep answers under two sentences.",
        },
        "speak": {"provider": {"type": "deepgram", "model": "aura-2-thalia-en"}},
    },
}


async def main():
    headers = {"Authorization": f"Token {os.environ['DEEPGRAM_API_KEY']}"}
    audio = bytearray()
    async with websockets.connect(
        "wss://agent.deepgram.com/v1/agent/converse", additional_headers=headers
    ) as ws:
        await ws.send(json.dumps(settings))
        while True:
            try:
                message = await asyncio.wait_for(ws.recv(), timeout=3 if audio else 20)
            except asyncio.TimeoutError:
                break
            if isinstance(message, bytes):
                audio.extend(message)
                continue
            event = json.loads(message)
            kind = event["type"]
            if kind == "SettingsApplied":
                await ws.send(json.dumps({"type": "InjectUserMessage", "content": QUESTION}))
            elif kind == "ConversationText":
                print(f"{event['role']}: {event['content']}")
            elif kind == "LatencyReport":
                print({k: round(v, 3) for k, v in event.items() if k != "type"})
            elif kind in ("Warning", "Error"):
                print(f"{kind}: {event['description']}")
                if kind == "Error":
                    break

    with wave.open("reply.wav", "wb") as wav:
        wav.setnchannels(1)
        wav.setsampwidth(2)
        wav.setframerate(24000)
        wav.writeframes(bytes(audio))
    print(f"Saved reply.wav ({len(audio)} bytes of audio)")


asyncio.run(main())
```

```bash theme={}
python voice_agent.py
```

```text Output theme={}
user: What can you do?
{'ttt_token_latency': 1.534}
{'ttt_text_latency': 1.534}
assistant: I can answer questions, explain concepts, brainstorm ideas, draft and edit text, summarize information, and help with planning or problem-solving.
{'tts_latency': 0.066}
{'total_latency': 1.599}
Saved reply.wav (503040 bytes of audio)
```

> [!NOTE]
>
> `ttt_text_latency` is the time to the first text token from the LLM, which is the number to watch for a voice agent. `total_latency` is the time from the end of the user's turn to the first byte of audio. Your numbers depend on the model, the prompt, and your network. Because this test sends text, the figures do not include speech-to-text time.

## Choose a model for voice

The script uses `MiniMax-M3.1-Flash`. Voice is where first-token latency matters most, so it is worth comparing models. Each row is `ttt_text_latency` measured with the script above, changing only `MODEL`:

| Model | Time to first text token |
| :- | :- |
| `MiniMax-M3.1-Flash` | 1.4 to 3.5 s in 8 of 10 runs, 10 to 11 s in two runs |
| `Meta-Llama-3.3-70B-Instruct` | 0.24 to 0.41 s in 10 of 12 runs, over 3 s in two runs |
| `gpt-oss-120b` | 0.22 to 0.26 s in 4 of 5 runs, 4.9 s in one run |
| `gemma-4-31B-it` | 1.1 to 10.8 s in 34 runs, median 4.3 s (8 runs under 2 s, 11 runs over 5 s) |

> [!WARNING]
>
> `MiniMax-M3.1-Flash` is a reasoning model. It thinks before it answers, and that time counts toward the first text token, so the reply starts later than with a non-reasoning model and the delay varies from turn to turn. For a voice agent where the pause matters most, set `MODEL` to `Meta-Llama-3.3-70B-Instruct`, which answered in under half a second in most runs.

> [!NOTE]
>
> `gemma-4-31B-it` varied the most. Its first token took from about 1.1 seconds to over 10 seconds, with most runs between 3 and 6 seconds. Both `Meta-Llama-3.3-70B-Instruct` and `gpt-oss-120b` also had an occasional slow first token of several seconds in testing. Measure over several runs before you quote a latency number.

## Troubleshooting

> [accordion] KeyError: 'DEEPGRAM\_API\_KEY' or 'SAMBANOVA\_API\_KEY'
>
> An environment variable is not set in the shell you run Python from. Set it as shown in [Installation and setup](#installation-and-setup), then run the script again from the same shell.

> [accordion] server rejected WebSocket connection: HTTP 401
>
> Deepgram rejected the connection, so `DEEPGRAM_API_KEY` is not valid. Create a key in the Deepgram console under **API keys** and export it again.

> [accordion] Code: 401 Unauthorized ... Incorrect API key provided
>
> The warning comes from the LLM request to SambaCloud, so the key in the `authorization` header is not valid. Copy a key from the [SambaCloud API keys page](https://cloud.sambanova.ai/apis) and keep the `Bearer ` prefix.

> [accordion] Code: 404 Not Found ... model\_not\_found
>
> The `model` value is not a SambaCloud model ID. See [SambaCloud models](https://docs.sambanova.ai/docs/en/models/sambacloud-models.md) for the current list.

> [accordion] Code: 404 Not Found. Body: 404 page not found
>
> The `endpoint.url` is the base URL. Use the full path, `https://api.sambanova.ai/v1/chat/completions`.

> [accordion] We have now waited 5 seconds for a think response
>
> Deepgram prints this warning when the LLM has not started answering after 5 seconds, and again at 10 seconds. A slower model, such as the reasoning model `MiniMax-M3.1-Flash` or `gemma-4-31B-it`, can take more than 5 seconds to produce its first token. The reply still arrives. To avoid long pauses, switch to a faster model as described in [Choose a model for voice](#choose-a-model-for-voice).

> [accordion] FAILED\_TO\_THINK: Failed to think. Please check your agent.think settings.
>
> Deepgram retried the LLM request three times and gave up. The `Warning` messages printed just before this error show the real cause, such as one of the two errors above.

> [accordion] CLIENT\_MESSAGE\_TIMEOUT: We did not receive audio within our timeout
>
> Deepgram closes a session that receives no audio for a while. This error can appear when the model is slow to answer. In testing, it appeared in runs where the first text token took about 4 seconds or longer. The script prints the error, stops, and still saves `reply.wav`. If you keep a text-only session open longer, expect this error as well.

## Additional resources

* [Deepgram Voice Agent API](https://developers.deepgram.com/docs/voice-agent)
* [Deepgram LLM models and custom endpoints](https://developers.deepgram.com/docs/voice-agent-llm-models)
* [Deepgram Voice Agent latency report](https://developers.deepgram.com/docs/voice-agent-latency-report)
* [Supported SambaCloud models](https://docs.sambanova.ai/docs/en/models/sambacloud-models.md)
