> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Braintrust integration guide

[Braintrust](https://www.braintrust.dev/) is an evaluation and observability platform for LLM applications. It logs every model call, scores it against a metric, and compares experiments over time.

SambaNova connects two ways: register it as a custom OpenAI-compatible provider in Braintrust's **AI providers** settings for the Playground and gateway, and call it directly through the standard OpenAI Python client for programmatic evaluations. This guide covers both.

## Prerequisites

* A [SambaCloud](https://cloud.sambanova.ai/?utm_source=braintrust\&utm_medium=external\&utm_campaign=cloud_signup) account and API key
* A [Braintrust](https://www.braintrust.dev/signup) account
* Python 3.10 or later

## Setup

**1. Install the Braintrust SDK and the OpenAI client**

```bash theme={}
pip install braintrust openai
```

**2. Export your API keys**

```bash theme={}
export SAMBANOVA_API_KEY="your-sambanova-api-key"
export BRAINTRUST_API_KEY="your-braintrust-api-key"
```

> [!TIP]
>
> Generate a Braintrust API key from any **AI providers** or **API keys** page in Settings — the same "Generate API key" control works everywhere in the app.

**3. Register SambaNova as a custom AI provider**

In Braintrust, go to **Settings → AI providers → Organization provider**, then under **Custom providers** click **+**. Configure:

* **Provider type**: OpenAI
* **Name**: `SambaNova`
* **API key**: your SambaCloud key
* Under **More settings → API base URL**: `https://api.sambanova.ai/v1`

![Braintrust AI providers settings page showing an Organization AI provider named SambaNova, type Custom, status Saved, with a masked API key.](https://mintcdn.com/sambanova-systems/HObY37FkTNsi58im/images/docs/integrations/braintrust/braintrust-custom-provider-configured.png?fit=max\&auto=format\&n=HObY37FkTNsi58im\&q=85\&s=f76c51af2d46c6e3749f50a9dad2ebfa)

> [!WARNING]
>
> The in-dialog **Test** button calls the provider directly from your browser and fails with `TypeError: Failed to fetch` against SambaNova's endpoint — this is a browser CORS restriction on the test call, not a sign the configuration is wrong. Click **Create** anyway; the provider works (see below).

**4. Register each SambaCloud model you want to use**

The provider's **Models → Available models** list only shows Braintrust's static catalog for the "OpenAI" provider type — it is not a live fetch from your configured endpoint, so searching it for `MiniMax` or any other SambaNova model name returns no matches. This is cosmetic, but it means SambaNova models don't show up anywhere in the UI (Playground, gateway) until you add them explicitly.

In the same provider's **Setup** tab, scroll to **Models**, select **All models** or **None** under Registry models, then click **+ Custom model** and enter the exact model ID (e.g. `MiniMax-M3`). Repeat for every model you plan to use.

> [!NOTE]
>
> This step is what makes SambaNova models usable anywhere in Braintrust's UI — the Playground model picker and the gateway endpoint below both read from this list, not from the static catalog.

## Example: use it in the Playground

Once registered as a custom model (previous step), SambaNova models appear in the Playground's model picker under a **SambaNova** group, next to Braintrust's own named providers:

![Braintrust Playground with MiniMax-M3 selected as the model under the SambaNova provider group, showing a completed response listing the model's capabilities.](https://mintcdn.com/sambanova-systems/HObY37FkTNsi58im/images/docs/integrations/braintrust/braintrust-playground-minimax-m3.png?fit=max\&auto=format\&n=HObY37FkTNsi58im\&q=85\&s=65ec5694ad7b773ea2262b2e5618f584)

## Example: call it through the gateway

Braintrust's gateway (`https://gateway.braintrust.dev`) proxies requests to any registered custom provider and model, authenticated with your Braintrust API key rather than the provider's own key:

```bash theme={}
curl -i -X POST 'https://gateway.braintrust.dev/chat/completions' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BRAINTRUST_API_KEY" \
  -H "x-bt-org-name: Your Org Name" \
  -H "x-bt-endpoint-name: SambaNova" \
  -d '{"model":"MiniMax-M3","messages":[{"role":"user","content":"What can you do?"}]}'
```

```text Output (abridged) theme={}
HTTP/2 200
x-bt-used-endpoint: SambaNova

{"choices":[{"message":{"content":"I'm a versatile AI assistant...","role":"assistant"}}], "model":"MiniMax-M3", ...}
```

> [!WARNING]
>
> This only works for a model you've explicitly added as a **custom model** on the provider (previous section). Calling it with a model ID that's only in the static catalog, or before adding it, returns `404` with `Braintrust gateway error: no provider named '...' configured ... Configured providers: []` — which looks like the provider itself is missing, but actually means the requested *model* isn't registered on it yet.

## Example: run a scored evaluation

This is the path for programmatic evals: a real evaluation, scored, with results in the dashboard. Point a standard `OpenAI` client at SambaNova and wrap it with `braintrust.wrap_openai` so Braintrust captures full call tracing (tokens, duration, call count) alongside your scores.

```python eval_sambanova.py theme={}
import os
from openai import OpenAI
from braintrust import Eval, wrap_openai

client = wrap_openai(
    OpenAI(
        base_url="https://api.sambanova.ai/v1",
        api_key=os.environ["SAMBANOVA_API_KEY"],
    )
)

def task(input):
    response = client.chat.completions.create(
        model="MiniMax-M3",
        messages=[{"role": "user", "content": input}],
    )
    return response.choices[0].message.content

def exact_match(input, output, expected):
    return 1.0 if expected.lower() in output.lower() else 0.0

Eval(
    "SambaNova Docs Verification",
    data=lambda: [
        {"input": "What is the capital of France?", "expected": "Paris"},
        {"input": "What is the capital of Japan?", "expected": "Tokyo"},
    ],
    task=task,
    scores=[exact_match],
)
```

Run it:

```bash theme={}
braintrust eval eval_sambanova.py
```

```text Output (abridged) theme={}
Experiment your-name-1792348392 is running at https://www.braintrust.dev/app/.../experiments/...

=========================SUMMARY=========================
100.00% 'exact_match' score

1 'llm_calls'
134tok 'prompt_tokens'
46.50tok 'completion_tokens'
180.50tok 'total_tokens'
7.00s 'duration'
```

![Braintrust experiment results table with two rows, each scoring 100% on the exact\_match column, with Tokyo and Paris as the correct outputs.](https://mintcdn.com/sambanova-systems/HObY37FkTNsi58im/images/docs/integrations/braintrust/braintrust-experiment-results.png?fit=max\&auto=format\&n=HObY37FkTNsi58im\&q=85\&s=149de99f11935fd56f52ada77dad8212)

> [!WARNING]
>
> `wrap_openai` is what makes the difference here. The identical eval run **without** it still completes and scores correctly, but the summary reports `0 llm_calls` and `0tok` across every token metric — Braintrust has no visibility into the underlying model call unless the client is wrapped. If your experiment dashboard shows zero LLM calls despite a passing eval, this is why. Note this path doesn't require the custom model registration from Setup step 4 — a plain `OpenAI` client works with any model string, registered or not, since it talks to SambaCloud directly rather than through Braintrust's gateway or UI.

## Additional resources

> **[Braintrust documentation](https://www.braintrust.dev/docs)**
>
> Official guides covering evals, logging, datasets, and the Playground.

> **[Supported models](/docs/en/models/sambacloud-models)**
>
> Current list of models available on SambaCloud.

## Troubleshooting

> **Gateway or Playground: model not found, or 'Configured providers: \[]'**
>
> The model hasn't been added as a custom model on the provider yet. Go to **Settings → AI providers → SambaNova → Setup → Models** and click **+ Custom model** to add the exact model ID. The static catalog search in **Models → Available models** will never show a SambaNova model, even after this step — that list is unrelated to which models actually work.

> **Custom provider Test button fails with TypeError: Failed to fetch**
>
> This is a browser CORS restriction on the in-dialog test call against SambaNova's endpoint, not a configuration error. Click **Create** and verify with the Playground or a real evaluation instead.

> **Eval runs but shows 0 llm\_calls and 0tok in every metric**
>
> The `OpenAI` client wasn't wrapped. Use `braintrust.wrap_openai(OpenAI(...))` instead of a plain `OpenAI(...)` instance — see [Run a scored evaluation](#example-run-a-scored-evaluation).

> **Failed to import eval\_sambanova.py: 'SAMBANOVA\_API\_KEY'**
>
> `SAMBANOVA_API_KEY` isn't set in the shell running `braintrust eval`. Export it: `export SAMBANOVA_API_KEY="your-key"`.

> **ValueError: Could not login to Braintrust**
>
> `BRAINTRUST_API_KEY` isn't set. Export it (`export BRAINTRUST_API_KEY="your-key"`) or add it to a `.env.braintrust` file. To run the eval without logging to Braintrust, add `--no-send-logs`.