Build with Braintrust - SambaNova Integration Guide

Braintrust is an evaluation and observability platform for LLM applications. It logs every model call, scores it against a metric, and compares experiments over time. SambaNova connects two ways: register it as a custom OpenAI-compatible provider in Braintrust’s AI providers settings for the Playground and gateway, and call it directly through the standard OpenAI Python client for programmatic evaluations. This guide covers both.

Prerequisites

A SambaCloud account and API key
A Braintrust account
Python 3.10 or later

Setup

Install the Braintrust SDK and the OpenAI client

pip install braintrust openai

Export your API keys

export SAMBANOVA_API_KEY="your-sambanova-api-key"
export BRAINTRUST_API_KEY="your-braintrust-api-key"

Generate a Braintrust API key from any AI providers or API keys page in Settings — the same “Generate API key” control works everywhere in the app.

In Braintrust, go to Settings → AI providers → Organization provider, then under Custom providers click +. Configure:

Provider type: OpenAI
Name: SambaNova
API key: your SambaCloud key
Under More settings → API base URL: https://api.sambanova.ai/v1

Braintrust AI providers settings page showing an Organization AI provider named SambaNova, type Custom, status Saved, with a masked API key.

The in-dialog Test button calls the provider directly from your browser and fails with TypeError: Failed to fetch against SambaNova’s endpoint — this is a browser CORS restriction on the test call, not a sign the configuration is wrong. Click Create anyway; the provider works (see below).

The provider’s Models → Available models list only shows Braintrust’s static catalog for the “OpenAI” provider type — it is not a live fetch from your configured endpoint, so searching it for MiniMax or any other SambaNova model name returns no matches. This is cosmetic, but it means SambaNova models don’t show up anywhere in the UI (Playground, gateway) until you add them explicitly.In the same provider’s Setup tab, scroll to Models, select All models or None under Registry models, then click + Custom model and enter the exact model ID (e.g. MiniMax-M3). Repeat for every model you plan to use.

This step is what makes SambaNova models usable anywhere in Braintrust’s UI — the Playground model picker and the gateway endpoint below both read from this list, not from the static catalog.

Example: use it in the Playground

Once registered as a custom model (previous step), SambaNova models appear in the Playground’s model picker under a SambaNova group, next to Braintrust’s own named providers:

Braintrust Playground with MiniMax-M3 selected as the model under the SambaNova provider group, showing a completed response listing the model's capabilities.

Example: call it through the gateway

Braintrust’s gateway (https://gateway.braintrust.dev) proxies requests to any registered custom provider and model, authenticated with your Braintrust API key rather than the provider’s own key:

curl -i -X POST 'https://gateway.braintrust.dev/chat/completions' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BRAINTRUST_API_KEY" \
  -H "x-bt-org-name: Your Org Name" \
  -H "x-bt-endpoint-name: SambaNova" \
  -d '{"model":"MiniMax-M3","messages":[{"role":"user","content":"What can you do?"}]}'

Output (abridged)

HTTP/2 200
x-bt-used-endpoint: SambaNova

{"choices":[{"message":{"content":"I'm a versatile AI assistant...","role":"assistant"}}], "model":"MiniMax-M3", ...}

This only works for a model you’ve explicitly added as a custom model on the provider (previous section). Calling it with a model ID that’s only in the static catalog, or before adding it, returns 404 with Braintrust gateway error: no provider named '...' configured ... Configured providers: [] — which looks like the provider itself is missing, but actually means the requested model isn’t registered on it yet.

Example: run a scored evaluation

This is the path for programmatic evals: a real evaluation, scored, with results in the dashboard. Point a standard OpenAI client at SambaNova and wrap it with braintrust.wrap_openai so Braintrust captures full call tracing (tokens, duration, call count) alongside your scores.

eval_sambanova.py

import os
from openai import OpenAI
from braintrust import Eval, wrap_openai

client = wrap_openai(
    OpenAI(
        base_url="https://api.sambanova.ai/v1",
        api_key=os.environ["SAMBANOVA_API_KEY"],
    )
)

def task(input):
    response = client.chat.completions.create(
        model="MiniMax-M3",
        messages=[{"role": "user", "content": input}],
    )
    return response.choices[0].message.content

def exact_match(input, output, expected):
    return 1.0 if expected.lower() in output.lower() else 0.0

Eval(
    "SambaNova Docs Verification",
    data=lambda: [
        {"input": "What is the capital of France?", "expected": "Paris"},
        {"input": "What is the capital of Japan?", "expected": "Tokyo"},
    ],
    task=task,
    scores=[exact_match],
)

Run it:

braintrust eval eval_sambanova.py

Output (abridged)

Experiment your-name-1792348392 is running at https://www.braintrust.dev/app/.../experiments/...

=========================SUMMARY=========================
100.00% 'exact_match' score

1 'llm_calls'
134tok 'prompt_tokens'
46.50tok 'completion_tokens'
180.50tok 'total_tokens'
7.00s 'duration'

Braintrust experiment results table with two rows, each scoring 100% on the exact_match column, with Tokyo and Paris as the correct outputs.

wrap_openai is what makes the difference here. The identical eval run without it still completes and scores correctly, but the summary reports 0 llm_calls and 0tok across every token metric — Braintrust has no visibility into the underlying model call unless the client is wrapped. If your experiment dashboard shows zero LLM calls despite a passing eval, this is why. Note this path doesn’t require the custom model registration from Setup step 4 — a plain OpenAI client works with any model string, registered or not, since it talks to SambaCloud directly rather than through Braintrust’s gateway or UI.

Additional resources

Braintrust documentation

Official guides covering evals, logging, datasets, and the Playground.

Supported models

Current list of models available on SambaCloud.

Troubleshooting

Gateway or Playground: model not found, or 'Configured providers: []'

The model hasn’t been added as a custom model on the provider yet. Go to Settings → AI providers → SambaNova → Setup → Models and click + Custom model to add the exact model ID. The static catalog search in Models → Available models will never show a SambaNova model, even after this step — that list is unrelated to which models actually work.

Custom provider Test button fails with TypeError: Failed to fetch

This is a browser CORS restriction on the in-dialog test call against SambaNova’s endpoint, not a configuration error. Click Create and verify with the Playground or a real evaluation instead.

Eval runs but shows 0 llm_calls and 0tok in every metric

The OpenAI client wasn’t wrapped. Use braintrust.wrap_openai(OpenAI(...)) instead of a plain OpenAI(...) instance — see Run a scored evaluation.

Failed to import eval_sambanova.py: 'SAMBANOVA_API_KEY'

SAMBANOVA_API_KEY isn’t set in the shell running braintrust eval. Export it: export SAMBANOVA_API_KEY="your-key".

Overview

Developer tools

Frameworks

Interfaces and low-code

Data and infrastructure

Multimodal

Evaluation and monitoring

Braintrust integration guide

Prerequisites

Setup

Example: use it in the Playground

Example: call it through the gateway

Example: run a scored evaluation

Additional resources

Braintrust documentation

Supported models

Troubleshooting

​Prerequisites

​Setup

​Example: use it in the Playground

​Example: call it through the gateway

​Example: run a scored evaluation

​Additional resources

Braintrust documentation

Supported models

​Troubleshooting

Prerequisites

Setup

Example: use it in the Playground

Example: call it through the gateway

Example: run a scored evaluation

Additional resources

Troubleshooting