Build with DSPy - SambaNova Integration Guide

What you’ll build: A DSPy program running on a SambaCloud-hosted model, with verified examples for a basic prediction, tool-calling agent, structured output, and a prompt optimizer run.

Overview

DSPy is a framework from Stanford NLP for programming, rather than prompting, language models. It compiles declarative signatures into prompts and can automatically optimize those prompts against a metric and a training set. Wire it to SambaCloud and you get:

No framework changes

dspy.LM accepts an openai/-prefixed model name plus api_base and api_key, so pointing it at SambaCloud is a configuration change, not a code change.

Tool-calling agents

dspy.ReAct wraps plain Python functions as tools and runs a real reasoning-and-acting loop against your model.

Structured output

Declare a Pydantic model as a signature’s output field and get a validated object back.

Prompt optimization

Optimizers like BootstrapFewShot run real inference against SambaCloud to compile better prompts from a handful of labeled examples, which fits an inference-heavy usage profile well.

Prerequisites

A SambaCloud account and API key from the SambaCloud portal
Python 3.10 or later
Familiarity with Python virtual environments

Setup

Create and activate a virtual environment

python3 -m venv .venv
source .venv/bin/activate

python -m venv .venv
.venv\Scripts\Activate.ps1

Install DSPy

pip install dspy

Export your SambaCloud API key

export SAMBANOVA_API_KEY="your-sambacloud-api-key"

Never commit your API key to source control. Use a .env file or your shell’s secret manager.

Example: basic prediction

dspy.LM routes through LiteLLM under the hood. Prefix the model name with openai/ and pass api_base/api_key to route to SambaCloud:

basic_example.py

import os
import dspy

lm = dspy.LM(
    "openai/MiniMax-M3",
    api_base="https://api.sambanova.ai/v1",
    api_key=os.environ["SAMBANOVA_API_KEY"],
)
dspy.configure(lm=lm)

class QA(dspy.Signature):
    """Answer the question."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

predict = dspy.Predict(QA)
result = predict(question="What is the capital of Japan?")
print(result.answer)

Run it:

python basic_example.py

Output

Tokyo

Swap "MiniMax-M3" for any model on the SambaCloud supported models page, keeping the openai/ prefix.

Example: tool calling with ReAct

dspy.ReAct turns plain Python functions into tools and runs a reasoning-and-acting loop: the model decides which tool to call, DSPy runs it, and feeds the result back until the model finishes.

react_example.py

import os
import dspy

lm = dspy.LM(
    "openai/MiniMax-M3",
    api_base="https://api.sambanova.ai/v1",
    api_key=os.environ["SAMBANOVA_API_KEY"],
)
dspy.configure(lm=lm)

def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"It is sunny and 22C in {city}."

agent = dspy.ReAct("question -> answer", tools=[get_weather])
result = agent(question="What's the weather like in Tokyo right now?")
print(result.answer)

Run it:

python react_example.py

Output

The current weather in Tokyo is sunny with a temperature of 22°C (about 72°F). It sounds like a pleasant day!

Inspect the full trajectory (result.trajectory)

ReAct records every thought, tool call, and observation in result.trajectory, which confirms the model actually called get_weather rather than guessing a plausible answer:

{
    'thought_0': 'The user is asking about the current weather in Tokyo. I should use the get_weather tool to fetch this information.',
    'tool_name_0': 'get_weather',
    'tool_args_0': {'city': 'Tokyo'},
    'observation_0': 'It is sunny and 22C in Tokyo.',
    'thought_1': "I have the weather information for Tokyo. It's sunny and 22C. I can now finish the task and provide the answer to the user.",
    'tool_name_1': 'finish',
    'tool_args_1': {},
    'observation_1': 'Completed.',
}

Example: structured output

Declare a Pydantic model as a signature’s output field type. DSPy does not use the API’s response_format parameter for this: it writes the JSON schema directly into the prompt and parses the model’s response itself, so this works even on models without native structured-output support.

structured_example.py

import os
import dspy
from pydantic import BaseModel

lm = dspy.LM(
    "openai/MiniMax-M3",
    api_base="https://api.sambanova.ai/v1",
    api_key=os.environ["SAMBANOVA_API_KEY"],
)
dspy.configure(lm=lm)

class CityInfo(BaseModel):
    city: str
    country: str

class Extract(dspy.Signature):
    """Extract structured location info from text."""
    text: str = dspy.InputField()
    info: CityInfo = dspy.OutputField()

extractor = dspy.Predict(Extract)
result = extractor(text="Paris is the capital of France.")
print(result.info)

Run it:

python structured_example.py

Output

city='Paris' country='France'

What DSPy actually sends to SambaCloud

Inspecting lm.history[-1]["messages"] shows the real request: the JSON schema is embedded as text in the system prompt, not sent as a response_format parameter.

System message (excerpt)

Your input fields are:
1. `text` (str):
Your output fields are:
1. `info` (CityInfo):
...
[[ ## info ## ]]
{info}        # note: the value you produce must adhere to the JSON schema: {"type": "object", "properties": {"city": {"type": "string", "title": "City"}, "country": {"type": "string", "title": "Country"}}, "required": ["city", "country"], "title": "CityInfo"}

DSPy then parses the model’s formatted text response and validates it against the Pydantic model, which is what turns prose output into the CityInfo(...) object result.info returns.

Example: optimize a program

Optimizers compile a program by running real inference against your training set and keeping the examples that improve a metric. BootstrapFewShot is the simplest one:

optimize_example.py

import os
import dspy

lm = dspy.LM(
    "openai/MiniMax-M3",
    api_base="https://api.sambanova.ai/v1",
    api_key=os.environ["SAMBANOVA_API_KEY"],
)
dspy.configure(lm=lm)

class QA(dspy.Signature):
    """Answer the question with a single word."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

trainset = [
    dspy.Example(question="What is the capital of France?", answer="Paris").with_inputs("question"),
    dspy.Example(question="What is the capital of Japan?", answer="Tokyo").with_inputs("question"),
    dspy.Example(question="What is the capital of Italy?", answer="Rome").with_inputs("question"),
]

def exact_match(example, pred, trace=None):
    return example.answer.lower() in pred.answer.lower()

program = dspy.Predict(QA)
optimizer = dspy.BootstrapFewShot(metric=exact_match, max_bootstrapped_demos=2)
optimized = optimizer.compile(program, trainset=trainset)

result = optimized(question="What is the capital of Germany?")
print(result.answer)
print(f"Demos in compiled program: {len(optimized.demos)}")

Run it:

python optimize_example.py

Output

Bootstrapped 2 full traces after 2 examples for up to 1 rounds, amounting to 2 attempts.
Berlin
Demos in compiled program: 3

Each optimizer run makes one real inference call against SambaCloud per bootstrap attempt, on top of the calls your program makes when evaluated. This usage profile, inference-heavy by design, is the main reason DSPy’s cost scales with trainset size and optimizer rounds, not just with how often the compiled program is later called.

Additional resources

DSPy repository

Source code, examples, and release notes for DSPy.

DSPy documentation

Official guides covering signatures, modules, optimizers, and evaluation.

Supported models

Current list of models available on SambaCloud.

Function calling and JSON mode

Which SambaCloud models support tool calling and structured output natively.

Troubleshooting

ModuleNotFoundError: No module named 'dspy'

Install with pip install dspy, then confirm your virtual environment is active: which python3 should point inside .venv.

KeyError: SAMBANOVA_API_KEY

Export the key before running: export SAMBANOVA_API_KEY="your-key". Verify with echo $SAMBANOVA_API_KEY.

LMInvalidRequestError: Unmapped LLM provider for this endpoint

The model name is missing the openai/ prefix. DSPy routes through LiteLLM, which uses the prefix to pick the request format. Use "openai/MiniMax-M3", not "MiniMax-M3".

LMAuthError: Incorrect API key provided

The key in SAMBANOVA_API_KEY is wrong or expired. Generate a new one in the SambaCloud portal and export it again.

ReAct agent answers without calling the tool

dspy.ReAct doesn’t use the API’s native function calling. It describes your tools in the prompt and parses the model’s reply to decide which tool to run. If the model guesses an answer instead, check result.trajectory for a tool_name_0 entry, then run dspy.inspect_history(n=1) to see the exact prompt and reply. Give the tool a clear docstring and type hints, since that text is all the model sees.

AdapterParseError: Adapter ChatAdapter failed to parse the LM response

The model’s reply didn’t follow the field format DSPy expected. Run dspy.inspect_history(n=1) to see the raw response. Retrying usually succeeds; if it fails consistently, try another model from the supported models page.

Overview

Developer tools

Frameworks

Interfaces and low-code

Data and infrastructure

Multimodal

Evaluation and monitoring

DSPy integration guide

Overview

No framework changes

Tool-calling agents

Structured output

Prompt optimization

Prerequisites

Setup

Example: basic prediction

Example: tool calling with ReAct

Example: structured output

Example: optimize a program

Additional resources

DSPy repository

DSPy documentation

Supported models

Function calling and JSON mode

Troubleshooting

​Overview

No framework changes

Tool-calling agents

Structured output

Prompt optimization

​Prerequisites

​Setup

​Example: basic prediction

​Example: tool calling with ReAct

​Example: structured output

​Example: optimize a program

​Additional resources

DSPy repository

DSPy documentation

Supported models

Function calling and JSON mode

​Troubleshooting

Overview

Prerequisites

Setup

Example: basic prediction

Example: tool calling with ReAct

Example: structured output

Example: optimize a program

Additional resources

Troubleshooting