> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# DSPy integration guide

> [!NOTE]
>
> **What you'll build:** A DSPy program running on a SambaCloud-hosted model, with verified examples for a basic prediction, tool-calling agent, structured output, and a prompt optimizer run.

## Overview

[DSPy](https://dspy.ai/) is a framework from Stanford NLP for programming, rather than prompting, language models. It compiles declarative signatures into prompts and can automatically optimize those prompts against a metric and a training set. Wire it to SambaCloud and you get:

> **No framework changes**
>
> `dspy.LM` accepts an `openai/`-prefixed model name plus `api_base` and `api_key`, so pointing it at SambaCloud is a configuration change, not a code change.

> **Tool-calling agents**
>
> `dspy.ReAct` wraps plain Python functions as tools and runs a real reasoning-and-acting loop against your model.

> **Structured output**
>
> Declare a Pydantic model as a signature's output field and get a validated object back.

> **Prompt optimization**
>
> Optimizers like `BootstrapFewShot` run real inference against SambaCloud to compile better prompts from a handful of labeled examples, which fits an inference-heavy usage profile well.

## Prerequisites

* A [SambaCloud account](https://cloud.sambanova.ai/?utm_source=dspy\&utm_medium=external\&utm_campaign=cloud_signup) and API key from the [SambaCloud portal](https://cloud.sambanova.ai/apis?utm_source=dspy\&utm_medium=external\&utm_campaign=cloud_signup)
* Python 3.10 or later
* Familiarity with Python virtual environments

## Setup

**1. Create and activate a virtual environment**

```bash macOS / Linux theme={}
python3 -m venv .venv
source .venv/bin/activate
```

```powershell Windows theme={}
python -m venv .venv
.venv\Scripts\Activate.ps1
```

**2. Install DSPy**

```bash theme={}
pip install dspy
```

**3. Export your SambaCloud API key**

```bash theme={}
export SAMBANOVA_API_KEY="your-sambacloud-api-key"
```

> [!WARNING]
>
> Never commit your API key to source control. Use a `.env` file or your shell's secret manager.

## Example: basic prediction

`dspy.LM` routes through LiteLLM under the hood. Prefix the model name with `openai/` and pass `api_base`/`api_key` to route to SambaCloud:

```python basic_example.py theme={}
import os
import dspy

lm = dspy.LM(
    "openai/MiniMax-M3",
    api_base="https://api.sambanova.ai/v1",
    api_key=os.environ["SAMBANOVA_API_KEY"],
)
dspy.configure(lm=lm)

class QA(dspy.Signature):
    """Answer the question."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

predict = dspy.Predict(QA)
result = predict(question="What is the capital of Japan?")
print(result.answer)
```

Run it:

```bash theme={}
python basic_example.py
```

```text Output theme={}
Tokyo
```

> [!NOTE]
>
> Swap `"MiniMax-M3"` for any model on the [SambaCloud supported models](/docs/en/models/sambacloud-models.md) page, keeping the `openai/` prefix.

## Example: tool calling with ReAct

`dspy.ReAct` turns plain Python functions into tools and runs a reasoning-and-acting loop: the model decides which tool to call, DSPy runs it, and feeds the result back until the model finishes.

```python react_example.py theme={}
import os
import dspy

lm = dspy.LM(
    "openai/MiniMax-M3",
    api_base="https://api.sambanova.ai/v1",
    api_key=os.environ["SAMBANOVA_API_KEY"],
)
dspy.configure(lm=lm)

def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"It is sunny and 22C in {city}."

agent = dspy.ReAct("question -> answer", tools=[get_weather])
result = agent(question="What's the weather like in Tokyo right now?")
print(result.answer)
```

Run it:

```bash theme={}
python react_example.py
```

```text Output theme={}
The current weather in Tokyo is sunny with a temperature of 22°C (about 72°F). It sounds like a pleasant day!
```

> **Inspect the full trajectory (result.trajectory)**
>
> `ReAct` records every thought, tool call, and observation in `result.trajectory`, which confirms the model actually called `get_weather` rather than guessing a plausible answer:
>
> ```text theme={}
> {
>     'thought_0': 'The user is asking about the current weather in Tokyo. I should use the get_weather tool to fetch this information.',
>     'tool_name_0': 'get_weather',
>     'tool_args_0': {'city': 'Tokyo'},
>     'observation_0': 'It is sunny and 22C in Tokyo.',
>     'thought_1': "I have the weather information for Tokyo. It's sunny and 22C. I can now finish the task and provide the answer to the user.",
>     'tool_name_1': 'finish',
>     'tool_args_1': {},
>     'observation_1': 'Completed.',
> }
> ```

## Example: structured output

Declare a Pydantic model as a signature's output field type. DSPy does not use the API's `response_format` parameter for this: it writes the JSON schema directly into the prompt and parses the model's response itself, so this works even on models without native structured-output support.

```python structured_example.py theme={}
import os
import dspy
from pydantic import BaseModel

lm = dspy.LM(
    "openai/MiniMax-M3",
    api_base="https://api.sambanova.ai/v1",
    api_key=os.environ["SAMBANOVA_API_KEY"],
)
dspy.configure(lm=lm)

class CityInfo(BaseModel):
    city: str
    country: str

class Extract(dspy.Signature):
    """Extract structured location info from text."""
    text: str = dspy.InputField()
    info: CityInfo = dspy.OutputField()

extractor = dspy.Predict(Extract)
result = extractor(text="Paris is the capital of France.")
print(result.info)
```

Run it:

```bash theme={}
python structured_example.py
```

```text Output theme={}
city='Paris' country='France'
```

> **What DSPy actually sends to SambaCloud**
>
> Inspecting `lm.history[-1]["messages"]` shows the real request: the JSON schema is embedded as text in the system prompt, not sent as a `response_format` parameter.
>
> ```text System message (excerpt) theme={}
> Your input fields are:
> 1. `text` (str):
> Your output fields are:
> 1. `info` (CityInfo):
> ...
> [[ ## info ## ]]
> {info}        # note: the value you produce must adhere to the JSON schema: {"type": "object", "properties": {"city": {"type": "string", "title": "City"}, "country": {"type": "string", "title": "Country"}}, "required": ["city", "country"], "title": "CityInfo"}
> ```
>
> DSPy then parses the model's formatted text response and validates it against the Pydantic model, which is what turns prose output into the `CityInfo(...)` object `result.info` returns.

## Example: optimize a program

Optimizers compile a program by running real inference against your training set and keeping the examples that improve a metric. `BootstrapFewShot` is the simplest one:

```python optimize_example.py theme={}
import os
import dspy

lm = dspy.LM(
    "openai/MiniMax-M3",
    api_base="https://api.sambanova.ai/v1",
    api_key=os.environ["SAMBANOVA_API_KEY"],
)
dspy.configure(lm=lm)

class QA(dspy.Signature):
    """Answer the question with a single word."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

trainset = [
    dspy.Example(question="What is the capital of France?", answer="Paris").with_inputs("question"),
    dspy.Example(question="What is the capital of Japan?", answer="Tokyo").with_inputs("question"),
    dspy.Example(question="What is the capital of Italy?", answer="Rome").with_inputs("question"),
]

def exact_match(example, pred, trace=None):
    return example.answer.lower() in pred.answer.lower()

program = dspy.Predict(QA)
optimizer = dspy.BootstrapFewShot(metric=exact_match, max_bootstrapped_demos=2)
optimized = optimizer.compile(program, trainset=trainset)

result = optimized(question="What is the capital of Germany?")
print(result.answer)
print(f"Demos in compiled program: {len(optimized.demos)}")
```

Run it:

```bash theme={}
python optimize_example.py
```

```text Output theme={}
Bootstrapped 2 full traces after 2 examples for up to 1 rounds, amounting to 2 attempts.
Berlin
Demos in compiled program: 3
```

> [!NOTE]
>
> Each optimizer run makes one real inference call against SambaCloud per bootstrap attempt, on top of the calls your program makes when evaluated. This usage profile, inference-heavy by design, is the main reason DSPy's cost scales with trainset size and optimizer rounds, not just with how often the compiled program is later called.

## Additional resources

> **[DSPy repository](https://github.com/stanfordnlp/dspy)**
>
> Source code, examples, and release notes for DSPy.

> **[DSPy documentation](https://dspy.ai/)**
>
> Official guides covering signatures, modules, optimizers, and evaluation.

> **[Supported models](/docs/en/models/sambacloud-models.md)**
>
> Current list of models available on SambaCloud.

> **[Function calling and JSON mode](/docs/en/features/function-calling.md)**
>
> Which SambaCloud models support tool calling and structured output natively.

## Troubleshooting

> **ModuleNotFoundError: No module named 'dspy'**
>
> Install with `pip install dspy`, then confirm your virtual environment is active: `which python3` should point inside `.venv`.

> **KeyError: SAMBANOVA\_API\_KEY**
>
> Export the key before running: `export SAMBANOVA_API_KEY="your-key"`. Verify with `echo $SAMBANOVA_API_KEY`.

> **LMInvalidRequestError: Unmapped LLM provider for this endpoint**
>
> The model name is missing the `openai/` prefix. DSPy routes through LiteLLM, which uses the prefix to pick the request format. Use `"openai/MiniMax-M3"`, not `"MiniMax-M3"`.

> **LMAuthError: Incorrect API key provided**
>
> The key in `SAMBANOVA_API_KEY` is wrong or expired. Generate a new one in the [SambaCloud portal](https://cloud.sambanova.ai/apis?utm_source=dspy\&utm_medium=external\&utm_campaign=cloud_signup) and export it again.

> **ReAct agent answers without calling the tool**
>
> `dspy.ReAct` doesn't use the API's native function calling. It describes your tools in the prompt and parses the model's reply to decide which tool to run. If the model guesses an answer instead, check `result.trajectory` for a `tool_name_0` entry, then run `dspy.inspect_history(n=1)` to see the exact prompt and reply. Give the tool a clear docstring and type hints, since that text is all the model sees.

> **AdapterParseError: Adapter ChatAdapter failed to parse the LM response**
>
> The model's reply didn't follow the field format DSPy expected. Run `dspy.inspect_history(n=1)` to see the raw response. Retrying usually succeeds; if it fails consistently, try another model from the [supported models](/docs/en/models/sambacloud-models.md) page.