> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# DeepEval integration guide

[DeepEval](https://deepeval.com/) is an open-source LLM evaluation framework that plugs into `pytest`, so judge-model calls run as part of your normal test suite and CI pipeline.

## Prerequisites

* A [SambaCloud](https://cloud.sambanova.ai/?utm_source=deepeval\&utm_medium=external\&utm_campaign=cloud_signup) account and [API key](https://cloud.sambanova.ai/apis?utm_source=deepeval\&utm_medium=external\&utm_campaign=cloud_signup)
* Python 3.9 or later

## Setup

**1. Install DeepEval**

```bash theme={}
pip install deepeval pytest
```

**2. Export your SambaCloud API key**

```bash theme={}
export SAMBANOVA_API_KEY="your-sambanova-api-key"
```

**3. Point DeepEval's judge model at SambaCloud**

Use DeepEval's `OpenAIModel` class with a custom `base_url` — this is the same class DeepEval uses for real OpenAI, just redirected:

```python theme={}
import os
from deepeval.models import OpenAIModel

model = OpenAIModel(
    model="MiniMax-M3",
    api_key=os.environ["SAMBANOVA_API_KEY"],
    base_url="https://api.sambanova.ai/v1",
)
```

> [!NOTE]
>
> See [DeepEval's OpenAI model reference](https://deepeval.com/integrations/models/openai) for the full parameter list. The older `GPTModel` name works identically but is deprecated in favor of `OpenAIModel` as of DeepEval 4.x.

## Example: score a test case

```python test_sambanova.py theme={}
import os
from deepeval.models import OpenAIModel
from deepeval.test_case import LLMTestCase
from deepeval.metrics import AnswerRelevancyMetric

model = OpenAIModel(
    model="MiniMax-M3",
    api_key=os.environ["SAMBANOVA_API_KEY"],
    base_url="https://api.sambanova.ai/v1",
)

test_case = LLMTestCase(
    input="What is the capital of France?",
    actual_output="The capital of France is Paris.",
)

metric = AnswerRelevancyMetric(model=model, threshold=0.5)
metric.measure(test_case)
print(metric.score)
print(metric.reason)
```

```text Output theme={}
1.0
The score is 1.00 because the response directly and accurately addresses the question about the capital of France with no irrelevant information. Excellent work!
```

The `reason` string is generated by the judge model, so it explains the score rather than restating it.

## Example: run as a pytest CI check

This is DeepEval's main use case: judge-model calls inside your normal test suite.

```python test_ci_example.py theme={}
import os
from deepeval import assert_test
from deepeval.models import OpenAIModel
from deepeval.test_case import LLMTestCase
from deepeval.metrics import AnswerRelevancyMetric

model = OpenAIModel(
    model="MiniMax-M3",
    api_key=os.environ["SAMBANOVA_API_KEY"],
    base_url="https://api.sambanova.ai/v1",
)

def test_answer_relevancy():
    test_case = LLMTestCase(
        input="What is the capital of France?",
        actual_output="The capital of France is Paris.",
    )
    metric = AnswerRelevancyMetric(model=model, threshold=0.5)
    assert_test(test_case, [metric])
```

Run it with DeepEval's own CLI (its documented command, backed by `pytest` under the hood):

```bash theme={}
deepeval test run test_ci_example.py
```

```text Output (abridged) theme={}
✓ Evaluation completed 🎉! (time taken: 5.44s | token cost: None)
» Test Results (1 total tests):
   » Pass Rate: 100.0% | Passed: 1 | Failed: 0
```

> [!NOTE]
>
> Plain `pytest test_ci_example.py -v` also works, since DeepEval is a `pytest` plugin, but `deepeval test run` is the documented command and adds the richer test-result table shown above.

> [!TIP]
>
> DeepEval itself needs no login or API key to run evaluations locally — `deepeval login` is optional and only needed for Confident AI's cloud dashboard. The only credential required for the examples on this page is your SambaCloud API key, passed to `OpenAIModel`.

## Troubleshooting

> **DeprecationWarning: 'GPTModel' is deprecated**
>
> Use `OpenAIModel` instead — same constructor, same `base_url` support, just the current name as of DeepEval 4.x.

> **KeyError: SAMBANOVA\_API\_KEY**
>
> Export the key before running: `export SAMBANOVA_API_KEY="your-key"`. Verify with `echo $SAMBANOVA_API_KEY`.

> **Low or unexpected AnswerRelevancyMetric scores**
>
> Check `metric.reason` — it's generated by the judge model itself and explains exactly what it penalized, which is more useful than the score alone for figuring out whether the issue is your prompt, your test case, or the judge model's interpretation.

## Additional resources

* [DeepEval getting started](https://deepeval.com/docs/getting-started)
* [DeepEval OpenAI model reference](https://deepeval.com/integrations/models/openai)
* [Supported SambaCloud models](/docs/en/models/sambacloud-models)