Inspect AI integration guide

​

Prerequisites

1. A SambaCloud account and an active API key at SambaCloud API Keys.
2. To use InspectAI with SambaNova, set your SambaNova API key as an environment variable:

export SAMBANOVA_API_KEY=your-sambacloud-api-key

3. Python environment with required packages installed.

python3 -m venv .venv
source .venv/bin/activate
pip install inspect-ai
pip install openai

​

Running evaluations

1. Dataset – the list of inputs and expected results
2. Solver – how the model produces its outputs
3. Scorer – how outputs are evaluated against the expected results

​

Example: Hello world

from inspect_ai import Task, task
from inspect_ai.dataset import Sample
from inspect_ai.scorer import exact
from inspect_ai.solver import generate

@task
def hello_world():
    return Task(
        dataset=[
            Sample(
                input="Just reply with Hello World",
                target="Hello World",
            )
        ],
        solver=[generate()],
        scorer=exact(),
    )

inspect eval hello_world.py --model sambanova/llama-4-maverick-17b-128e-instruct

​

Viewing results

• Results are stored in the ./logs directory.
• Use the Inspect web UI for interactive viewing

inspect view

• You can also use the Inspect Visual Studio Code extension for easier log exploration.

​

Additional resources

• See the InspectAI repo for more evaluation examples.
• For more details, see the official InspectAI documentation.