Inspect AI integration guide

​

Prerequisites

• A SambaCloud account
• A SambaNova API key
• Python 3.8 or later

​

Installation

python3 -m venv .venv
source .venv/bin/activate

pip install inspect-ai
pip install openai

export SAMBANOVA_API_KEY=your-sambacloud-api-key

​

Running evaluations

1. Dataset – the list of inputs and expected results
2. Solver – how the model produces its outputs
3. Scorer – how outputs are evaluated against the expected results

​

Example: Hello world

from inspect_ai import Task, task
from inspect_ai.dataset import Sample
from inspect_ai.scorer import exact
from inspect_ai.solver import generate

@task
def hello_world():
    return Task(
        dataset=[
            Sample(
                input="Just reply with Hello World",
                target="Hello World",
            )
        ],
        solver=[generate()],
        scorer=exact(),
    )

inspect eval hello_world.py --model sambanova/Llama-4-Maverick-17B-128E-Instruct

​

Viewing results

• Results are stored in the logs directory.
• Use the Inspect web UI for interactive viewing.

inspect view

• You can also use the Inspect Visual Studio Code extension for easier log exploration.

​

Additional resources

• See the Inspect AI examples for more evaluation examples.
• For more details, see the official Inspect AI documentation.

​

Troubleshooting