> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Inspect AI integration guide

[InspectAI](https://inspect.aisi.org.uk/) is an evaluation framework created by the [UK AI Security Institute](https://aisi.gov.uk/). It can be used to run a wide range of evaluations that measure coding, reasoning, agentic tasks, knowledge, behavior, and multimodal understanding. With InspectAI, evaluations and benchmarking become simple, reproducible, and consistent across multiple models and providers.

## Prerequisites

Before you begin, ensure you have:

1. A [SambaCloud](http://cloud.sambanova.ai?utm_source=inspectai\&utm_medium=external\&utm_campaign=cloud_signup) account and an active API key at [SambaCloud API Keys](https://cloud.sambanova.ai/apis).
2. To use InspectAI with SambaNova, set your SambaNova API key as an environment variable:

```bash theme={null}
export SAMBANOVA_API_KEY=your-sambacloud-api-key`
```

3. Python environment with required packages installed.

```bash theme={null}
python3 -m venv .venv
source .venv/bin/activate
pip install inspect-ai
pip install openai
```

## Running evaluations

Before you can run your first evaluation, you’ll need to define a task in a Python script.

Each task has three main components:

1. Dataset – the list of inputs and expected results
2. Solver – how the model produces its outputs
3. Scorer – how outputs are evaluated against the expected results

### Example: Hello world

Save the following code into a `hello_world.py` file.

```python theme={null}
from inspect_ai import Task, task
from inspect_ai.dataset import Sample
from inspect_ai.scorer import exact
from inspect_ai.solver import generate

@task
def hello_world():
    return Task(
        dataset=[
            Sample(
                input="Just reply with Hello World",
                target="Hello World",
            )
        ],
        solver=[generate()],
        scorer=exact(),
    )
```

Then run the evaluation with SambaCloud. Here’s an example using the `Llama-4-Maverick-17B-128E-Instruct` model:

```bash theme={null}
inspect eval hello_world.py --model sambanova/llama-4-maverick-17b-128e-instruct
```

## Viewing results

* Results are stored in the ./logs directory.
* Use the Inspect web UI for interactive viewing

```bash theme={null}
inspect view
```

* You can also use the [Inspect Visual Studio Code extension](https://inspect.aisi.org.uk/log-viewer.html#vs-code-extension) for easier log exploration.

## Additional resources

* See the [InspecAI repo](https://github.com/UKGovernmentBEIS/inspect_ai/tree/main/build) for more evaluation examples.
* For more details, see the official [InspectAI documentation](https://inspect.aisi.org.uk/reference/).