> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Firecrawl integration guide

[Firecrawl](https://www.firecrawl.dev/) is a web scraping and crawling API that turns pages into clean markdown or structured data for LLM applications. Its JSON extraction feature sends the page to a language model and returns data that matches your schema. When you self-host Firecrawl, you choose that model with three environment variables, so a SambaCloud model can do the extraction.

> [!NOTE]
>
> This guide covers self-hosted Firecrawl. Firecrawl's self-hosting documentation lists connecting your own OpenAI-compatible provider as a self-hosting option, and this guide does not cover Firecrawl Cloud.

## Prerequisites

* A [SambaCloud](https://cloud.sambanova.ai/?utm_source=firecrawl\&utm_medium=external\&utm_campaign=cloud_signup) account and [API key](https://cloud.sambanova.ai/apis?utm_source=firecrawl\&utm_medium=external\&utm_campaign=cloud_signup)
* [Git](https://git-scm.com/) and [Docker](https://docs.docker.com/get-docker/) with Docker Compose
* Python 3.9 or higher

## Installation and setup

This guide was tested with Firecrawl `v2.11.0`. The first start builds the API, Playwright, and database images from source, so it takes longer than later starts.

**1. Clone Firecrawl at a release tag**

Use a release tag rather than `main`, so the setup does not change underneath you.

```bash theme={}
git clone --branch v2.11.0 --depth 1 https://github.com/firecrawl/firecrawl.git
cd firecrawl
```

**2. Point Firecrawl at SambaCloud**

Create a file named `.env` in the `firecrawl` folder, next to `docker-compose.yaml`:

```text .env theme={}
PORT=3002
USE_DB_AUTHENTICATION=false
OPENAI_API_KEY=your-sambanova-api-key
OPENAI_BASE_URL=https://api.sambanova.ai/v1
MODEL_NAME=MiniMax-M3.1-Flash
```

* `OPENAI_API_KEY` is your SambaCloud API key. The variable name is Firecrawl's, not OpenAI's.
* `OPENAI_BASE_URL` is the SambaCloud base URL.
* `MODEL_NAME` is a SambaCloud model ID. Firecrawl uses it for every extraction call, whatever model it would otherwise pick.

> [!WARNING]
>
> With `USE_DB_AUTHENTICATION=false` the Firecrawl API has no authentication. Keep port `3002` on your own machine or a trusted network, and keep the `.env` file out of version control because it holds your API key.

**3. Start Firecrawl**

```bash theme={}
docker compose up -d --build
```

Compose prints several `The "..." variable is not set. Defaulting to a blank string` warnings for settings this guide does not use. They are harmless. When the build finishes, check that the `api` service is running:

```bash theme={}
docker compose ps
```

**4. Install the Python client**

In a second terminal, create a virtual environment and install `requests`.

```bash macOS / Linux theme={}
python3 -m venv .venv
source .venv/bin/activate
pip install requests
```

```powershell Windows theme={}
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install requests
```

This guide was tested with `requests` 2.32.5 on Python 3.9.

## Extract structured data from a page

The `json` format takes a JSON schema and an optional prompt. This script asks Firecrawl for the first five books on a sample bookstore page:

```python extract.py theme={}
import json

import requests

payload = {
    "url": "https://books.toscrape.com/",
    "formats": [
        {
            "type": "json",
            "prompt": "Extract the first 5 books listed.",
            "schema": {
                "type": "object",
                "properties": {
                    "books": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "title": {"type": "string"},
                                "price": {"type": "string"},
                            },
                            "required": ["title", "price"],
                        },
                    }
                },
                "required": ["books"],
            },
        }
    ],
}

response = requests.post("http://localhost:3002/v2/scrape", json=payload, timeout=180)
response.raise_for_status()
data = response.json()["data"]

if "json" not in data:
    raise SystemExit("The response has no json field. Check the API logs.")
print(json.dumps(data["json"], indent=2, ensure_ascii=False))
```

```bash theme={}
python extract.py
```

```json Output theme={}
{
  "books": [
    { "price": "£51.77", "title": "A Light in the Attic" },
    { "price": "£53.74", "title": "Tipping the Velvet" },
    { "price": "£50.10", "title": "Soumission" },
    { "price": "£47.82", "title": "Sharp Objects" },
    { "price": "£54.23", "title": "Sapiens: A Brief History of Humankind" }
  ]
}
```

The script checks for a `json` field because Firecrawl can return `success: true` without one when the model call fails. See [Troubleshooting](#troubleshooting).

## Choose a model

Firecrawl calls the SambaCloud Responses API (`/v1/responses`), not chat completions, so only models that support that endpoint work. Each row is the full request time for the script above, changing only `MODEL_NAME` in `.env` and running `docker compose up -d api` to apply it:

| Model | Works with Firecrawl | Request time |
| :- | :- | :- |
| `MiniMax-M3.1-Flash` | Yes | 6 to 37 s in 21 of 22 runs, 103 s in one run |
| `gpt-oss-120b` | Yes | 3 to 4 s in 12 runs |
| `gemma-4-31B-it` | Yes | 3 to 36 s in 12 runs |
| `Meta-Llama-3.3-70B-Instruct` | No | Returns no data |

> [!NOTE]
>
> `MiniMax-M3.1-Flash` is a reasoning model. It thinks before it answers, so extraction takes much longer than with `gpt-oss-120b`. Request times varied widely between runs and over the day, most of all for `gemma-4-31B-it`, so measure on your own setup before you rely on a number. A plain scrape of the same page, with no model involved, took 1 to 2 seconds. If you extract many pages, set `MODEL_NAME` to `gpt-oss-120b` for the fastest results in this test.

> [!WARNING]
>
> `Meta-Llama-3.3-70B-Instruct` does not support the Responses API on SambaCloud, so Firecrawl cannot use it. Extraction fails quietly instead of returning an error. See the first item in [Troubleshooting](#troubleshooting).

## Troubleshooting

> [accordion] The response has no json field, but success is true
>
> Firecrawl scraped the page, but the model call failed, so it dropped the `json` field and logged the cause. Read the cause from the API logs:
>
> ```bash theme={}
> docker compose logs api | grep "LLM extraction failed"
> ```
>
> The `message` in that line names the problem. The next three items are the ones reproduced while writing this guide.

> [accordion] Unsupported model ... on Response API
>
> The log shows `statusCode` 400 and `unsupported_model` for a request to `https://api.sambanova.ai/v1/responses`. The model in `MODEL_NAME`, such as `Meta-Llama-3.3-70B-Instruct`, is not available on the Responses API. Set `MODEL_NAME` to one of the models in [Choose a model](#choose-a-model).

> [accordion] Incorrect API key provided
>
> The log shows `statusCode` 401. The `OPENAI_API_KEY` value is not a valid SambaCloud key. Copy a key from the [SambaCloud API keys page](https://cloud.sambanova.ai/apis), update `.env`, and run `docker compose up -d api`.

> [accordion] The model does not exist or you do not have access to it
>
> The log shows `statusCode` 404. The `MODEL_NAME` value is not a SambaCloud model ID. See [SambaCloud models](https://docs.sambanova.ai/docs/en/models/sambacloud-models.md) for the current list.

> [accordion] A change to .env has no effect
>
> Compose reads `.env` when it creates a container. After you edit the file, run `docker compose up -d api` to recreate the `api` service with the new values.

## Additional resources

* [Firecrawl self-hosting guide](https://docs.firecrawl.dev/contributing/self-host)
* [Firecrawl JSON extraction](https://docs.firecrawl.dev/features/llm-extract)
* [Supported SambaCloud models](https://docs.sambanova.ai/docs/en/models/sambacloud-models.md)
