> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Qdrant integration guide

> Generate embeddings with SambaStack, store and search them in Qdrant, and answer questions with a SambaNova chat model.

[Qdrant](https://qdrant.tech/) is an open-source vector database. You can use it with SambaNova by generating embeddings with a SambaStack embedding model, storing the vectors in a Qdrant collection, and answering questions with a SambaNova chat model.

> [!WARNING]
>
> Embeddings are available on SambaStack only. `E5-Mistral-7B-Instruct` is not available on SambaCloud. See [Implement embeddings features](https://docs.sambanova.ai/docs/en/features/embeddings.md).
>
> On SambaCloud, you can still use Qdrant. Set `SAMBANOVA_BASE_URL` to `https://api.sambanova.ai/v1`, install FastEmbed with `pip install "qdrant-client[fastembed]"`, and replace the `embed` function in step 1 with one that embeds locally:
>
> ```python theme={}
> from fastembed import TextEmbedding
>
> local_model = TextEmbedding()
>
> def embed(texts):
>     return [vector.tolist() for vector in local_model.embed(texts)]
> ```
>
> The rest of the guide works unchanged.

## How Qdrant uses SambaNova

Qdrant can generate embeddings for you, but only from models it knows about. [Qdrant Cloud Inference](https://qdrant.tech/documentation/inference/cloud-inference/) hosts its own models, and [external providers](https://qdrant.tech/documentation/inference/external-inference-providers/) are limited to OpenAI, Cohere, Jina AI, and OpenRouter. Both are reached through Qdrant Cloud, and there is no setting for a custom base URL, so Qdrant cannot call a SambaNova endpoint for you.

Qdrant's [inference options](https://qdrant.tech/documentation/inference/) describe the alternative for teams that run their own inference service: client-side inference. You embed the text yourself, then store and query the vectors. This guide uses that approach, so your code makes both calls: it embeds text with SambaStack, then sends the vectors to Qdrant.

## Prerequisites

* A SambaStack instance that serves `E5-Mistral-7B-Instruct`, plus the instance's base URL and API key. See [API keys and URLs](https://docs.sambanova.ai/docs/en/get-started/api-keys-urls.md).
* A chat model on that instance. This guide uses `gpt-oss-120b`.
* A running Qdrant server, or a [Qdrant Cloud](https://cloud.qdrant.io/) cluster. To run one locally, follow the [Qdrant quickstart](https://qdrant.tech/documentation/quickstart/). To try the guide without a server, create the client with `QdrantClient(":memory:")` instead.
* Python 3.10 or higher.

> [!NOTE]
>
> macOS ships with Python 3.9 and exposes it as `python3` rather than `python`. Check your version with `python3 --version`. If it is below 3.10, install a supported version with [pyenv](https://github.com/pyenv/pyenv), [uv](https://docs.astral.sh/uv/), or `brew install python@3.12` before continuing.

## Installation and setup

**1. Create a virtual environment**

```bash macOS / Linux theme={}
python3 -m venv .venv
source .venv/bin/activate
```

```powershell Windows theme={}
python -m venv .venv
.venv\Scripts\Activate.ps1
```

**2. Install dependencies**

```bash theme={}
pip install qdrant-client openai
```

This guide was tested with `qdrant-client` 1.20.0, `openai` 3.40.0, and Qdrant server 1.20.0.

**3. Set your SambaStack base URL, API key, and Qdrant URL**

The SambaStack base URL must end in `/v1`. The Qdrant URL below is the default for a local server. For Qdrant Cloud, set `QDRANT_URL` to your cluster URL and also set `QDRANT_API_KEY` to your cluster's API key. A local server needs no key.

```bash macOS / Linux theme={}
export SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
export SAMBANOVA_API_KEY="your-sambanova-api-key"
export QDRANT_URL="http://localhost:6333"
```

```powershell Windows theme={}
$env:SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
$env:SAMBANOVA_API_KEY="your-sambanova-api-key"
$env:QDRANT_URL="http://localhost:6333"
```

## Embed, store, and query

Run the steps in this section in one Python file or session. Each step uses variables from the steps before it.

**1. Embed your documents**

The OpenAI client works against the SambaStack embeddings endpoint. `E5-Mistral-7B-Instruct` returns 4096-dimension vectors, which sets the size of the Qdrant collection in the next step.

```python theme={}
import os
from openai import OpenAI

sn = OpenAI(
    base_url=os.environ["SAMBANOVA_BASE_URL"],
    api_key=os.environ["SAMBANOVA_API_KEY"],
)

docs = [
    "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference.",
    "Qdrant is an open-source vector database for similarity search.",
    "Sourdough bread needs a live starter, flour, water, and salt, and a long slow fermentation.",
    "The capital of Brazil is Brasilia, which was inaugurated in 1960.",
]

def embed(texts):
    response = sn.embeddings.create(model="E5-Mistral-7B-Instruct", input=texts)
    return [d.embedding for d in response.data]

vectors = embed(docs)
dimension = len(vectors[0])  # 4096
```

**2. Create a collection and upsert the vectors**

Set the collection's `size` to the length of the vectors you just created. Qdrant rejects an upsert whose vectors do not match that size. Point IDs must be unsigned integers or UUIDs, so the text goes in the payload.

```python theme={}
from qdrant_client import QdrantClient, models

client = QdrantClient(
    url=os.environ["QDRANT_URL"],
    api_key=os.environ.get("QDRANT_API_KEY"),  # Qdrant Cloud only
)

if not client.collection_exists("sambanova-demo"):
    client.create_collection(
        collection_name="sambanova-demo",
        vectors_config=models.VectorParams(size=dimension, distance=models.Distance.COSINE),
    )

client.upsert(
    collection_name="sambanova-demo",
    points=[
        models.PointStruct(id=i, vector=vector, payload={"text": text})
        for i, (text, vector) in enumerate(zip(docs, vectors))
    ],
)
```

`upsert` waits for the write to finish by default, so you can query as soon as it returns.

**3. Query the collection**

`E5-Mistral-7B-Instruct` is an instruction-tuned model, so write each query as an instruction followed by the question. Documents are embedded as plain text.

```python theme={}
task = "Given a question, retrieve passages that answer the question"
question = "Which chip does SambaNova make?"

query_vector = embed([f"Instruct: {task}\nQuery: {question}"])[0]
hits = client.query_points(
    collection_name="sambanova-demo",
    query=query_vector,
    limit=2,
    with_payload=True,
).points

for hit in hits:
    print(f"{hit.score:.3f} {hit.id} {hit.payload['text']}")
```

The closest match is the passage about the SN40L chip, with a cosine score of about 0.71. The next closest scores about 0.47. Higher is closer.

**4. Answer with a SambaNova chat model**

Pass the retrieved passages to a chat model as context.

```python theme={}
context = "\n".join(hit.payload["text"] for hit in hits)

answer = sn.chat.completions.create(
    model="gpt-oss-120b",
    messages=[
        {"role": "system", "content": f"Answer using only the context below.\n\nContext:\n{context}"},
        {"role": "user", "content": question},
    ],
)
print(answer.choices[0].message.content)
```

`question` is the same question you searched with in step 3.

The model answers that SambaNova makes the SN40L chip.

**5. Delete the collection when you are done**

```python theme={}
client.delete_collection("sambanova-demo")
```

## Additional resources

> [card] [Qdrant documentation](https://qdrant.tech/documentation/)
>
> Official guides covering collections, points, search, and deployment.

> [card] [Qdrant inference options](https://qdrant.tech/documentation/inference/)
>
> Server-side and client-side ways to generate embeddings for Qdrant.

> [card] [FastEmbed](https://qdrant.tech/documentation/fastembed/)
>
> Qdrant's local embedding library, used in the SambaCloud tip above.

> [card] [Implement embeddings features](https://docs.sambanova.ai/docs/en/features/embeddings.md)
>
> How to use SambaNova embedding models on SambaStack.

## Troubleshooting

> [accordion] KeyError: 'QDRANT\_URL'
>
> An environment variable from the setup step is not set in the shell you run Python from. The error names the missing variable, such as `QDRANT_URL`, `SAMBANOVA_BASE_URL`, or `SAMBANOVA_API_KEY`. Set it as shown in [Installation and setup](#installation-and-setup), then run the code again from the same shell.

> [accordion] Wrong input: Vector dimension error: expected dim: 4096, got 1536
>
> The collection was created with a different `size` than the vectors you are upserting. `E5-Mistral-7B-Instruct` returns 4096 values per input. Delete the collection and create it again with `size=4096`, or read the length from a returned vector as shown above.

> [accordion] 409 Conflict: Collection already exists
>
> The collection already exists. The code above skips `create_collection` when it does; if you removed that check, delete the collection with `client.delete_collection("sambanova-demo")`.

> [accordion] 400 Bad Request: value doc1 is not a valid point ID
>
> Point IDs must be an unsigned integer or a UUID. Use a number or `str(uuid.uuid4())` as the ID and keep your own label in the payload.

> [accordion] Connection refused from QdrantClient
>
> No Qdrant server is listening at `QDRANT_URL`. Start one by following the [Qdrant quickstart](https://qdrant.tech/documentation/quickstart/), then check that the URL and port match. To try the guide without a server, use `QdrantClient(":memory:")`.

> [accordion] 401 invalid\_api\_key from the embeddings endpoint
>
> The SambaNova key is wrong for the base URL. Use the key issued for your SambaStack instance, and make sure `SAMBANOVA_BASE_URL` points at that instance and ends in `/v1`. Without `/v1`, the embeddings request returns `404 page not found`.

> [accordion] 404 model\_not\_found for E5-Mistral-7B-Instruct
>
> The instance you are calling does not serve the model, or your key cannot access it. List the models the instance serves:
>
> ```bash macOS / Linux theme={}
> curl -H "Authorization: Bearer $SAMBANOVA_API_KEY" "$SAMBANOVA_BASE_URL/models"
> ```
>
> ```powershell Windows theme={}
> curl.exe -H "Authorization: Bearer $env:SAMBANOVA_API_KEY" "$env:SAMBANOVA_BASE_URL/models"
> ```
>
> SambaCloud does not serve embeddings. See the SambaCloud tip at the top of this page.

> [accordion] Qdrant does not offer a SambaNova model for server-side inference
>
> This is expected. Qdrant generates embeddings only with its Cloud Inference models and the OpenAI, Cohere, Jina AI, and OpenRouter providers. Create the collection with a `size` and upsert vectors you generate yourself, as shown above.