Build with Qdrant - SambaNova Integration Guide

Qdrant is an open-source vector database. You can use it with SambaNova by generating embeddings with a SambaStack embedding model, storing the vectors in a Qdrant collection, and answering questions with a SambaNova chat model.

Embeddings are available on SambaStack only. E5-Mistral-7B-Instruct is not available on SambaCloud. See Implement embeddings features.On SambaCloud, you can still use Qdrant. Set SAMBANOVA_BASE_URL to https://api.sambanova.ai/v1, install FastEmbed with pip install "qdrant-client[fastembed]", and replace the embed function in step 1 with one that embeds locally:

from fastembed import TextEmbedding

local_model = TextEmbedding()

def embed(texts):
    return [vector.tolist() for vector in local_model.embed(texts)]

The rest of the guide works unchanged.

How Qdrant uses SambaNova

Qdrant can generate embeddings for you, but only from models it knows about. Qdrant Cloud Inference hosts its own models, and external providers are limited to OpenAI, Cohere, Jina AI, and OpenRouter. Both are reached through Qdrant Cloud, and there is no setting for a custom base URL, so Qdrant cannot call a SambaNova endpoint for you. Qdrant’s inference options describe the alternative for teams that run their own inference service: client-side inference. You embed the text yourself, then store and query the vectors. This guide uses that approach, so your code makes both calls: it embeds text with SambaStack, then sends the vectors to Qdrant.

Prerequisites

A SambaStack instance that serves E5-Mistral-7B-Instruct, plus the instance’s base URL and API key. See API keys and URLs.
A chat model on that instance. This guide uses gpt-oss-120b.
A running Qdrant server, or a Qdrant Cloud cluster. To run one locally, follow the Qdrant quickstart. To try the guide without a server, create the client with QdrantClient(":memory:") instead.
Python 3.10 or higher.

macOS ships with Python 3.9 and exposes it as python3 rather than python. Check your version with python3 --version. If it is below 3.10, install a supported version with pyenv, uv, or brew install python@3.12 before continuing.

Installation and setup

Create a virtual environment

python3 -m venv .venv
source .venv/bin/activate

python -m venv .venv
.venv\Scripts\Activate.ps1

Install dependencies

pip install qdrant-client openai

This guide was tested with qdrant-client 1.20.0, openai 3.40.0, and Qdrant server 1.20.0.

Set your SambaStack base URL, API key, and Qdrant URL

The SambaStack base URL must end in /v1. The Qdrant URL below is the default for a local server. For Qdrant Cloud, set QDRANT_URL to your cluster URL and also set QDRANT_API_KEY to your cluster’s API key. A local server needs no key.

export SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
export SAMBANOVA_API_KEY="your-sambanova-api-key"
export QDRANT_URL="http://localhost:6333"

$env:SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
$env:SAMBANOVA_API_KEY="your-sambanova-api-key"
$env:QDRANT_URL="http://localhost:6333"

Embed, store, and query

Run the steps in this section in one Python file or session. Each step uses variables from the steps before it.

Embed your documents

The OpenAI client works against the SambaStack embeddings endpoint. E5-Mistral-7B-Instruct returns 4096-dimension vectors, which sets the size of the Qdrant collection in the next step.

import os
from openai import OpenAI

sn = OpenAI(
    base_url=os.environ["SAMBANOVA_BASE_URL"],
    api_key=os.environ["SAMBANOVA_API_KEY"],
)

docs = [
    "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference.",
    "Qdrant is an open-source vector database for similarity search.",
    "Sourdough bread needs a live starter, flour, water, and salt, and a long slow fermentation.",
    "The capital of Brazil is Brasilia, which was inaugurated in 1960.",
]

def embed(texts):
    response = sn.embeddings.create(model="E5-Mistral-7B-Instruct", input=texts)
    return [d.embedding for d in response.data]

vectors = embed(docs)
dimension = len(vectors[0])  # 4096

Create a collection and upsert the vectors

Set the collection’s size to the length of the vectors you just created. Qdrant rejects an upsert whose vectors do not match that size. Point IDs must be unsigned integers or UUIDs, so the text goes in the payload.

from qdrant_client import QdrantClient, models

client = QdrantClient(
    url=os.environ["QDRANT_URL"],
    api_key=os.environ.get("QDRANT_API_KEY"),  # Qdrant Cloud only
)

if not client.collection_exists("sambanova-demo"):
    client.create_collection(
        collection_name="sambanova-demo",
        vectors_config=models.VectorParams(size=dimension, distance=models.Distance.COSINE),
    )

client.upsert(
    collection_name="sambanova-demo",
    points=[
        models.PointStruct(id=i, vector=vector, payload={"text": text})
        for i, (text, vector) in enumerate(zip(docs, vectors))
    ],
)

upsert waits for the write to finish by default, so you can query as soon as it returns.

Query the collection

E5-Mistral-7B-Instruct is an instruction-tuned model, so write each query as an instruction followed by the question. Documents are embedded as plain text.

task = "Given a question, retrieve passages that answer the question"
question = "Which chip does SambaNova make?"

query_vector = embed([f"Instruct: {task}\nQuery: {question}"])[0]
hits = client.query_points(
    collection_name="sambanova-demo",
    query=query_vector,
    limit=2,
    with_payload=True,
).points

for hit in hits:
    print(f"{hit.score:.3f} {hit.id} {hit.payload['text']}")

The closest match is the passage about the SN40L chip, with a cosine score of about 0.71. The next closest scores about 0.47. Higher is closer.

Answer with a SambaNova chat model

Pass the retrieved passages to a chat model as context.

context = "\n".join(hit.payload["text"] for hit in hits)

answer = sn.chat.completions.create(
    model="gpt-oss-120b",
    messages=[
        {"role": "system", "content": f"Answer using only the context below.\n\nContext:\n{context}"},
        {"role": "user", "content": question},
    ],
)
print(answer.choices[0].message.content)

question is the same question you searched with in step 3.The model answers that SambaNova makes the SN40L chip.

Delete the collection when you are done

client.delete_collection("sambanova-demo")

Additional resources

Qdrant documentation

Official guides covering collections, points, search, and deployment.

Qdrant inference options

Server-side and client-side ways to generate embeddings for Qdrant.

FastEmbed

Qdrant’s local embedding library, used in the SambaCloud tip above.

Implement embeddings features

How to use SambaNova embedding models on SambaStack.

Troubleshooting

KeyError: 'QDRANT_URL'

An environment variable from the setup step is not set in the shell you run Python from. The error names the missing variable, such as QDRANT_URL, SAMBANOVA_BASE_URL, or SAMBANOVA_API_KEY. Set it as shown in Installation and setup, then run the code again from the same shell.

Wrong input: Vector dimension error: expected dim: 4096, got 1536

The collection was created with a different size than the vectors you are upserting. E5-Mistral-7B-Instruct returns 4096 values per input. Delete the collection and create it again with size=4096, or read the length from a returned vector as shown above.

409 Conflict: Collection already exists

The collection already exists. The code above skips create_collection when it does; if you removed that check, delete the collection with client.delete_collection("sambanova-demo").

400 Bad Request: value doc1 is not a valid point ID

Point IDs must be an unsigned integer or a UUID. Use a number or str(uuid.uuid4()) as the ID and keep your own label in the payload.

Connection refused from QdrantClient

No Qdrant server is listening at QDRANT_URL. Start one by following the Qdrant quickstart, then check that the URL and port match. To try the guide without a server, use QdrantClient(":memory:").

401 invalid_api_key from the embeddings endpoint

The SambaNova key is wrong for the base URL. Use the key issued for your SambaStack instance, and make sure SAMBANOVA_BASE_URL points at that instance and ends in /v1. Without /v1, the embeddings request returns 404 page not found.

404 model_not_found for E5-Mistral-7B-Instruct

The instance you are calling does not serve the model, or your key cannot access it. List the models the instance serves:

curl -H "Authorization: Bearer $SAMBANOVA_API_KEY" "$SAMBANOVA_BASE_URL/models"

curl.exe -H "Authorization: Bearer $env:SAMBANOVA_API_KEY" "$env:SAMBANOVA_BASE_URL/models"

SambaCloud does not serve embeddings. See the SambaCloud tip at the top of this page.

Qdrant does not offer a SambaNova model for server-side inference

This is expected. Qdrant generates embeddings only with its Cloud Inference models and the OpenAI, Cohere, Jina AI, and OpenRouter providers. Create the collection with a size and upsert vectors you generate yourself, as shown above.

Overview

Developer tools

Frameworks

Interfaces and low-code

Data and infrastructure

Multimodal

Evaluation and monitoring

Qdrant integration guide

How Qdrant uses SambaNova

Prerequisites

Installation and setup

Embed, store, and query

Additional resources

Qdrant documentation

Qdrant inference options

FastEmbed

Implement embeddings features

Troubleshooting

​How Qdrant uses SambaNova

​Prerequisites

​Installation and setup

​Embed, store, and query

​Additional resources

Qdrant documentation

Qdrant inference options

FastEmbed

Implement embeddings features

​Troubleshooting

How Qdrant uses SambaNova

Prerequisites

Installation and setup

Embed, store, and query

Additional resources

Troubleshooting