Build with Weaviate - SambaNova Integration Guide

Weaviate is an open-source vector database that runs vectorization and generation as database modules, so your application code sends text and Weaviate does the rest. Its text2vec-openai and generative-openai modules accept a baseURL, which is the documented route to any OpenAI-compatible endpoint. Pointing both at a SambaStack instance lets Weaviate embed your data and answer questions with SambaNova models.

Embeddings are available on SambaStack only. E5-Mistral-7B-Instruct is not available on SambaCloud. See Implement embeddings features.On SambaCloud, you can still use Weaviate with your own vectors and a SambaNova chat model. See Use Weaviate with SambaCloud.

Prerequisites

A SambaStack instance that serves E5-Mistral-7B-Instruct, plus the instance’s base URL and API key. See API keys and URLs.
A chat model on that instance. This guide uses gpt-oss-120b.
A running Weaviate server, or a Weaviate Cloud cluster. To run one locally, follow the Weaviate Docker installation guide.
Network access from the Weaviate server to your SambaStack instance. The Weaviate server calls SambaStack, not your Python code, so a Weaviate Cloud cluster cannot reach a SambaStack instance on a private network.
Python 3.10 or higher.

macOS ships with Python 3.9 and exposes it as python3 rather than python. Check your version with python3 --version. If it is below 3.10, install a supported version with pyenv, uv, or brew install python@3.12 before continuing.

The text2vec-openai and generative-openai modules must be enabled on your Weaviate server. Weaviate Cloud enables them by default. On a self-hosted server, open /v1/meta and check that both appear under modules. If they do not, enable them with the ENABLE_MODULES environment variable, as described in Weaviate’s module configuration guide.

Installation and setup

Create a virtual environment

python3 -m venv .venv
source .venv/bin/activate

python -m venv .venv
.venv\Scripts\Activate.ps1

Install dependencies

pip install weaviate-client

This guide was tested with weaviate-client 4.24.1 and Weaviate 1.40.1.

Set your SambaStack base URL and API key

Do not add /v1 to the base URL here. Weaviate appends /v1/embeddings and /v1/chat/completions itself, so a base URL that already ends in /v1 produces a 404.

export SAMBANOVA_BASE_URL="your-sambastack-base-url"
export SAMBANOVA_API_KEY="your-sambanova-api-key"

$env:SAMBANOVA_BASE_URL="your-sambastack-base-url"
$env:SAMBANOVA_API_KEY="your-sambanova-api-key"

Configure the modules and query

Run the steps in this section in one Python file or session. Each step uses variables from the steps before it.

Connect to Weaviate

Weaviate sends your SambaNova key to the modules in the X-OpenAI-Api-Key header.

import os
import weaviate

client = weaviate.connect_to_local(
    headers={"X-OpenAI-Api-Key": os.environ["SAMBANOVA_API_KEY"]},
)

connect_to_local connects to a server on localhost. For Weaviate Cloud, set WEAVIATE_URL to your cluster URL and WEAVIATE_API_KEY to your cluster’s API key, then connect with connect_to_weaviate_cloud and the same headers:

from weaviate.classes.init import Auth

client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
    headers={"X-OpenAI-Api-Key": os.environ["SAMBANOVA_API_KEY"]},
)

Create a collection that uses SambaNova

Set base_url on both modules, and name the SambaNova models. Weaviate reads the vector size from the model, so you do not set a dimension.

from weaviate.classes.config import Configure, DataType, Property

base_url = os.environ["SAMBANOVA_BASE_URL"]

if client.collections.exists("SambaNovaDemo"):
    client.collections.delete("SambaNovaDemo")

collection = client.collections.create(
    "SambaNovaDemo",
    properties=[Property(name="text", data_type=DataType.TEXT)],
    vector_config=Configure.Vectors.text2vec_openai(
        base_url=base_url,
        model="E5-Mistral-7B-Instruct",
    ),
    generative_config=Configure.Generative.openai(
        base_url=base_url,
        model="gpt-oss-120b",
    ),
)

If a SambaNovaDemo collection already exists, the code deletes it first, so each run starts with a clean collection.text2vec_openai handles embeddings and Generative.openai handles chat completions. Weaviate calls SambaNova from the server, so the SambaStack instance must be reachable from where Weaviate runs.

Add your documents

Weaviate embeds each object as it is inserted. You pass text and never handle vectors.

collection.data.insert_many([
    {"text": "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference."},
    {"text": "Weaviate is an open-source vector database."},
    {"text": "Sourdough bread needs a live starter, flour, water, and salt, and a long slow fermentation."},
    {"text": "The capital of Brazil is Brasilia, which was inaugurated in 1960."},
])

Search by meaning

from weaviate.classes.query import MetadataQuery

response = collection.query.near_text(
    query="Which chip does SambaNova make?",
    limit=2,
    return_metadata=MetadataQuery(distance=True),
)

for obj in response.objects:
    print(f"{obj.metadata.distance:.3f} {obj.properties['text']}")

Lower distance means a closer match. The closest match is the passage about the SN40L chip, at a cosine distance of about 0.33. The next closest is about 0.43.

Answer with a SambaNova chat model

Weaviate retrieves the closest objects and passes them to the generative module in one call.

response = collection.generate.near_text(
    query="Which chip does SambaNova make?",
    limit=2,
    grouped_task="Which chip does SambaNova make? Answer using only these passages.",
)
print(response.generative.text)

The model answers that SambaNova makes the SN40L chip.

Delete the collection and close the connection

client.collections.delete("SambaNovaDemo")
client.close()

Use Weaviate with SambaCloud

SambaCloud does not serve embeddings, so embed the text yourself with a local model and let Weaviate’s generative module call SambaCloud for answers. Install FastEmbed, and set SAMBANOVA_BASE_URL to https://api.sambanova.ai.

pip install fastembed

import os
import weaviate
from fastembed import TextEmbedding
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.data import DataObject

local_model = TextEmbedding()

def embed(texts):
    return [vector.tolist() for vector in local_model.embed(texts)]

client = weaviate.connect_to_local(
    headers={"X-OpenAI-Api-Key": os.environ["SAMBANOVA_API_KEY"]},
)

if client.collections.exists("SambaNovaDemo"):
    client.collections.delete("SambaNovaDemo")

collection = client.collections.create(
    "SambaNovaDemo",
    properties=[Property(name="text", data_type=DataType.TEXT)],
    vector_config=Configure.Vectors.self_provided(),
    generative_config=Configure.Generative.openai(
        base_url=os.environ["SAMBANOVA_BASE_URL"],
        model="gpt-oss-120b",
    ),
)

docs = [
    "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference.",
    "Weaviate is an open-source vector database.",
]
collection.data.insert_many(
    [DataObject(properties={"text": d}, vector=v) for d, v in zip(docs, embed(docs))]
)

question = "Which chip does SambaNova make?"
response = collection.generate.near_vector(
    near_vector=embed([question])[0],
    limit=2,
    grouped_task=f"{question} Answer using only these passages.",
)
print(response.generative.text)

client.collections.delete("SambaNovaDemo")
client.close()

Additional resources

Weaviate documentation

Official guides covering collections, search, and deployment.

OpenAI embeddings in Weaviate

Reference for the text2vec-openai module and its parameters.

OpenAI generative AI in Weaviate

Reference for the generative-openai module and its parameters.

Implement embeddings features

How to use SambaNova embedding models on SambaStack.

Troubleshooting

KeyError: 'SAMBANOVA_BASE_URL'

An environment variable is not set in the shell you run Python from. The error names the missing variable, such as SAMBANOVA_BASE_URL or SAMBANOVA_API_KEY. Set it as shown in Installation and setup, then run the code again from the same shell. For Weaviate Cloud, also set WEAVIATE_URL and WEAVIATE_API_KEY.

failed to parse vectorization response (status 404)

The base URL ends in /v1. Weaviate adds /v1/embeddings to the base URL, so it ends up calling /v1/v1/embeddings. Set base_url to your SambaStack base URL without /v1, then recreate the collection.

API Key: no api key found neither in request header: X-Openai-Api-Key

The connection has no key to send to the module. Pass headers={"X-OpenAI-Api-Key": ...} to connect_to_local or connect_to_weaviate_cloud, and check that SAMBANOVA_API_KEY is exported in the shell you run Python from.

connection to: OpenAI API failed with status: 401

The key does not match the base URL. Use the key issued for your SambaStack instance.

connection to: OpenAI API failed with status: 404 ... model does not exist

The instance does not serve the model named in the collection, or your key cannot access it. List the models the instance serves:

curl -H "Authorization: Bearer $SAMBANOVA_API_KEY" "$SAMBANOVA_BASE_URL/v1/models"

curl.exe -H "Authorization: Bearer $env:SAMBANOVA_API_KEY" "$env:SAMBANOVA_BASE_URL/v1/models"

SambaCloud does not serve embeddings. See Use Weaviate with SambaCloud.

422: class name SambaNovaDemo already exists

The code above deletes an existing SambaNovaDemo collection first. If you removed that check, delete it with client.collections.delete("SambaNovaDemo").

Overview

Developer tools

Frameworks

Interfaces and low-code

Data and infrastructure

Multimodal

Evaluation and monitoring

Weaviate integration guide

Prerequisites

Installation and setup

Configure the modules and query

Use Weaviate with SambaCloud

Additional resources

Weaviate documentation

OpenAI embeddings in Weaviate

OpenAI generative AI in Weaviate

Implement embeddings features

Troubleshooting

​Prerequisites

​Installation and setup

​Configure the modules and query

​Use Weaviate with SambaCloud

​Additional resources

Weaviate documentation

OpenAI embeddings in Weaviate

OpenAI generative AI in Weaviate

Implement embeddings features

​Troubleshooting

Prerequisites

Installation and setup

Configure the modules and query

Use Weaviate with SambaCloud

Additional resources

Troubleshooting