Build with Chroma - SambaNova Integration Guide

Chroma is an open-source vector database that runs in your Python process or as a server. Its OpenAIEmbeddingFunction accepts an api_base, so Chroma can embed your documents and queries with a SambaStack embedding model. You then answer questions with a SambaNova chat model.

Embeddings are available on SambaStack only. E5-Mistral-7B-Instruct is not available on SambaCloud. See Implement embeddings features.On SambaCloud, you can still use Chroma. Set SAMBANOVA_BASE_URL to https://api.sambanova.ai/v1, leave out embedding_function so Chroma embeds with its built-in local model, and skip the instruction example in step 3. Then answer with a SambaNova chat model as in step 4. The LangChain section needs SambaStack.

Prerequisites

A SambaStack instance that serves E5-Mistral-7B-Instruct, plus the instance’s base URL and API key. See API keys and URLs.
A chat model on that instance. This guide uses gpt-oss-120b.
Python 3.10 or higher.

macOS ships with Python 3.9 and exposes it as python3 rather than python. Check your version with python3 --version. If it is below 3.10, install a supported version with pyenv, uv, or brew install python@3.12 before continuing.

Installation and setup

Create a virtual environment

python3 -m venv .venv
source .venv/bin/activate

python -m venv .venv
.venv\Scripts\Activate.ps1

Install dependencies

pip install chromadb openai

This guide was tested with chromadb 1.5.9 and openai 3.35.1.

Set your SambaStack base URL and API key

The base URL must end in /v1.

export SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
export SAMBANOVA_API_KEY="your-sambanova-api-key"

$env:SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
$env:SAMBANOVA_API_KEY="your-sambanova-api-key"

Embed, store, and query

Run the steps in this section in one Python file or session. Each step uses variables from the steps before it.

Create a collection that embeds with SambaNova

Pass api_base and api_key_env_var to Chroma’s OpenAIEmbeddingFunction. api_key_env_var makes Chroma read your SambaNova key instead of the default CHROMA_OPENAI_API_KEY. Do not set api_type. Chroma’s own example sets it to azure for Azure OpenAI, which is not what you want here.

import os
import chromadb
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction

embedding_function = OpenAIEmbeddingFunction(
    api_key_env_var="SAMBANOVA_API_KEY",
    api_base=os.environ["SAMBANOVA_BASE_URL"],
    model_name="E5-Mistral-7B-Instruct",
)

client = chromadb.PersistentClient(path="./chroma-data")
collection = client.get_or_create_collection(
    name="sambanova-demo",
    embedding_function=embedding_function,
    configuration={"hnsw": {"space": "cosine"}},
)

PersistentClient saves the collection to ./chroma-data. Use chromadb.EphemeralClient() to keep everything in memory instead. Chroma defaults to squared L2 distance, so the configuration argument sets cosine distance, which is the usual choice for text embeddings.

Add your documents

Chroma calls the embedding function for you, so you pass text and never handle vectors.

collection.add(
    ids=["doc1", "doc2", "doc3", "doc4"],
    documents=[
        "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference.",
        "Chroma is an open-source vector database for AI applications.",
        "Sourdough bread needs a live starter, flour, water, and salt, and a long slow fermentation.",
        "The capital of Brazil is Brasilia, which was inaugurated in 1960.",
    ],
)

Query the collection

question = "Which chip does SambaNova make?"
results = collection.query(query_texts=[question], n_results=2)

for doc_id, distance, text in zip(
    results["ids"][0], results["distances"][0], results["documents"][0]
):
    print(f"{distance:.3f} {doc_id} {text}")

Lower distance means a closer match. The closest match is doc1, the passage about the SN40L chip, at a cosine distance of about 0.24. The next closest is about 0.36.E5-Mistral-7B-Instruct is an instruction-tuned model. Chroma embeds query_texts as plain text, and this example still ranks the right passage first. To format a query as an instruction, embed it yourself and pass the vector instead:

task = "Given a question, retrieve passages that answer the question"
query_embedding = embedding_function([f"Instruct: {task}\nQuery: {question}"])
results = collection.query(query_embeddings=query_embedding, n_results=2)

Answer with a SambaNova chat model

Pass the retrieved passages to a chat model as context.

from openai import OpenAI

sn = OpenAI(
    base_url=os.environ["SAMBANOVA_BASE_URL"],
    api_key=os.environ["SAMBANOVA_API_KEY"],
)

context = "\n".join(results["documents"][0])
answer = sn.chat.completions.create(
    model="gpt-oss-120b",
    messages=[
        {"role": "system", "content": f"Answer using only the context below.\n\nContext:\n{context}"},
        {"role": "user", "content": question},
    ],
)
print(answer.choices[0].message.content)

question is the same question you searched with in step 3.The model answers that SambaNova makes the SN40L chip.

Delete the collection when you are done

client.delete_collection("sambanova-demo")

This removes the collection. To delete the stored data too, remove the ./chroma-data folder.

Use Chroma with LangChain

The langchain-chroma package wraps the same database. Install it with the OpenAI embeddings integration:

pip install langchain-chroma langchain-openai

This guide was tested with langchain-chroma 1.1.0 and langchain-openai 1.7.2. Set check_embedding_ctx_length=False. LangChain otherwise sends token IDs instead of text, and the SambaStack embeddings endpoint rejects them.

import os
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="E5-Mistral-7B-Instruct",
    base_url=os.environ["SAMBANOVA_BASE_URL"],
    api_key=os.environ["SAMBANOVA_API_KEY"],
    check_embedding_ctx_length=False,
)

vector_store = Chroma.from_texts(
    [
        "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference.",
        "Chroma is an open-source vector database for AI applications.",
        "Sourdough bread needs a live starter, flour, water, and salt, and a long slow fermentation.",
        "The capital of Brazil is Brasilia, which was inaugurated in 1960.",
    ],
    embeddings,
    ids=["doc1", "doc2", "doc3", "doc4"],
    collection_name="sambanova-langchain",
    collection_configuration={"hnsw": {"space": "cosine"}},
)

for doc, score in vector_store.similarity_search_with_score("Which chip does SambaNova make?", k=2):
    print(f"{score:.3f} {doc.page_content}")

The scores are cosine distances, the same as in the direct example. For a persistent store, pass persist_directory="./chroma-data" to Chroma.from_texts.

Additional resources

Chroma documentation

Official guides covering collections, querying, and running Chroma as a server.

Chroma OpenAI embedding function

Reference for OpenAIEmbeddingFunction and its parameters.

LangChain Chroma integration

Reference for the langchain-chroma vector store.

Implement embeddings features

How to use SambaNova embedding models on SambaStack.

Troubleshooting

The SAMBANOVA_API_KEY environment variable is not set

Chroma raises ValueError: The SAMBANOVA_API_KEY environment variable is not set. when the variable named in api_key_env_var is missing. Export it in the shell you run Python from. If you omit api_key_env_var, Chroma looks for CHROMA_OPENAI_API_KEY instead.

404 page not found from the embeddings endpoint

The base URL is missing /v1. Set SAMBANOVA_BASE_URL to your SambaStack base URL followed by /v1.

401 invalid_api_key from the embeddings endpoint

The key does not match the base URL. Use the key issued for your SambaStack instance.

404 model_not_found for text-embedding-ada-002

You did not set model_name, so Chroma requested its default model, text-embedding-ada-002. Set model_name="E5-Mistral-7B-Instruct". If the error names E5-Mistral-7B-Instruct, your SambaStack instance does not serve it, or your key cannot access it. List the models the instance serves with curl -H "Authorization: Bearer $SAMBANOVA_API_KEY" "$SAMBANOVA_BASE_URL/models".

400 Invalid type for 'input': expected string, got array

This error comes from the LangChain example. Pass check_embedding_ctx_length=False to OpenAIEmbeddings, as shown above.

Overview

Developer tools

Frameworks

Interfaces and low-code

Data and infrastructure

Multimodal

Evaluation and monitoring

Chroma integration guide

Prerequisites

Installation and setup

Embed, store, and query

Use Chroma with LangChain

Additional resources

Chroma documentation

Chroma OpenAI embedding function

LangChain Chroma integration

Implement embeddings features

Troubleshooting

​Prerequisites

​Installation and setup

​Embed, store, and query

​Use Chroma with LangChain

​Additional resources

Chroma documentation

Chroma OpenAI embedding function

LangChain Chroma integration

Implement embeddings features

​Troubleshooting

Prerequisites

Installation and setup

Embed, store, and query

Use Chroma with LangChain

Additional resources

Troubleshooting