> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Chroma integration guide

> Use SambaStack embeddings and chat models with Chroma, an open-source vector database, through the Chroma Python client or LangChain.

[Chroma](https://www.trychroma.com/) is an open-source vector database that runs in your Python process or as a server. Its `OpenAIEmbeddingFunction` accepts an `api_base`, so Chroma can embed your documents and queries with a SambaStack embedding model. You then answer questions with a SambaNova chat model.

> [!WARNING]
>
> Embeddings are available on SambaStack only. `E5-Mistral-7B-Instruct` is not available on SambaCloud. See [Implement embeddings features](https://docs.sambanova.ai/docs/en/features/embeddings.md).
>
> On SambaCloud, you can still use Chroma. Set `SAMBANOVA_BASE_URL` to `https://api.sambanova.ai/v1`, leave out `embedding_function` so Chroma embeds with its built-in local model, and skip the instruction example in step 3. Then answer with a SambaNova chat model as in step 4. The LangChain section needs SambaStack.

## Prerequisites

* A SambaStack instance that serves `E5-Mistral-7B-Instruct`, plus the instance's base URL and API key. See [API keys and URLs](https://docs.sambanova.ai/docs/en/get-started/api-keys-urls.md).
* A chat model on that instance. This guide uses `gpt-oss-120b`.
* Python 3.10 or higher.

> [!NOTE]
>
> macOS ships with Python 3.9 and exposes it as `python3` rather than `python`. Check your version with `python3 --version`. If it is below 3.10, install a supported version with [pyenv](https://github.com/pyenv/pyenv), [uv](https://docs.astral.sh/uv/), or `brew install python@3.12` before continuing.

## Installation and setup

**1. Create a virtual environment**

```bash macOS / Linux theme={}
python3 -m venv .venv
source .venv/bin/activate
```

```powershell Windows theme={}
python -m venv .venv
.venv\Scripts\Activate.ps1
```

**2. Install dependencies**

```bash theme={}
pip install chromadb openai
```

This guide was tested with `chromadb` 1.5.9 and `openai` 3.35.1.

**3. Set your SambaStack base URL and API key**

The base URL must end in `/v1`.

```bash macOS / Linux theme={}
export SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
export SAMBANOVA_API_KEY="your-sambanova-api-key"
```

```powershell Windows theme={}
$env:SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
$env:SAMBANOVA_API_KEY="your-sambanova-api-key"
```

## Embed, store, and query

Run the steps in this section in one Python file or session. Each step uses variables from the steps before it.

**1. Create a collection that embeds with SambaNova**

Pass `api_base` and `api_key_env_var` to Chroma's `OpenAIEmbeddingFunction`. `api_key_env_var` makes Chroma read your SambaNova key instead of the default `CHROMA_OPENAI_API_KEY`. Do not set `api_type`. Chroma's own example sets it to `azure` for Azure OpenAI, which is not what you want here.

```python theme={}
import os
import chromadb
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction

embedding_function = OpenAIEmbeddingFunction(
    api_key_env_var="SAMBANOVA_API_KEY",
    api_base=os.environ["SAMBANOVA_BASE_URL"],
    model_name="E5-Mistral-7B-Instruct",
)

client = chromadb.PersistentClient(path="./chroma-data")
collection = client.get_or_create_collection(
    name="sambanova-demo",
    embedding_function=embedding_function,
    configuration={"hnsw": {"space": "cosine"}},
)
```

`PersistentClient` saves the collection to `./chroma-data`. Use `chromadb.EphemeralClient()` to keep everything in memory instead. Chroma defaults to squared L2 distance, so the `configuration` argument sets cosine distance, which is the usual choice for text embeddings.

**2. Add your documents**

Chroma calls the embedding function for you, so you pass text and never handle vectors.

```python theme={}
collection.add(
    ids=["doc1", "doc2", "doc3", "doc4"],
    documents=[
        "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference.",
        "Chroma is an open-source vector database for AI applications.",
        "Sourdough bread needs a live starter, flour, water, and salt, and a long slow fermentation.",
        "The capital of Brazil is Brasilia, which was inaugurated in 1960.",
    ],
)
```

**3. Query the collection**

```python theme={}
question = "Which chip does SambaNova make?"
results = collection.query(query_texts=[question], n_results=2)

for doc_id, distance, text in zip(
    results["ids"][0], results["distances"][0], results["documents"][0]
):
    print(f"{distance:.3f} {doc_id} {text}")
```

Lower distance means a closer match. The closest match is `doc1`, the passage about the SN40L chip, at a cosine distance of about 0.24. The next closest is about 0.36.

`E5-Mistral-7B-Instruct` is an instruction-tuned model. Chroma embeds `query_texts` as plain text, and this example still ranks the right passage first. To format a query as an instruction, embed it yourself and pass the vector instead:

```python theme={}
task = "Given a question, retrieve passages that answer the question"
query_embedding = embedding_function([f"Instruct: {task}\nQuery: {question}"])
results = collection.query(query_embeddings=query_embedding, n_results=2)
```

**4. Answer with a SambaNova chat model**

Pass the retrieved passages to a chat model as context.

```python theme={}
from openai import OpenAI

sn = OpenAI(
    base_url=os.environ["SAMBANOVA_BASE_URL"],
    api_key=os.environ["SAMBANOVA_API_KEY"],
)

context = "\n".join(results["documents"][0])
answer = sn.chat.completions.create(
    model="gpt-oss-120b",
    messages=[
        {"role": "system", "content": f"Answer using only the context below.\n\nContext:\n{context}"},
        {"role": "user", "content": question},
    ],
)
print(answer.choices[0].message.content)
```

`question` is the same question you searched with in step 3.

The model answers that SambaNova makes the SN40L chip.

**5. Delete the collection when you are done**

```python theme={}
client.delete_collection("sambanova-demo")
```

This removes the collection. To delete the stored data too, remove the `./chroma-data` folder.

## Use Chroma with LangChain

The `langchain-chroma` package wraps the same database. Install it with the OpenAI embeddings integration:

```bash theme={}
pip install langchain-chroma langchain-openai
```

This guide was tested with `langchain-chroma` 1.1.0 and `langchain-openai` 1.7.2.

Set `check_embedding_ctx_length=False`. LangChain otherwise sends token IDs instead of text, and the SambaStack embeddings endpoint rejects them.

```python theme={}
import os
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="E5-Mistral-7B-Instruct",
    base_url=os.environ["SAMBANOVA_BASE_URL"],
    api_key=os.environ["SAMBANOVA_API_KEY"],
    check_embedding_ctx_length=False,
)

vector_store = Chroma.from_texts(
    [
        "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference.",
        "Chroma is an open-source vector database for AI applications.",
        "Sourdough bread needs a live starter, flour, water, and salt, and a long slow fermentation.",
        "The capital of Brazil is Brasilia, which was inaugurated in 1960.",
    ],
    embeddings,
    ids=["doc1", "doc2", "doc3", "doc4"],
    collection_name="sambanova-langchain",
    collection_configuration={"hnsw": {"space": "cosine"}},
)

for doc, score in vector_store.similarity_search_with_score("Which chip does SambaNova make?", k=2):
    print(f"{score:.3f} {doc.page_content}")
```

The scores are cosine distances, the same as in the direct example. For a persistent store, pass `persist_directory="./chroma-data"` to `Chroma.from_texts`.

## Additional resources

> [card] [Chroma documentation](https://docs.trychroma.com/)
>
> Official guides covering collections, querying, and running Chroma as a server.

> [card] [Chroma OpenAI embedding function](https://docs.trychroma.com/integrations/embedding-models/openai)
>
> Reference for `OpenAIEmbeddingFunction` and its parameters.

> [card] [LangChain Chroma integration](https://docs.langchain.com/oss/python/integrations/vectorstores/chroma)
>
> Reference for the `langchain-chroma` vector store.

> [card] [Implement embeddings features](https://docs.sambanova.ai/docs/en/features/embeddings.md)
>
> How to use SambaNova embedding models on SambaStack.

## Troubleshooting

> [accordion] The SAMBANOVA\_API\_KEY environment variable is not set
>
> Chroma raises `ValueError: The SAMBANOVA_API_KEY environment variable is not set.` when the variable named in `api_key_env_var` is missing. Export it in the shell you run Python from. If you omit `api_key_env_var`, Chroma looks for `CHROMA_OPENAI_API_KEY` instead.

> [accordion] 404 page not found from the embeddings endpoint
>
> The base URL is missing `/v1`. Set `SAMBANOVA_BASE_URL` to your SambaStack base URL followed by `/v1`.

> [accordion] 401 invalid\_api\_key from the embeddings endpoint
>
> The key does not match the base URL. Use the key issued for your SambaStack instance.

> [accordion] 404 model\_not\_found for text-embedding-ada-002
>
> You did not set `model_name`, so Chroma requested its default model, `text-embedding-ada-002`. Set `model_name="E5-Mistral-7B-Instruct"`. If the error names `E5-Mistral-7B-Instruct`, your SambaStack instance does not serve it, or your key cannot access it. List the models the instance serves with `curl -H "Authorization: Bearer $SAMBANOVA_API_KEY" "$SAMBANOVA_BASE_URL/models"`.

> [accordion] 400 Invalid type for 'input': expected string, got array
>
> This error comes from the LangChain example. Pass `check_embedding_ctx_length=False` to `OpenAIEmbeddings`, as shown above.