> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# pgvector integration guide

> Store SambaStack embeddings in PostgreSQL with pgvector, search them, and index 4096-dimension vectors with binary quantization.

[pgvector](https://github.com/pgvector/pgvector) is a PostgreSQL extension for storing and searching vectors. You can use it with SambaNova by generating embeddings with a SambaStack embedding model, writing the vectors to a Postgres table, and answering questions with a SambaNova chat model.

> [!WARNING]
>
> Embeddings are available on SambaStack only. `E5-Mistral-7B-Instruct` is not available on SambaCloud. See [Implement embeddings features](https://docs.sambanova.ai/docs/en/features/embeddings.md).
>
> On SambaCloud, you can still use pgvector. Set `SAMBANOVA_BASE_URL` to `https://api.sambanova.ai/v1`, install FastEmbed with `pip install fastembed`, and replace the `embed` function in step 1 with one that embeds locally:
>
> ```python theme={}
> from fastembed import TextEmbedding
>
> local_model = TextEmbedding()
>
> def embed(texts):
>     return [vector.tolist() for vector in local_model.embed(texts)]
> ```
>
> The steps in [Embed, store, and query](#embed-store-and-query) work unchanged. These vectors have 384 dimensions, so you do not need [Index 4096-dimension vectors](#index-4096-dimension-vectors). A standard `hnsw` index works.

## How pgvector uses SambaNova

pgvector never calls an embedding or chat API. It stores vectors and compares them, so there is no model setting to configure. Your code makes both calls: it embeds text with SambaStack, writes the vectors to Postgres, and sends the retrieved text to a chat model.

The same steps work on any Postgres that has pgvector, including managed services that offer it.

`E5-Mistral-7B-Instruct` returns 4096-dimension vectors, which is more than pgvector can index with its standard options. This guide shows what works: exact search for small tables, and a binary quantization index for larger ones. See [Index 4096-dimension vectors](#index-4096-dimension-vectors).

## Prerequisites

* A SambaStack instance that serves `E5-Mistral-7B-Instruct`, plus the instance's base URL and API key. See [API keys and URLs](https://docs.sambanova.ai/docs/en/get-started/api-keys-urls.md).
* A chat model on that instance. This guide uses `gpt-oss-120b`.
* A PostgreSQL database with pgvector 0.7.0 or higher, and a connection URL for it. Version 0.7.0 added the binary quantization function used below. To install pgvector, follow its [installation instructions](https://github.com/pgvector/pgvector#installation).
* Python 3.10 or higher.

> [!NOTE]
>
> macOS ships with Python 3.9 and exposes it as `python3` rather than `python`. Check your version with `python3 --version`. If it is below 3.10, install a supported version with [pyenv](https://github.com/pyenv/pyenv), [uv](https://docs.astral.sh/uv/), or `brew install python@3.12` before continuing.

## Installation and setup

**1. Create a virtual environment**

```bash macOS / Linux theme={}
python3 -m venv .venv
source .venv/bin/activate
```

```powershell Windows theme={}
python -m venv .venv
.venv\Scripts\Activate.ps1
```

**2. Install dependencies**

```bash theme={}
pip install "psycopg[binary]" openai
```

This guide was tested with `psycopg` 3.3.6, `openai` 3.35.1, PostgreSQL 17.11, and pgvector 0.8.7.

**3. Set your SambaStack base URL, API key, and database URL**

The SambaStack base URL must end in `/v1`.

```bash macOS / Linux theme={}
export SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
export SAMBANOVA_API_KEY="your-sambanova-api-key"
export DATABASE_URL="postgresql://user:password@localhost:5432/your-database"
```

```powershell Windows theme={}
$env:SAMBANOVA_BASE_URL="your-sambastack-base-url/v1"
$env:SAMBANOVA_API_KEY="your-sambanova-api-key"
$env:DATABASE_URL="postgresql://user:password@localhost:5432/your-database"
```

## Embed, store, and query

Run the steps in this section in one Python file or session. Each step uses variables from the steps before it.

**1. Embed your documents**

The OpenAI client works against the SambaStack embeddings endpoint.

```python theme={}
import os
from openai import OpenAI

sn = OpenAI(
    base_url=os.environ["SAMBANOVA_BASE_URL"],
    api_key=os.environ["SAMBANOVA_API_KEY"],
)

docs = [
    "SambaNova builds the SN40L chip, a reconfigurable dataflow unit for fast AI inference.",
    "PostgreSQL is an open-source relational database, and pgvector adds vector search to it.",
    "Sourdough bread needs a live starter, flour, water, and salt, and a long slow fermentation.",
    "The capital of Brazil is Brasilia, which was inaugurated in 1960.",
]

def embed(texts):
    response = sn.embeddings.create(model="E5-Mistral-7B-Instruct", input=texts)
    return [d.embedding for d in response.data]

vectors = embed(docs)
dimension = len(vectors[0])  # 4096
```

The embeddings endpoint accepts a `dimensions` argument but ignores it, so you cannot shorten the vectors this way. Every vector has 4096 values.

**2. Create a table and insert the vectors**

Enable the extension, then create a `vector` column whose size matches `dimension`. A `vector` column holds up to 16,000 dimensions. The `DROP TABLE IF EXISTS` line lets you run the guide again without an `already exists` error.

```python theme={}
import psycopg

conn = psycopg.connect(os.environ["DATABASE_URL"], autocommit=True)
conn.execute("CREATE EXTENSION IF NOT EXISTS vector")
conn.execute("DROP TABLE IF EXISTS items")
conn.execute(f"CREATE TABLE items (id bigserial PRIMARY KEY, content text, embedding vector({dimension}))")

for content, vector in zip(docs, vectors):
    conn.execute(
        "INSERT INTO items (content, embedding) VALUES (%s, %s::vector)",
        (content, vector),
    )
```

Cast every Python list to `vector` with `::vector`. Without the cast, Postgres sees the list as a `double precision[]` and the distance operators fail.

**3. Query the table**

`E5-Mistral-7B-Instruct` is an instruction-tuned model, so write each query as an instruction followed by the question. Documents are embedded as plain text. The `<=>` operator returns cosine distance, so `1 - distance` is the cosine similarity.

```python theme={}
task = "Given a question, retrieve passages that answer the question"
question = "Which chip does SambaNova make?"

query_vector = embed([f"Instruct: {task}\nQuery: {question}"])[0]
rows = conn.execute(
    """
    SELECT content, 1 - (embedding <=> %s::vector) AS score
    FROM items
    ORDER BY embedding <=> %s::vector
    LIMIT 2
    """,
    (query_vector, query_vector),
).fetchall()

for content, score in rows:
    print(f"{score:.3f} {content}")
```

The closest match is the passage about the SN40L chip, with a cosine score of about 0.71. The next closest scores about 0.52. Higher is closer.

Without an index, Postgres compares the query with every row, so the results are exact.

**4. Answer with a SambaNova chat model**

Pass the retrieved passages to a chat model as context.

```python theme={}
context = "\n".join(content for content, _ in rows)

answer = sn.chat.completions.create(
    model="gpt-oss-120b",
    messages=[
        {"role": "system", "content": f"Answer using only the context below.\n\nContext:\n{context}"},
        {"role": "user", "content": question},
    ],
)
print(answer.choices[0].message.content)
```

`question` is the same question you searched with in step 3.

The model answers that SambaNova makes the SN40L chip.

## Index 4096-dimension vectors

For large tables, an approximate index keeps queries fast. pgvector's `hnsw` and `ivfflat` indexes cannot cover 4096 dimensions. Their limits are 2,000 dimensions for `vector` and 4,000 for `halfvec`, so both of these fail:

```sql theme={}
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);
-- ERROR: column cannot have more than 2000 dimensions for hnsw index

CREATE INDEX ON items USING hnsw ((embedding::halfvec(4096)) halfvec_cosine_ops);
-- ERROR: column cannot have more than 4000 dimensions for hnsw index
```

Binary quantization indexes up to 64,000 dimensions. It keeps one bit per dimension in the index, finds a shortlist of candidates by Hamming distance, and then reranks the shortlist with exact cosine distance. Create the index on an expression, so the table keeps the full vectors:

```python theme={}
conn.execute(
    f"""
    CREATE INDEX items_embedding_bq ON items
    USING hnsw ((binary_quantize(embedding)::bit({dimension})) bit_hamming_ops)
    """
)
```

Query it by shortlisting with the quantized distance, then reranking:

```python theme={}
rows = conn.execute(
    f"""
    SELECT content, 1 - (embedding <=> %s::vector) AS score
    FROM (
        SELECT * FROM items
        ORDER BY binary_quantize(embedding)::bit({dimension}) <~> binary_quantize(%s::vector)
        LIMIT 20
    ) AS shortlist
    ORDER BY embedding <=> %s::vector
    LIMIT 2
    """,
    (query_vector, query_vector, query_vector),
).fetchall()

for content, score in rows:
    print(f"{score:.3f} {content}")
```

The shortlist size (`LIMIT 20`) trades speed for recall. A larger shortlist reranks more candidates. On this four-row table the results match the exact query.

The index expression in the query must match the one in `CREATE INDEX` exactly, or Postgres will not use the index. To check, run `EXPLAIN` on the shortlist query and look for `Index Scan using items_embedding_bq`. On a table this small, Postgres may choose a sequential scan. Run `SET enable_seqscan = off` first to confirm the index can be used.

## Clean up

```python theme={}
conn.execute("DROP TABLE items")
conn.close()
```

## Additional resources

> [card] [pgvector on GitHub](https://github.com/pgvector/pgvector)
>
> Installation, operators, index types, and limits.

> [card] [psycopg documentation](https://www.psycopg.org/psycopg3/docs/)
>
> The Python driver used in this guide to connect to PostgreSQL.

> [card] [FastEmbed](https://qdrant.tech/documentation/fastembed/)
>
> Local embedding library, used in the SambaCloud tip above.

> [card] [Implement embeddings features](https://docs.sambanova.ai/docs/en/features/embeddings.md)
>
> How to use SambaNova embedding models on SambaStack.

## Troubleshooting

> [accordion] KeyError: 'DATABASE\_URL'
>
> An environment variable from the setup step is not set in the shell you run Python from. The error names the missing variable, such as `DATABASE_URL`, `SAMBANOVA_BASE_URL`, or `SAMBANOVA_API_KEY`. Set it as shown in [Installation and setup](#installation-and-setup), then run the code again from the same shell.

> [accordion] column cannot have more than 2000 dimensions for hnsw index
>
> The same error ends in `for ivfflat index` when you create an `ivfflat` index. A `vector` column with 4096 dimensions is too large for an `hnsw` or `ivfflat` index. Use the binary quantization index in [Index 4096-dimension vectors](#index-4096-dimension-vectors), or search without an index.

> [accordion] column cannot have more than 4000 dimensions for hnsw index
>
> Casting to `halfvec` does not help, because its limit is 4,000 dimensions and the model returns 4096. Use binary quantization instead.

> [accordion] operator does not exist: vector \<=> double precision\[]
>
> The query vector reached Postgres as a plain array. Cast it with `%s::vector`, as in the examples above.

> [accordion] function binary\_quantize(vector) does not exist
>
> Your pgvector is older than 0.7.0. Upgrade pgvector, then run `ALTER EXTENSION vector UPDATE` in the database.

> [accordion] 401 invalid\_api\_key from the embeddings endpoint
>
> The SambaNova key is wrong for the base URL. Use the key issued for your SambaStack instance, and make sure `SAMBANOVA_BASE_URL` points at that instance and ends in `/v1`. Without `/v1`, the embeddings request returns `404 page not found`.

> [accordion] 404 model\_not\_found for E5-Mistral-7B-Instruct
>
> The instance you are calling does not serve the model, or your key cannot access it. List the models the instance serves:
>
> ```bash macOS / Linux theme={}
> curl -H "Authorization: Bearer $SAMBANOVA_API_KEY" "$SAMBANOVA_BASE_URL/models"
> ```
>
> ```powershell Windows theme={}
> curl.exe -H "Authorization: Bearer $env:SAMBANOVA_API_KEY" "$env:SAMBANOVA_BASE_URL/models"
> ```
>
> SambaCloud does not serve embeddings. See the SambaCloud tip at the top of this page.