> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OGX integration guide

In this guide, you'll learn how to set up and use **OGX**, an open-source, OpenAI-compatible API server for building AI applications. OGX (formerly Llama Stack) provides a standardized interface that works with any model and any infrastructure. This guide walks you through running the SambaNova distribution server, installing the client, and running your first model inference.

OGX includes two main components:

* **Server** – A running distribution that hosts inference providers and API adapters.
* **Client** – A consumer of the server's API, interacting with the hosted adapters.

## Prerequisites

Before you begin, ensure you have:

* A [SambaCloud account](https://cloud.sambanova.ai?utm_source=ogx\&utm_medium=external\&utm_campaign=cloud_signup)
* A [SambaNova API key](https://cloud.sambanova.ai/apis?utm_source=ogx\&utm_medium=external\&utm_campaign=cloud_signup)
* Python 3.12 or higher installed on your machine

## Setup

### Option 1: Install and run with uv (recommended)

<Steps>
  <Step title="Set up a Python virtual environment">
    ```bash theme={}
    python -m venv .venv
    source .venv/bin/activate
    ```
  </Step>

  <Step title="Install required dependencies">
    ```bash theme={}
    pip install uv
    uv pip install "ogx[starter]"
    ```
  </Step>

  <Step title="Export required environment variables and start the server">
    ```bash theme={}
    export OGX_PORT=8321
    export SAMBANOVA_API_KEY="your-api-key-here"

    uv run ogx run starter --port $OGX_PORT --insecure
    ```

    The `--insecure` flag disables TLS for local development. Without it, the server exits with `TLS required: set tls_certfile/tls_keyfile in server config or pass '--insecure' to disable.`
  </Step>
</Steps>

### Option 2: Run with Docker

```bash theme={}
export OGX_PORT=8321
export SAMBANOVA_API_KEY="your-api-key-here"

docker run -it \
  -p $OGX_PORT:$OGX_PORT \
  -v ~/.ogx:/root/.ogx \
  -e SAMBANOVA_API_KEY=$SAMBANOVA_API_KEY \
  ogxai/distribution-starter \
  --port $OGX_PORT \
  --insecure
```

### Install the OGX client

Both options need the client to run the example below. In the same or another environment, run:

```bash theme={}
pip install ogx-client
```

## Example code

The following Python code demonstrates basic usage:

```python theme={}
from ogx_client import OgxClient

OGX_PORT = 8321
client = OgxClient(base_url=f"http://localhost:{OGX_PORT}")

# List all available models
models = client.models.list()
print("--- Available models: ---")
for m in models.data:
    print(f"- {m.id}")
print()

# Choose a model from the list
model = "sambanova/Meta-Llama-3.3-70B-Instruct"

# Run chat completion
response = client.chat.completions.create(
    model=model,
    messages=[
        {"role": "system", "content": "You are a friendly assistant."},
        {"role": "user", "content": "Write a two-sentence poem about llamas."},
    ],
)

print(response.choices[0].message.content)
```

This demonstrates the full client-server loop: connecting, listing models, and running inference.

Explore the [SambaNova OGX integration repo](https://github.com/sambanova/integrations/tree/main/ogx) to find several use cases using SambaNova distribution LLMs, embeddings, tools, and agent adapters.

## Troubleshooting

<AccordionGroup>
  <Accordion title="TLS required">
    The server refuses to start without TLS configured:

    ```
    TLS required: set tls_certfile/tls_keyfile in server config or pass '--insecure' to disable.
    ```

    Add the `--insecure` flag to the server run command, as shown in the setup steps above. Use TLS certificates instead if you are running in a shared or production environment.
  </Accordion>

  <Accordion title="Incorrect API key provided">
    The key reached SambaCloud but was rejected:

    ```
    Incorrect API key provided: ... You can find your API key at https://cloud.sambanova.ai/apis
    ```

    Verify `SAMBANOVA_API_KEY` is exported in the environment where the *server* is running, not just the one running the client, and that it matches a valid key from the [SambaCloud console](https://cloud.sambanova.ai/apis?utm_source=ogx\&utm_medium=external\&utm_campaign=cloud_signup).
  </Accordion>

  <Accordion title="Package 'ogx' requires a different Python">
    pip refuses the install outright:

    ```
    ERROR: Package 'ogx' requires a different Python: 3.11.x not in '>=3.12'
    ```

    `ogx` requires Python 3.12 or later. Create the virtual environment with an explicit interpreter, for example `python3.12 -m venv .venv`. A system `python3` older than 3.12 is the usual cause.
  </Accordion>

  <Accordion title="The server exits on startup with the port already in use">
    Another process is holding `$OGX_PORT`. Identify it, then either stop it or choose a free port:

    ```bash theme={}
    lsof -i :$OGX_PORT
    export OGX_PORT=8322
    ```

    Both the `uv run` and `docker run` commands read `$OGX_PORT`, so nothing else needs to change.
  </Accordion>
</AccordionGroup>

## Additional resources

Refer to the [OGX documentation](https://ogx-ai.github.io/docs) to:

* Understand core concepts
* Explore sample applications
* Learn how to extend and customize the framework