Use the SambaNova Cloud API with OpenAI’s client libraries to easily upgrade your existing applications to use the fastest inference on the best open source models!

​

Download the library

Run the command below to download the library.

pip install openai

​

Use SambaNova APIs with OpenAI client libraries

Switching to SambaNova APIs with OpenAI’s client libraries is as simple as setting two values, as shown below. Start a new file and create the client variable by passing the base_url and api_key. The api_key should be saved in a secure location. 

1. Set the base_url to the “https://api.sambanova.ai/v1”.

2. Set the api_key to the "YOUR SAMBANOVA CLOUD API KEY".

from openai import OpenAI

client = OpenAI(
    base_url="https://api.sambanova.ai/v1", 
    api_key="YOUR SAMBANOVA CLOUD API KEY"
)

Now you can make an API request to a model and choose how to receive your output. 

​

Non-streaming example

The following code demonstrates using the OpenAI python client for non-streaming completions.

completion = client.chat.completions.create(
  model="Meta-Llama-3.1-8B-Instruct",
  messages = [
      {"role": "system", "content": "Answer the question in a couple sentences."},
      {"role": "user", "content": "Share a happy story with me"}
    ]
)

print(completion.choices[0].message)

​

Streaming example

The following code demonstrates using the OpenAI python client for streaming completions.

completion = client.chat.completions.create(
  model="Meta-Llama-3.1-8B-Instruct",
  messages = [
      {"role": "system", "content": "Answer the question in a couple sentences."},
      {"role": "user", "content": "Share a happy story with me"}
    ],
  stream= True
)

for chunk in completion:
  print(chunk.choices[0].delta.content)

​

Unsupported OpenAI features

The following features are not yet supported and will be ignored:

• logprobs

• top_logprobs

• n

• presence_penalty

• frequency_penalty

• logit_bias

• seed

​

Feature differences

temperature: The SambaNova Cloud API supports a value between 0 and 1.

​

Features unsupported by OpenAI clients

The SambaNova API supports the top_k parameter. This is not supported by the OpenAI client.