Skip to main content

Documentation Index

Fetch the complete documentation index at: https://sambanova-systems.mintlify.dev/docs/llms.txt

Use this file to discover all available pages before exploring further.

Instructor enables structured output generation with SambaNova models. It allows large language models (LLMs) to produce responses in predefined formats—such as JSON, XML, or custom data schemas—ensuring consistency and making the output easier to parse and integrate into downstream systems. This functionality is particularly valuable for APIs, automation pipelines, and AI-driven applications that require reliable and predictable outputs.

​
Prerequisites

​
Installation

pip install "instructor[openai]"

​
Basic usage

The following code demonstrates how to use the SambaCloud API with Instructor to generate structured output from the Meta-Llama-3.3-70B-Instruct model. A User schema is defined using Pydantic, requiring the model to return a response with a name (string) and age (integer). Instructor handles the response validation and parsing, resulting in a structured Python object. 
from openai import OpenAI
import instructor
import os
from pydantic import BaseModel
client = instructor.from_openai(
    OpenAI(
        base_url="https://api.sambanova.ai/v1",
        api_key=os.environ["SAMBANOVA_API_KEY"]
    )
)
class User(BaseModel):
    name: str
    age: int
user = client.chat.completions.create(
    model="Meta-Llama-3.3-70B-Instruct",
    messages=[
        {"role": "user", "content": "Ivan is 28"},
    ],
    response_model=User,
)
print(user)
# > User(name='Ivan', age=28)

​
Async usage

This code also uses SambaCloud API with Instructor to enforce structured output. The result is fetched asynchronously and printed, outputting User(name='Ivan', age=28'). 
from openai import AsyncOpenAI
import instructor
import os
from pydantic import BaseModel

client = instructor.from_openai(
    AsyncOpenAI(
        base_url="https://api.sambanova.ai/v1",
        api_key=os.environ["SAMBANOVA_API_KEY"]
    )
)

class User(BaseModel):
    name: str
    age: int

async def get_user():
    user = await client.chat.completions.create(
        model="Meta-Llama-3.3-70B-Instruct",
        messages=[ {"role": "user", "content": "Ivan is 28"},],
        response_model=User,
    )
    return user

# Run with asyncio
import asyncio
user = asyncio.run(get_user())
print(user)
# > User(name='Ivan', age=28)
Explore an example notebook explaining how to create an email classification tool.