LlamaIndex

python3 -m venv .venv
source .venv/bin/activate
python --version

pip install "llama-index-llms-sambanovasystems>=0.6.0" sseclient-py

export SAMBANOVA_API_KEY="your-api-key"

from llama_index.llms.sambanovasystems import SambaNovaCloud

# Reads your key from the SAMBANOVA_API_KEY environment variable
llm = SambaNovaCloud(model="Meta-Llama-3.3-70B-Instruct")

response = llm.complete("Say hello in one sentence.")
print(response)
# → Hello! It's great to connect with you. How can I help you today?

from llama_index.llms.sambanovasystems import SambaNovaCloud

llm = SambaNovaCloud(model="Meta-Llama-3.3-70B-Instruct")

for chunk in llm.stream_complete("Explain what an LLM is in one sentence."):
    print(chunk.delta, end="", flush=True)
print()

pip install "llama-index-embeddings-huggingface>=0.7.0"

mkdir -p data
cat > data/sambanova.txt <<'EOF'
SambaCloud serves open models on SambaNova RDU hardware.
Meta-Llama-3.3-70B-Instruct accepts a context window of 128k tokens.
The LlamaIndex integration package for SambaCloud is llama-index-llms-sambanovasystems.
EOF

from llama_index.core import Settings, SimpleDirectoryReader, VectorStoreIndex
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.sambanovasystems import SambaNovaCloud

# SambaCloud writes the answer.
Settings.llm = SambaNovaCloud(
    model="Meta-Llama-3.3-70B-Instruct",
    context_window=128000,
)

# A local model creates the vectors. It downloads on the first run.
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

response = query_engine.query("What context window does Llama 3.3 70B accept?")
print(response)

Meta-Llama-3.3-70B-Instruct accepts a context window of 128k tokens.