Function calling in SambaNova Cloud enables dynamic workflows by allowing the model to select and suggest function calls based on user input, which helps in building agentic workflows. By defining a set of functions, or tools, you provide context that lets the model recommend and fill in function arguments as needed.

​
How function calling works

Function calling enables adaptive workflows that leverage real-time data and structured outputs, creating more dynamic and responsive model interactions.

  1. Submit a Query with tools: Start by submitting a user query along with available tools defined in JSON Schema. This schema specifies parameters for each function.

  2. The model processes and suggests: The model interprets the query, assesses intent, and decides if it will respond conversationally or suggest function calls. If a function is called, it fills in the arguments based on the schema.

  3. Receive a model response: You’ll get a response from the model, which may include a function call suggestion. Execute the function with the provided arguments and return the result to the model for further interaction.

​
Supported models

  • Meta-Llama-3.1-8B-Instruct

  • Meta-Llama-3.1-70B-Instruct

  • Meta-Llama-3.1-405B-Instruct

  • Meta-Llama-3.3-70B-Instruct

Meta recommends using Llama 70B-Instruct or Llama 405B-Instruct for applications that combine conversation and tool calling. Llama 8B-Instruct cannot reliably maintain a conversation alongside tool-calling definitions. It can be used for zero-shot tool calling, but tool instructions should be removed for regular conversations.

​
Example usage

The examples below describe each step of using function calling with an end-to-end example after the last step.

​
Step 1: Define the function schema

Define a JSON schema for your function. You will need to specify:

  • The name of the function.

  • A description of what it does.

  • The parameters, their data types, and descriptions.

Example schema for solving quadratic equations
{
    "type": "function",
    "function": {
        "name": "solve_quadratic",
        "description": "Solves a quadratic equation given coefficients a, b, and c.",
        "parameters": {
            "type": "object",
            "properties": {
                "a": {"type": "integer", "description": "Coefficient of the squared term"},
                "b": {"type": "integer", "description": "Coefficient of the linear term"},
                "c": {"type": "integer", "description": "Constant term"},
                "root_type": {"type": "string", "description": "Type of roots to return: 'real' or 'all'"}
            },
            "required": ["a", "b", "c"]
        }
    }
}

​
Step 2: Configure function calling in your request

When sending a request to SN Cloud, include the function definition in the tools parameter and set tool_choice to the following:

  • auto : allows the model to choose between generating a message or calling a function. This is the default tool choice when the field is not specified.

  • required : This forces the model to generate a function call. The model will then always select one or more function(s) to call.

  • To enforce a specific function call, set tool_choice = {"type": "function", "function": {"name": "solve_quadratic"}}. This ensures the model will only use the specified function.

Example Python request
import openai
import cmath
import json

# Initialize the client with SN Cloud base URL and your API key
client = openai.OpenAI(
    base_url="https://api.sambanova.ai/v1", 
    api_key="YOUR SAMBANOVA API KEY"
)

def solve_quadratic(a, b, c, root_type="real"):
    """
    Solve a quadratic equation of the form ax^2 + bx + c = 0.
    """
    discriminant = b**2 - 4*a*c
    
    if root_type == "real":
        if discriminant < 0:
            return []  # No real roots
        else:
            root1 = (-b + discriminant**0.5) / (2 * a)
            root2 = (-b - discriminant**0.5) / (2 * a)
            return [root1, root2]
    else:
        root1 = (-b + cmath.sqrt(discriminant)) / (2 * a)
        root2 = (-b - cmath.sqrt(discriminant)) / (2 * a)
        return [
            {"real": root1.real, "imag": root1.imag},
            {"real": root2.real, "imag": root2.imag}
        ]


# Define user input and function schema
user_prompt = "Find all the roots of a quadratic equation given coefficients a = 3, b = -11, and c = -4."
messages = [

        {
            "role": "user",
            "content": user_prompt,
        }
    ]

tools = [
    {
        "type": "function",
        "function": {
            "name": "solve_quadratic",
            "description": "Solves a quadratic equation given coefficients a, b, and c.",
            "parameters": {
                "type": "object",
                "properties": {
                    "a": {"type": "integer", "description": "Coefficient of the squared term"},
                    "b": {"type": "integer", "description": "Coefficient of the linear term"},
                    "c": {"type": "integer", "description": "Constant term"},
                    "root_type": {"type": "string", "description": "Type of roots: 'real' or 'all'"}
                },
                "required": ["a", "b", "c"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="Meta-Llama-3.1-70B-Instruct",
    messages=messages,
    tools=tools,
    tool_choice="required"
)

print(response)

​
Step 3: Handle tool calls

If the model chooses to call a function, you will find tool_calls in the response. Extract the function call details and execute the corresponding function with the provided parameters.

Example code
response_message = response.choices[0].message
tool_calls = response_message.tool_calls

# If tool call is present
if tool_calls:
    tool_call = tool_calls[0]
    function_name = tool_call.function.name
    arguments = tool_call.function.arguments
    arguments = json.loads(arguments)

    # Call the appropriate function with parsed arguments
    if function_name == "solve_quadratic":
        result = solve_quadratic(
            a=arguments["a"],
            b=arguments["b"],
            c=arguments["c"],
            root_type=arguments.get("root_type", "real")
        )
        print(result)

​
Step 4: Provide function results back to the model

Once you have computed the result, pass it back to the model to continue the conversation or confirm the output.

Example code
# Convert result to JSON string format to return to model
function_response = json.dumps({"result": result})

# Provide the function response back to the model as a message
messages.append(
    {
        "tool_call_id": tool_call.id,
        "role": "tool",
        "name": function_name,
        "content": function_response
    }
)

# Second API call to incorporate the function result into conversation
second_response = client.chat.completions.create(
    model="Meta-Llama-3.1-70B-Instruct",
    messages=messages,
    
)

# print the final response from the model
print(second_response.choices[0].message.content)

​
Step 5: Example output

An example output is shown below.

The roots of the quadratic equation with coefficients a = 3, b = -11, and c = -4 are 4 and -1/3.

​
End-to-end example using OpenAI compatibility

End-to-end example using OpenAI compatibility
import openai
import cmath
import json

# Define the OpenAI client
client = openai.OpenAI(
    base_url="https://api.sambanova.ai/v1", 
    api_key="YOUR SAMBANOVA API KEY"
)

MODEL = 'Meta-Llama-3.1-70B-Instruct'

# Function to solve the quadratic equation
def solve_quadratic(a, b, c, root_type="real"):
    """
    Solve a quadratic equation of the form ax^2 + bx + c = 0.
    """
    discriminant = b**2 - 4*a*c
    
    if root_type == "real":
        if discriminant < 0:
            return []  # No real roots
        else:
            root1 = (-b + discriminant**0.5) / (2 * a)
            root2 = (-b - discriminant**0.5) / (2 * a)
            return [root1, root2]
    else:
        root1 = (-b + cmath.sqrt(discriminant)) / (2 * a)
        root2 = (-b - cmath.sqrt(discriminant)) / (2 * a)
        return [
            {"real": root1.real, "imag": root1.imag},
            {"real": root2.real, "imag": root2.imag}
        ]

# Function to run conversation and provide tool result back to the model
def run_conversation(user_prompt):
    # Initial conversation with user input
    messages = [
        {
            "role": "system",
            "content": "You are an assistant that can solve quadratic equations given coefficients a, b, and c."
        },
        {
            "role": "user",
            "content": user_prompt,
        }
    ]

    # Define the tool
    tools = [
        {
            "type": "function",
            "function": {
                "name": "solve_quadratic",
                "description": "Solve a quadratic equation given coefficients a, b, and c.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "a": {"type": "integer", "description": "Coefficient of the squared term."},
                        "b": {"type": "integer", "description": "Coefficient of the linear term."},
                        "c": {"type": "integer", "description": "Constant term."},
                        "root_type": {"type": "string", "description": "Type of roots: 'real' or 'all'."}
                    },
                    "required": ["a", "b", "c"],
                }
            }
        }
    ]

    # First API call to get model's response
    response = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        tools=tools,
        tool_choice="auto",
        max_tokens=500
    )

    response_message = response.choices[0].message
    tool_calls = response_message.tool_calls

    # If tool call is present
    if tool_calls:
        tool_call = tool_calls[0]
        function_name = tool_call.function.name
        arguments = tool_call.function.arguments
        arguments = json.loads(arguments)

        # Call the appropriate function with parsed arguments
        if function_name == "solve_quadratic":
            result = solve_quadratic(
                a=arguments["a"],
                b=arguments["b"],
                c=arguments["c"],
                root_type=arguments.get("root_type", "real")
            )
            # Convert result to JSON string format to return to model
            function_response = json.dumps({"result": result})

            # Provide the function response back to the model as a message
            messages.append(
                {
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": function_response
                }
            )

            # Second API call to incorporate the function result into conversation
            second_response = client.chat.completions.create(
                model=MODEL,
                messages=messages,
                max_tokens=500
            )

            # Return the final response from the model
            return second_response.choices[0].message.content

# Example user prompt
user_prompt = "Find all the roots of a quadratic equation given coefficients a = 3, b = -11, and c = -4."
print(run_conversation(user_prompt))

​
JSON mode

You can set the response_format parameter to json_object in your request to ensure that the model outputs a valid JSON. In case the mode is not able to generate a valid JSON, an error will be returned.

In case the model fails to generate a valid JSON, you will get an error message Model did not output valid JSON.

Example code
import openai

# Define the OpenAI client
client = openai.OpenAI(
    base_url="https://api.sambanova.ai/v1", 
    api_key="YOUR SAMBANOVA API KEY"
)

MODEL = 'Meta-Llama-3.1-70B-Instruct'


def run_conversation(user_prompt):
    # Initial conversation with user input
    messages = [
        {
            "role": "system",
            "content": "Always provide the response in this JSON format: {\"country\": \"name\", \"capital\": \"xx\"}"
        },

        {
            "role": "user",
            "content": user_prompt,
        }
    ]

    # First API call to get model's response
    response = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        max_tokens=500,
        response_format = { "type": "json_object"},
        # stream = True
    )
    
    response_message = response.choices[0].message
    print(response_message)


run_conversation('what is the capital of Austria')
Example response
ChatCompletionMessage(content='{"country": "Austria", "capital": "Vienna"}', role='assistant', function_call=None, tool_calls=None)
Text generationVision