> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Checkpoint conversion tool

The Checkpoint Conversion Tool is a utility that converts external model artifacts, such as checkpoints, into the SambaNova-compatible format required for deployment on SambaRack and SambaCloud systems. The tool retrieves platform-specific metadata so it can stay aligned with the model architectures and releases supported by your SambaStack instance.

<Note>
  Checkpoint conversion is a substep of deploying custom checkpoints on SambaStack or SambaCloud. See the [Deploying custom checkpoints](/en/v1.2.0/sambastack/service-administration/deploy-custom-checkpoints) page for the high-level workflow.
</Note>

## Overview

The Checkpoint Conversion Tool provides two primary capabilities:

1. **Checkpoint conversion**: Transforms HuggingFace-format checkpoints into SambaNova-compatible format for deployment on SN40L hardware
2. **Speculative decoding validation**: Verifies whether a converted draft checkpoint is compatible with a target checkpoint for speculative decoding deployments

## Prerequisites

### System requirements

Before using the Checkpoint Conversion Tool, ensure your system meets the following requirements:

| Requirement          | Specification                                                                                                                                                            |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Memory**           | At least 2.5x the size of your checkpoint's SafeTensor or .bin files. For example, a 10GB checkpoint requires a minimum of 25GB available memory.                        |
| **Storage**          | Equal to the input checkpoint size for the output. For example, converting a 14GB checkpoint requires 14GB of free storage for the converted output (plus the original). |
| **Operating System** | macOS or Linux-based operating systems. Windows OS is **not supported**.                                                                                                 |

### Estimated conversion times

| Checkpoint Size | Example Model               | Estimated Time |
| --------------- | --------------------------- | -------------- |
| \~8GB           | Meta-Llama-3.3-8B-Instruct  | \~5 minutes    |
| \~140GB         | Meta-Llama-3.3-70B-Instruct | \~1 hour       |

<Tip>
  Run the conversion locally on your machine or workspace that has access to your checkpoint's storage mount. A cloud compute instance can also be used, but note that data transfer of checkpoints can take a long time.
</Tip>

### Required software

* **Docker Desktop** or **Docker Engine** - [Installation guide](https://docs.docker.com/engine/install/)
* **Google Cloud CLI** - [Installation guide](https://cloud.google.com/sdk/docs/install)

### Required access

* Network access to your SambaStack instance endpoint
* Authentication credentials for Google Cloud (the same account used for your organization's SambaStack artifact registry, if configured; otherwise contact your SambaNova account team or support)

## Supported models and checkpoint formats

### Supported model architectures

Custom checkpoint deployment is supported for a growing set of base models. To check whether custom checkpoints are supported for a model family, see the [Supported models](/en/v1.2.0/sambastack/service-administration/supported-models-and-bundles) table and check the **Custom checkpoints supported** field in the *Features and optimizations* column.

### Checkpoint format requirements

Checkpoints are accepted in the HuggingFace format. The tensors should be in the safetensors format and the checkpoint directory should contain the same relevant config files as the base model for the custom checkpoint.

For example, if the custom checkpoint is a finetuned variant of [meta-llama/Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/tree/main), the checkpoint directory should contain files similar to the following:

| File                                                                      | Description                                                                                                                              |
| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `config.json`                                                             | Defines the model architecture configuration, including hidden size, number of layers, attention heads, and other structural parameters. |
| `generation_config.json`                                                  | Stores default text generation parameters such as sampling strategy, temperature, and maximum tokens.                                    |
| `model-00001-of-00030.safetensors` ... `model-00030-of-00030.safetensors` | Sharded model weight files containing the trained tensor parameters of the model. Large models split weights across multiple files.      |
| `model.safetensors.index.json`                                            | Index file mapping model weight tensors to the corresponding safetensor shard files.                                                     |
| `special_tokens_map.json`                                                 | Defines special tokens used by the tokenizer such as BOS, EOS, and padding tokens.                                                       |
| `tokenizer.json`                                                          | Contains the tokenizer vocabulary and tokenization rules used to convert text into model input tokens.                                   |
| `tokenizer_config.json`                                                   | Configuration metadata for the tokenizer, including tokenization behavior and settings.                                                  |

### Checkpoint compatibility

Given that a checkpoint is fine-tuned or derived from one of the supported models for your platform, checkpoints are compatible when their computational graph **has not been modified from the original checkpoint** (i.e., tensor weights and shapes).

**Aspects that must remain unchanged:**

* Number of attention heads
* Rope type (rope theta)
* Model vocabulary size
* Optimizer type
* Static architectural attributes in `config.json` such as: `head_dim`, `hidden_act`, `intermediate_size`, `attention_bias`, `attention_dropout`, `vocab_size`

**Aspects that can be modified:**

* Model weights or model weight tensor values
* Tokenizer and vocabulary (as long as it retains the exact vocabulary size of the original model checkpoint–useful for multilingual use-cases)

<Tip>
  It can be helpful to think about this in terms of a static graph. Aspects of a model that are typically static in engines such as TensorRT-LLM are also static for custom checkpoints.
</Tip>

### Practical compatibility examples

Take the base model [meta-llama/Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) (a base model supported by SambaNova). The following checkpoints use the same computational graph as the original 70B model and **can** be converted and deployed on SambaNova platforms:

* [deepseek-ai/DeepSeek-R1-Distill-Llama-70B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B)
* [tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4](https://huggingface.co/tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4)

These checkpoints have undergone updates to their model weights, which have been adjusted and refined to improve performance or adapt to specific tasks or datasets.

## Download and set up

The Checkpoint Conversion Tool is distributed as a Docker container that encapsulates all conversion and validation utilities.

<Note>
  Contact your SambaNova account team or SambaNova support to obtain your `<REGISTRY_HOST>` and `<REGISTRY_PATH>` values.
</Note>

<Steps>
  <Step stepNumber={1} titleSize="h3" title="Install Docker Desktop or Docker Engine">
    Install the Docker engine in your conversion environment. You can follow the official [Docker Engine Installation](https://docs.docker.com/engine/install/) guide.

    After installation, start Docker (this can be done via the desktop application on macOS).
  </Step>

  <Step stepNumber={2} titleSize="h3" title="Install Google Cloud CLI">
    Install Google Cloud CLI in your conversion environment. You can follow the official [Google Cloud CLI Installation](https://cloud.google.com/sdk/docs/install) guide.
  </Step>

  <Step stepNumber={3} titleSize="h3" title="Authenticate and pull the container image">
    First, configure the Docker client to authenticate with `<REGISTRY_HOST>` (one-time setup):

    ```bash theme={null}
    gcloud auth configure-docker <REGISTRY_HOST>
    ```

    <Note>
      Replace `<REGISTRY_HOST>` with your registry domain, provided by your SambaNova account team or SambaNova support.
    </Note>

    Authenticate with Google Cloud:

    ```bash theme={null}
    gcloud auth login
    ```

    <Note>
      Use the Google account associated with your organization's SambaStack artifact registry access, if you have one configured.
    </Note>

    Pull the Docker image:

    ```bash theme={null}
    docker pull <REGISTRY_PATH>
    ```

    <Note>
      Replace `<REGISTRY_PATH>` with your Checkpoint Conversion Tool image path, provided by your SambaNova account team or SambaNova support.
    </Note>
  </Step>

  <Step stepNumber={4} titleSize="h3" title="Sync model metadata">
    This step downloads platform-specific model metadata from your SambaStack instance and stores it locally. The tool uses this metadata to perform checkpoint conversions. You can additionally inspect the model metadata to understand how artifacts are converted to run on SambaNova's SN40L hardware.

    The metadata is fetched and cached by running the Checkpoint Conversion Tool container with the `download-serving-cache` command.

    ### Command template

    ```bash theme={null}
    docker run -v $HOST_WORKING_DIR:$DOCKER_WORKING_DIR --rm -it \
        --platform linux/amd64 \
        $IMAGE_NAME \
        download-serving-cache \
        --server $SERVER \
        --cache_location $DOCKER_WORKING_DIR/$CACHE_LOCATION
    ```

    <Note>
      On native Linux/amd64 environments, the `--platform linux/amd64` flag is optional and can be omitted. It is primarily helpful on macOS (especially Apple Silicon) to ensure the correct architecture is used.
    </Note>

    ### Parameters

    | Input                | Type  | Description                                                                                                                                                                              |
    | -------------------- | ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
    | `HOST_WORKING_DIR`   |       | Directory on the **host** machine that will be mounted into the container. This directory will receive the downloaded metadata and may also contain checkpoints you plan to convert.     |
    | `DOCKER_WORKING_DIR` |       | Path **inside the container** where `HOST_WORKING_DIR` is mounted. This must match the right-hand side of the `-v` flag.                                                                 |
    | `IMAGE_NAME`         | `str` | Full name of the Checkpoint Conversion Tool Docker image. Same value as `<REGISTRY_PATH>` in the previous step.                                                                          |
    | `SERVER`             | `str` | Base endpoint URL for your SambaStack platform and product version. Refer to the [API keys and URLs](/en/get-started/api-keys-urls) section of the documentation to obtain this value.   |
    | `CACHE_LOCATION`     | `str` | Path (relative to `DOCKER_WORKING_DIR`) where metadata will be stored inside the container. The metadata will be visible on the host under the corresponding path in `HOST_WORKING_DIR`. |

    <Check>
      Your setup should now be complete. The full setup only needs to be done once. You may need to repeat Step 4 under the following conditions:

      * SambaNova or your organization make more model architectures available for custom checkpoint inference
      * Updates to any model metadata are released for your SambaStack instance (scoped by the base URL)
      * You are switching to another platform (e.g., SambaCloud) or your organization is using multiple SambaStack instances (scoped by base URLs)
    </Check>
  </Step>
</Steps>

## Convert and validate checkpoint

The Checkpoint Conversion Tool converts custom checkpoints (from HuggingFace or otherwise) into a format that can run on SambaNova's SN40L hardware. This step occurs prior to uploading or deploying custom checkpoints on any SambaStack or SambaCloud.

### Command template

```bash theme={null}
docker run -v $HOST_WORKING_DIR:$DOCKER_WORKING_DIR --rm -it \
    --platform linux/amd64 \
    $IMAGE_NAME \
    prepare-ckpt \
    --model $MODEL_NAME \
    --original_checkpoint_path "$DOCKER_WORKING_DIR/$CHECKPOINT_DIR" \
    --output_checkpoint_path "$DOCKER_WORKING_DIR/$OUTPUT_DIR" \
    --transformers_version $TRANSFORMERS_VERSION \
    --server $SERVER \
    --cache_location $DOCKER_WORKING_DIR/$CACHE_LOCATION \
    --ignore_transformers_version
```

### Parameters

These are the primary input flags to the `prepare-ckpt` command:

| Flag / Variable                                   | Type   | Description                                                                                                                                                                                                                                                                                                                                                                     |
| ------------------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--model` / `MODEL_NAME`                          | `str`  | The base model to convert (e.g., `Meta-Llama-3.1-8B`). To check whether custom checkpoints are supported for a model family, see the [Model and Bundle Directory](/en/v1.2.0/sambastack/service-administration/supported-models-and-bundles) table and check the **Custom checkpoints supported** field in the *Features and optimizations* column.                             |
| `--original_checkpoint_path` / `CHECKPOINT_DIR`   | `str`  | Path **inside the container** to the directory containing the input checkpoint (e.g., config, tokenizer, safetensors). This is typically `$DOCKER_WORKING_DIR/<subdir>`, where `<subdir>` is mounted from `HOST_WORKING_DIR`.                                                                                                                                                   |
| `--output_checkpoint_path` / `OUTPUT_DIR`         | `str`  | Path **inside the container** where the converted checkpoint will be written. The converted artifacts will appear on the host under `$HOST_WORKING_DIR/$OUTPUT_DIR`.                                                                                                                                                                                                            |
| `--transformers_version` / `TRANSFORMERS_VERSION` | `str`  | (Optional but recommended.) Specifies the Hugging Face `transformers` version required for deployment (e.g., `4.45.1`). The tool checks that the version used to save the model is ≤ the deployment version. If this validation is not needed, omit the flag. If you encounter version errors, re-save the checkpoint with a newer `transformers` version or update this value. |
| `--ignore_transformers_version`                   | `bool` | When enabled, skips all `transformers` version checking. Default is `False`. Useful if you encounter a version validation error and explicitly want to bypass it. Ignoring the Transformers version of your checkpoint uses the default backend version for converting your checkpoints (i.e., `4.45.1`).                                                                       |
| `--server` / `SERVER`                             | `str`  | Source of serving metadata. Can be `embedded`, a local path, or a remote URL such as `https://api.sambanova.ai/`. Typically this is the base endpoint URL for your SambaStack instance.                                                                                                                                                                                         |
| `--cache_location` / `CACHE_LOCATION`             | `str`  | Location (inside the container) where serving metadata/configs are stored. Usually a subdirectory of `$DOCKER_WORKING_DIR` and visible on the host under `$HOST_WORKING_DIR/$CACHE_LOCATION`.                                                                                                                                                                                   |

### Host-level variables

In addition to the flags above, you will typically set the following environment variables for the Docker command:

| Variable             | Description                                                                                                              |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `HOST_WORKING_DIR`   | Directory on the **host** that contains your input checkpoint and where output will be written. Must be writable.        |
| `DOCKER_WORKING_DIR` | Directory **inside the container** where `HOST_WORKING_DIR` is mounted (matches the right-hand side of `-v`).            |
| `IMAGE_NAME`         | Full image name of the Checkpoint Conversion Tool container.                                                             |
| `CHECKPOINT_DIR`     | Subdirectory under `HOST_WORKING_DIR` (mirrored under `DOCKER_WORKING_DIR`) containing the original checkpoint.          |
| `OUTPUT_DIR`         | Subdirectory under `HOST_WORKING_DIR` (mirrored under `DOCKER_WORKING_DIR`) where converted checkpoints will be written. |

## Validating the conversion output

After a successful conversion, verify that the output directory is complete and consistent:

### Checklist

* The output directory contains the expected set of `safetensors` files, typically named `model-00001-of-000NN.safetensors`, `model-00002-of-000NN.safetensors`, ..., up to `NN`
* The output directory contains a `DONE` file indicating successful completion
* The tool logs show no errors

### Successful conversion log

A successful run ends with a log similar to:

```text theme={null}
-----------------------------------------------
[1] STEP: parse_args - Succeeded
-----------------------------------------------


> Errors (None)

-----------------------------------------------
[2] STEP: legalizer - Succeeded
-----------------------------------------------


> Errors (None)

***************************************************
YYYY-MM-DDTHH:MM:SSUTC - byoc_lib - INFO - The process is completed without any errors.
```

### Failed conversion log

If conversion fails, you may see error entries like:

```text theme={null}
-----------------------------------------------
[2] STEP: validate_transformer - Failed
-----------------------------------------------


> Errors
  Id: ByocErrorId.TRANSFORMERS_VERSION_ERROR
  Responsible: ByocErrorOwner.USAGE
  Reason: The transformers version of your checkpoint (4.52.4) is larger than the maximum supported transformers version (4.45.1).
  Suggestion: To fix this, please save your checkpoint with a transformers environment <= 4.45.1. Alternatively, you can include the --ignore_transformers_version flag, but this may lead to errors due to transformers version compatibility issues.

  Stack Trace:
    ...
```

<Note>
  If you do not see either `"One or more failures have occurred. Do not deploy"` or `"The process is completed without any errors"` and the process exits unexpectedly, you may have hit an out-of-memory (OOM) condition. In that case, increase available memory and rerun the conversion.
</Note>

<Check>
  Once you've confirmed that the conversion completed successfully and the output directory contains all expected files, you can proceed to the upload and deployment steps in [Deploying custom checkpoints](/en/v1.2.0/sambastack/service-administration/deploy-custom-checkpoints).
</Check>

## Interpreting output logs and troubleshooting

The Checkpoint Conversion Tool prints a structured log for each run to help diagnose issues, identify which part of the process failed, and decide what to do next.

### Log structure overview

At a high level, each run includes:

* A **final report header** showing the command that was executed
* A sequence of **STEP** entries indicating which module or test is running
* An **Errors** section (if any failures occur) with details and suggestions
* A **final status line** indicating whether the process completed successfully or failed

### Example output log

```text theme={null}
FINAL REPORT FOR: `prepare-ckpt --model Meta-Llama-3.1-8B --original_checkpoint_path /data/Meta-Llama-3.1-8B --output_checkpoint_path /data/outputs/Meta-Llama-3.1-8B --transformers_version 4.45.1`
***************************************************

-----------------------------------------------
[1] STEP: parse_args - Succeeded
-----------------------------------------------


> Errors (None)

-----------------------------------------------
[2] STEP: validate_transformer - Failed
-----------------------------------------------


> Errors
  Id: ByocErrorId.TRANSFORMERS_VERSION_ERROR
  Responsible: ByocErrorOwner.USAGE
  Reason: The transformers version of your checkpoint (4.52.4) is larger than the maximum supported transformers version (4.45.1).
  Suggestion: To fix this, please save your checkpoint with a transformers environment <= 4.45.1. Alternatively, you can include the --ignore_transformers_version flag, but this may lead to errors due to transformers version compatibility issues.

  Stack Trace:
    Traceback (most recent call last):
    File "/byoc_core/main.runfiles/_main/byoc_core/byoc.py", line 46, in _byoc
    warning = validate_transformer(convert_checkpoint_path,
    File "/byoc_core/main.runfiles/_main/byoc_core/transformers_validator/check_transformers.py", line 115, in validate_transformer
    raise TransformersVersionError(model_config_transformers_version,
    byoc_core.transformers_validator.check_transformers.TransformersVersionError: (<Version('4.52.4')>, '4.45.1')
    (<Version('4.52.4')>, '4.45.1')


***************************************************
2025-08-25T17:49:10UTC - byoc_lib - ERROR - One or more failures have occurred. Do not deploy
```

### Log components explained

**Final report header**

Shows the exact command and arguments used:

```text theme={null}
FINAL REPORT FOR: `prepare-ckpt --model Meta-Llama-3.1-8B --original_checkpoint_path /data/Meta-Llama-3.1-8B --output_checkpoint_path /data/outputs/Meta-Llama-3.1-8B --transformers_version 4.45.1`
***************************************************
```

**STEP blocks**

Each step represents a modular phase of the workflow (argument parsing, validation, conversion, etc.):

```text theme={null}
-----------------------------------------------
[1] STEP: parse_args - Succeeded
-----------------------------------------------
...
-----------------------------------------------
[2] STEP: validate_transformer - Failed
-----------------------------------------------
```

**Error blocks**

For each step, an **Errors** block follows with structured details:

* If there are no errors:

  ```text theme={null}
  > Errors (None)
  ```
* If a step results in error:

  ```text theme={null}
  > Errors
    Id: ByocErrorId.TRANSFORMERS_VERSION_ERROR
    Responsible: ByocErrorOwner.USAGE
    Reason: The transformers version of your checkpoint (4.52.4) is larger than the maximum supported transformers version (4.45.1).
    Suggestion: To fix this, please save your checkpoint with a transformers environment <= 4.45.1. Alternatively, you can include the --ignore_transformers_version flag, but this may lead to errors due to transformers version compatibility issues.

    Stack Trace:
      ...
  ```

**Final status line**

At the end of the log, you'll see either a success or failure message:

```text theme={null}
2025-08-25T17:49:10UTC - byoc_lib - ERROR - One or more failures have occurred. Do not deploy
```

### Error block components

If a step failed, each error block will contain the following fields:

| Field         | Description                                                                                                                                                          |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Id`          | A marker that can be used to trace the specific check or process that failed when inspecting the container.                                                          |
| `Responsible` | Indicates whether the error can be remedied by you (`ByocErrorOwner.USAGE`) or whether there is an issue with the conversion tool itself (`ByocErrorOwner.LIBRARY`). |
| `Reason`      | A human-readable explanation of what may have gone wrong.                                                                                                            |
| `Suggestion`  | A recommendation that can be used to fix or triage the error.                                                                                                        |
| `Stack Trace` | Helpful to identify where the error took place.                                                                                                                      |

### Common errors and solutions

| Error ID                     | Cause                                                                 | Solution                                                                                       |
| ---------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| `TRANSFORMERS_VERSION_ERROR` | Checkpoint was saved with a Transformers version newer than supported | Re-save the checkpoint with Transformers ≤ 4.45.1, or use `--ignore_transformers_version` flag |
| Out-of-memory (OOM)          | Insufficient system memory for checkpoint size                        | Increase available memory to at least 2.5x the checkpoint size                                 |
| Missing files                | Incomplete checkpoint directory                                       | Ensure all required files (config.json, safetensors, tokenizer files) are present              |

## Next steps

After successfully converting your checkpoint:

1. **Upload** the converted checkpoint to your GCS bucket
2. **Register** the checkpoint with a Model Manifest
3. **Deploy** the checkpoint using a Bundle configuration

See [Deploying custom checkpoints](/en/v1.2.0/sambastack/service-administration/deploy-custom-checkpoints) for the complete workflow.

For speculative decoding deployments, see [Deploying with speculative decoding](/en/v1.2.0/sambastack/service-administration/deploy-with-speculative-decoding) for draft-target validation and deployment instructions.