Use Python virtual environments

Python virtual environments allow you to install and manage Python packages separately from the system-level Python installation. Some of the advantages of using virtual environments include:

Avoiding package conflicts. You can install and manage Python packages independently of each other. This means that you can install different versions of the same package in different virtual environments without causing conflicts or compatibility issues.
Dependency management. Virtual environments make it easy to manage dependencies for your projects. You can create a requirements.txt file that lists all the required packages and their versions, making it easier to install the same dependencies on other machines.
Isolation. Virtual environments provide a sandboxed environment. This reduces the risk of accidentally installing or modifying packages that are needed by other applications or users on the same machine.

Virtual environments for sample applications

SambaNova engineers prepared virtual environments for example applications that are shipped with SambaFlow.

The application’s directory includes:

A pre-installed virtual environment that you can use to test the model without installing anything yourself.
A requirements.txt file that you can use if you want to create your own virtual environment and install the packages required by the model.

Each virtual environment provided by SambaNova includes only the packages that are necessary to run the corresponding example model. In many cases you need additional packages for your application, for example data processing packages.

To customize the virtual environment provided by SambaNova, follow these steps, discussed in detail in Prepare the virtual environment for the model:

Create your own virtual environment.
Install the packages required by the example model (and specified in requirements.txt).
Add other packages that you need.

This doc page explains how to set up and customize a virtual environment using Python venv. You can instead use another virtual environment tool.

Prepare the virtual environment for the model

Assume you want to use a SambaNova example model. In addition to the required packages, you want to add other Python packages. Let’s look at the /opt/sambaflow/apps/image/segmentation example and assume you want to add data processing packages for geophysics datasets.

Find the application’s code directory and list available models. For example, here’s how you look at image processing examples:
```
$ ls /opt/sambaflow/apps/image
```
Sample output:
```
classification  object_detection  segmentation  segmentation_3d
```
Copy the application code for the segmentation example into your home directory (or any other directory where you have write access).
```
$ mkdir -p $HOME/sambaflow-apps/image
$ cp -a /opt/sambaflow/apps/image/segmentation $HOME/sambaflow-apps/image/
```
Create a directory to hold all your virtual environments (if it doesn’t exist). We recommend creating it in $HOME/.virtualenvs because it’s used by tools like virtualenvwrapper by default.
```
$ mkdir -p $HOME/.virtualenvs
```
Create a virtual environment using the venv Python module.

Use a meaningful name for your virtual environment, e.g. the name of the model. Don’t use the generic name venv to avoid problems later.
```
$ python -m venv --system-site-packages $HOME/.virtualenvs/segmentation
```
Activate the virtual environment that you’ve just created.
```
$ source $HOME/.virtualenvs/segmentation/bin/activate
```
Upgrade the pip application:
```
$ pip install --upgrade pip
```
Install the required packages using the application’s requirements.txt file.
```
$ pip install -r $HOME/sambaflow-apps/image/segmentation/requirements.txt
```

Verify that you can compile the unmodified application code.

$ cd $HOME/sambaflow-apps/image/segmentation
$ python compile.py compile

Install additional packages that you need. For example, you need the segyio package to pre-process geophysics datasets.
```
$ pip install segyio
```

Now you are ready to work on your model.

Deactivate the virtual environment

After you are done with your model or when you want to switch to some other tasks, don’t forget to deactivate your virtual environment.

$ deactivate

Other virtual environment tools

You can use several methods to create your virtual environment:

Use the standard Python module: python -m venv
Use virtualenv
Use virtualenvwrapper
Use pyenv
Use Pipenv
and many others.

Here is a great article with details about virtual environments and how they work.