Run language example applications

From this tutorial you learn how to run a language application example on a SambaNova system and how to use application parameters.

Prerequisites

This tutorial assumes that:

  • You’ve downloaded the sambaflow package and checked that the SambaNova daemon is running.

  • Created a directory $HOME/sambaflow-apps and copied all the example applications into it. That way you will be able to modify them if necessary.

  • Created a directory $HOME/app-test where you will collect applications output: PEF files and logs.

See the Hello SambaFlow! tutorial.

In addition, download the following files to your $DATADIR before you compile and run the application,:

BERT model overview

BERT (Bidirectional Encoder Representations from Transformers) is a machine learning model based on Transformers that was developed by Google in 2018.

The original BERT implementation has two models:

  • BERT Base: 12 encoders, 12 bidirectional self-attention heads, 110 million parameters, 768 dimension size

  • BERT Large: 24 encoders, 16 bidirectional self-attention heads, 340 million parameters, 1024 dimension size

For more information about BERT, including an illustration, see the original paper.

BERT on RDU

We use the Transformers library from HuggingFace to run BERT models on a DataScale system.

Our scripts include significant modifications of the original scripts that ensure that the model can run on SambaNova RDU chips.

The commands below are used to run BERT models on a DataScale system.

Compile for training

The following set of command peforms some setup and then it compiles the model for training. It includes a significant number of arguments to compile to fine-tune the output.

The output of the training run is a PEF file, transformers_hook.pef, which we pass in when we start a training run in the next step.

$ export OUTDIR=$HOME/app-test
$ export DATADIR=$HOME/data/bert
$ mkdir -p $DATADIR
$ python $HOME/sambaflow-apps/nlp/transformers_on_rdu/transformers_hook.py compile \
  --tokenizer_name bert-large-uncased \
  --model_name_or_path bert-large-uncased \
  --do_eval \
  --do_lower_case \
  --data_dir $DATADIR \
  --max_seq_length 384 \
  --per_device_train_batch_size 32 \
  --per_device_eval_batch_size 32 \
  -b 32 \
  --output_dir=${OUTDIR}/hf_output_squad_compile \
  --overwrite_output_dir \
  --seed 1206287 \
  --task_name squad \
  --module_name squad \
  --mac-human-decision $HOME/sambaflow-apps/nlp/transformers_on_rdu/human_decisions/compiler_configs/faster_compile.json \
  --mac-v2 \
  --cache_dir ${OUTDIR}/squad_cache \
  --pef transformers_hook \
  --output-folder=${OUTDIR}

Initiate a training run

To initiate a training run, we have to call run and pass in parameters including the PEF file, transformers_hook.pef.

$ export OUTDIR=$HOME/app-test
$ export DATADIR=$HOME/data/bert
$ python $HOME/sambaflow-apps/nlp/transformers_on_rdu/transformers_hook.py run \
  --model_name_or_path bert-large-uncased \
  --tokenizer_name bert-large-uncased \
  --do_train \
  --do_eval \
  --do_lower_case \
  --data_dir $DATADIR \
  -p ${OUTDIR}/transformers_hook/transformers_hook.pef \
  --max_seq_length 384 \
  --per_device_train_batch_size 32 \
  --per_device_eval_batch_size 32 \
  -b 32 \
  --output_dir=${OUTDIR}/hf_output_squad_run  \
  --overwrite_output_dir \
  --seed 1206287 \
  --task_name squad \
  --module_name squad \
  --learning_rate 3e-05  \
  --eval_steps 1000 \
  --num_train_epochs 0.2 \
  --cache_dir ${OUTDIR}/squad_cache

Compile for inference

Compiling for inference is separate from compiling for a training run, and we name the PEF file transformers_hook_inf.

$ export OUTDIR=$HOME/app-test
$ export DATADIR=$HOME/data/bert
$ python $HOME/sambaflow-apps/nlp/transformers_on_rdu/transformers_hook.py compile \
  --inference \
  --model_name_or_path bert-large-uncased \
  --tokenizer_name bert-large-uncased \
  --do_eval \
  --do_lower_case \
  --data_dir $DATADIR \
  --max_seq_length 384 \
  --per_device_eval_batch_size 32 \
  -b 32 \
  --output_dir=${OUTDIR}/hf_output_squad_inference_compile \
  --overwrite_output_dir \
  --seed 1206287 \
  --task_name squad \
  --module_name squad \
  --mac-human-decision /opt/sambaflow/apps/nlp/transformers_on_rdu/human_decisions/compiler_configs/faster_compile.json \
  --mac-v2 \
  --cache_dir ${OUTDIR}/squad_cache \
  --pef transformers_hook_inf \
  --output-folder=${OUTDIR}

Run for inference

To run for inference, we specify --inference as an argument to run and specify the PEF file that we compiled for inference (transformers_hook_inf).

$ export OUTDIR=$HOME/app-test
$ export DATADIR=$HOME/data/bert
$ python $HOME/sambaflow-apps/nlp/transformers_on_rdu/transformers_hook.py run \
  --inference \
  --model_name_or_path ${OUTDIR}/hf_output_squad_run \
  --do_eval \
  --do_lower_case \
  --data_dir $DATADIR \
  -p ${OUTDIR}/transformers_hook_inf/transformers_hook_inf.pef \
  --max_seq_length 384 \
  --per_device_eval_batch_size 32 \
  -b 32 \
  --output_dir=${OUTDIR}/hf_output_squad_inference \
  --overwrite_output_dir \
  --seed 1206287 \
  --task_name squad \
  --module_name squad \
  --learning_rate 3e-05  \
  --eval_steps 6000 \
  --tokenizer_name bert-large-uncased \
  --per_device_train_batch_size 32 \
  --cache_dir ${OUTDIR}/squad_cache