Run language example applications
From this tutorial you learn how to run a language application example on a SambaNova system and how to use application parameters.
Prerequisites
This tutorial assumes that:
-
You’ve downloaded the
sambaflow
package and checked that the SambaNova daemon is running. -
Created a directory
$HOME/sambaflow-apps
and copied all the example applications into it. That way you will be able to modify them if necessary. -
Created a directory
$HOME/app-test
where you will collect applications output: PEF files and logs.
See the Hello SambaFlow! tutorial.
In addition, download the following files to your $DATADIR
before you compile and run the application,:
-
train-v1.1.json
: https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json -
dev-v1.1.json
: https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json
BERT model overview
BERT (Bidirectional Encoder Representations from Transformers) is a machine learning model based on Transformers that was developed by Google in 2018.
The original BERT implementation has two models:
-
BERT Base: 12 encoders, 12 bidirectional self-attention heads, 110 million parameters, 768 dimension size
-
BERT Large: 24 encoders, 16 bidirectional self-attention heads, 340 million parameters, 1024 dimension size
For more information about BERT, including an illustration, see the original paper.
BERT on RDU
We use the Transformers library from HuggingFace to run BERT models on a DataScale system.
Our scripts include significant modifications of the original scripts that ensure that the model can run on SambaNova RDU chips. |
The commands below are used to run BERT models on a DataScale system.
Compile for training
The following set of command peforms some setup and then it compiles the model for training. It includes a significant number of arguments to compile
to fine-tune the output.
The output of the training run is a PEF file, transformers_hook.pef
, which we pass in when we start a training run in the next step.
$ export OUTDIR=$HOME/app-test
$ export DATADIR=$HOME/data/bert
$ mkdir -p $DATADIR
$ python $HOME/sambaflow-apps/nlp/transformers_on_rdu/transformers_hook.py compile \
--tokenizer_name bert-large-uncased \
--model_name_or_path bert-large-uncased \
--do_eval \
--do_lower_case \
--data_dir $DATADIR \
--max_seq_length 384 \
--per_device_train_batch_size 32 \
--per_device_eval_batch_size 32 \
-b 32 \
--output_dir=${OUTDIR}/hf_output_squad_compile \
--overwrite_output_dir \
--seed 1206287 \
--task_name squad \
--module_name squad \
--mac-human-decision $HOME/sambaflow-apps/nlp/transformers_on_rdu/human_decisions/compiler_configs/faster_compile.json \
--mac-v2 \
--cache_dir ${OUTDIR}/squad_cache \
--pef transformers_hook \
--output-folder=${OUTDIR}
Initiate a training run
To initiate a training run, we have to call run
and pass in parameters including the PEF file, transformers_hook.pef
.
$ export OUTDIR=$HOME/app-test
$ export DATADIR=$HOME/data/bert
$ python $HOME/sambaflow-apps/nlp/transformers_on_rdu/transformers_hook.py run \
--model_name_or_path bert-large-uncased \
--tokenizer_name bert-large-uncased \
--do_train \
--do_eval \
--do_lower_case \
--data_dir $DATADIR \
-p ${OUTDIR}/transformers_hook/transformers_hook.pef \
--max_seq_length 384 \
--per_device_train_batch_size 32 \
--per_device_eval_batch_size 32 \
-b 32 \
--output_dir=${OUTDIR}/hf_output_squad_run \
--overwrite_output_dir \
--seed 1206287 \
--task_name squad \
--module_name squad \
--learning_rate 3e-05 \
--eval_steps 1000 \
--num_train_epochs 0.2 \
--cache_dir ${OUTDIR}/squad_cache
Compile for inference
Compiling for inference is separate from compiling for a training run, and we name the PEF file transformers_hook_inf
.
$ export OUTDIR=$HOME/app-test
$ export DATADIR=$HOME/data/bert
$ python $HOME/sambaflow-apps/nlp/transformers_on_rdu/transformers_hook.py compile \
--inference \
--model_name_or_path bert-large-uncased \
--tokenizer_name bert-large-uncased \
--do_eval \
--do_lower_case \
--data_dir $DATADIR \
--max_seq_length 384 \
--per_device_eval_batch_size 32 \
-b 32 \
--output_dir=${OUTDIR}/hf_output_squad_inference_compile \
--overwrite_output_dir \
--seed 1206287 \
--task_name squad \
--module_name squad \
--mac-human-decision /opt/sambaflow/apps/nlp/transformers_on_rdu/human_decisions/compiler_configs/faster_compile.json \
--mac-v2 \
--cache_dir ${OUTDIR}/squad_cache \
--pef transformers_hook_inf \
--output-folder=${OUTDIR}
Run for inference
To run for inference, we specify --inference
as an argument to run
and specify the PEF file that we compiled for inference (transformers_hook_inf
).
$ export OUTDIR=$HOME/app-test
$ export DATADIR=$HOME/data/bert
$ python $HOME/sambaflow-apps/nlp/transformers_on_rdu/transformers_hook.py run \
--inference \
--model_name_or_path ${OUTDIR}/hf_output_squad_run \
--do_eval \
--do_lower_case \
--data_dir $DATADIR \
-p ${OUTDIR}/transformers_hook_inf/transformers_hook_inf.pef \
--max_seq_length 384 \
--per_device_eval_batch_size 32 \
-b 32 \
--output_dir=${OUTDIR}/hf_output_squad_inference \
--overwrite_output_dir \
--seed 1206287 \
--task_name squad \
--module_name squad \
--learning_rate 3e-05 \
--eval_steps 6000 \
--tokenizer_name bert-large-uncased \
--per_device_train_batch_size 32 \
--cache_dir ${OUTDIR}/squad_cache