Language models

Language models icon Based on the GPT architecture, SambaStudio’s transformer language models enable computers to understand text by recognizing learned patterns. By implementing computational techniques and theories of linguistics, our language models are able to understand, generate, translate, analyze, and interpret natural language texts.

Model architecture

The table below describes the ML App and architecture for the SambaStudio provided language models. The linked document for each model provides the following information:

  • Data preparation

  • Prompt guidelines

  • Usage guidelines

  • Hyperparameter and settings

  • Inference settings

Filed of application ML App Architecture Models

Language

Generative Tuning 13B

GPT 13B

GPT 13B Human Aligned (HA) models

Language

Generative Tuning 13B

GPT 13B

GPT 13B Instruction Tuned (IT) models

Language

Bloom 176B

Bloom 176B

BLOOMChat

Data preparation

The Generative data preparation document describes how to access the generative data preparation package, used for training all SambaStudio language models, on the SambaNova public GitHub External link. Additionally, each language model’s document contains data preparation instructions specific to that model.