Gemma: Pretrained and instruction-tuned models

Key Concepts:

• Pre-training: Training a model to predict the next token in a sequence using cross-entropy loss.
• Instruction Tuning (IT): Fine-tuning a pre-trained model to behave like a chatbot, following instructions and adopting a helpful persona.
• Next Token Prediction: The core function of a pre-trained language model, predicting the most probable next word or token.
• Cross-Entropy Loss: A loss function used in pre-training to measure the difference between the predicted token and the actual token.
• BOS (Beginning of Sequence) Token: A special token used to indicate the start of an input sequence.
• EOS (End of Sequence) Token: A special token used to indicate the end of a generated sequence.
• Turn Markers: Special tokens used in instruction-tuned models to differentiate between user input and the model's response.
• Fine-tuning: Adapting a pre-trained model to a specific task or dataset.

1. Pre-training Phase

• Main Goal: To train the model to predict the next token in a sequence.
• Method: Uses cross-entropy loss to teach the model next token prediction on a massive dataset of text and images.
• Process: The model compresses world information from the data into its weights, implicitly learning representations of the knowledge.
• Output: The model becomes a next token predictor. Given a piece of text, it outputs the next most probable token based on its training.

2. Instruction Tuning Phase

• Main Goal: To transform the pre-trained model into a chatbot.
• Method: Fine-tuning the model with specific data and formatting to answer user queries effectively.
• Process: Shapes the model's behavior and personality, adopting a helpful and safe persona. Teaches it to reason and answer in ambiguous situations.
• Capabilities: The model learns to follow instructions, format answers in specific ways (e.g., JSON output), write in different languages, and adhere to specific constraints (e.g., paragraph count).

3. Choosing Between Pre-trained and Instruction-Tuned Models

• Instruction-Tuned Model:
  • Use Case: When a ready-to-use model is needed that can understand instructions, answer questions, and is capable in areas like math or coding out of the box.
  • Advantage for Fine-tuning: Provides a performance head start for tasks aligning with general instruction following, such as creating a specialized Q&A bot, especially if using Gemma's existing conversational format.
• Pre-trained Model:
  • Use Case: When fine-tuning is required for a specialized task with a completely different output format or a niche task far removed from typical interactions.
  • Advantage: Offers a cleaner slate for shaping the model for unique needs without unlearning instruction-tuned behavior.
  • Example: Fine-tuning a model to translate dolphin language to English or generate highly specific scientific data.

4. Technical Tips: Special Tokens

• Pre-trained Model:
  • BOS Token: Used to start the input sequence.
  • EOS Token: Indicates the end of the generated sequence.
• Instruction-Tuned Model:
  • BOS Token: Used to start the input sequence.
  • Turn Markers: Used to differentiate between user input and the model's response.
  • End of T Token: Indicates the end of the generated sequence (not the EOS token).
• Note: The handling of these tokens may be managed behind the scenes by the library used to run Gemma.

5. Conclusion

• Gemma offers both pre-trained and instruction-tuned models as open-source tools for developers.
• Developers are encouraged to explore, experiment, and build products with Gemma.

Notable Quotes:

• "During this phase, the model compresses the whole world's information using text and images we feed it into its weights."
• "Using specific data and formatting, we make the model answer the user query rather than just output the next most probable answer."
• "It provides a cleaner slate, allowing you for more flexibility to shape it for your unique needs without having to unlearn the IT behavior."