Get started with Gemma 3 on Vertex AI

Gemma 3 on Vertex AI: A Comprehensive Summary

Key Concepts:

• Gemma 3: Google's new family of open models, building on Gemini 2.0 research.
• Vertex AI: Google Cloud's platform for deploying, tuning, and serving AI models.
• Model Garden: A curated collection of open models on Vertex AI, including Gemma 3.
• Finetuning: Adapting a pre-trained model to a specific task or dataset.
• LoRA (Low-Rank Adaptation): A Parameter Efficient Finetuning Technique (PEFT) that reduces training time and compute requirements.
• JSONL: A file format where each line is a JSON object, used for training data in Vertex AI.
• Endpoints: A deployable service that serves the model for predictions.

1. Introduction to Gemma 3 and Challenges

• Gemma 3 is a family of open models from Google, based on the research behind Gemini 2.0.
• Models come in four sizes: 1, 4, 12, and 27 billion parameters, allowing deployment from smartphones to the cloud.
• Larger Gemma 3 models are multimodal, understanding text and images, handling multiple languages, and processing up to 128,000 tokens.
• Deploying and tuning these models can be challenging due to infrastructure and framework complexities.

2. Vertex AI as a Solution

• Vertex AI provides access to Google Cloud's infrastructure, including GPUs and TPUs, for adapting and serving models.
• The Vertex AI Model Garden offers a curated collection of open models, including Gemma 3, verified and ready for use.
• Integration with Vertex AI Training and Prediction simplifies tuning and deployment.
• Vertex AI abstracts away infrastructure complexity, allowing developers to focus on application building.

3. Deploying Gemma 3 with Vertex AI Model Garden SDK

• Deployment can be achieved with just three lines of code using the Vertex AI Model Garden SDK.
• Step 1: Import the Model Garden library.
• Step 2: Specify the desired Gemma 3 model (e.g., the 27 billion parameter Instruction Tuned version).
• Step 3: Call the deploy method.
• This command sets up the necessary infrastructure and creates a prediction endpoint.
• The demo uses a Colab Enterprise notebook for deployment.
• The deployed model can be used for predictions by calling the predict method with a prompt.
• Vertex AI provides a ChatCompletion API-compatible endpoint for multimodal tasks.
• Example: Sending an image URL and a text prompt ("Describe this image in two sentences") to the model.

4. Finetuning Gemma 3 on Vertex AI

• Finetuning allows modifying model behavior for specific languages, industry jargon, or classification tasks.
• Gemma models are open, allowing modification of weights through finetuning.
• Finetuning Techniques:
  • Full Finetuning: Updates all model weights, computationally intensive.
  • Parameter Efficient Finetuning (PEFT): Techniques like LoRA or QLoRA, which train a small number of additional parameters while keeping the original model frozen.
• Vertex AI provides built-in support for finetuning Gemma 3 using LoRA.
• Training data must be in JSONL format, with each line representing a training example.
• Example JSONL structure: {"role": "system", "content": "Overall context and instruction"}, {"role": "user", "content": "Input prompt"}, {"role": "assistant", "content": "Desired output"}

5. Finetuning Demo: UI-First Approach

• Access Gemma 3 models in Model Garden.
• Two options for finetuning:
  • Code Approach: Via the "Open Notebook" option, providing maximum flexibility.
  • UI-First Approach: Via the "Finetune" button, simplifying the process.
• Steps in the UI-First Approach:
  • Give the model a name.
  • Select the base model (e.g., Gemma 3 1b).
    • Instruction Tuned models are for keeping conversational capabilities.
    • PT models are for teaching a complete new task.
  • Select an output directory for storing model artifacts.
  • Configure finetuning hyperparameters:
    • Epochs: Number of times the model sees the entire dataset.
    • Learning Rate: Controls how much the model weights are adjusted.
  • Enable automatic deployment.
• Finetuning jobs can be monitored in "training pipelines."
• Once finetuning is complete, the model endpoint is available in "online prediction."
• Example: Finetuning the Gemma 3 1b model took 45 minutes.

6. Conclusion

• Vertex AI simplifies the deployment and finetuning of Gemma 3 models.
• Deployment can be done with minimal code using the Vertex AI Model Garden SDK.
• Finetuning can be achieved through both code-based and UI-driven approaches.
• Vertex AI abstracts away infrastructure complexity, allowing developers to focus on building AI applications.