Introducing FunctionGemma

Key Concepts

• Function Gemma: A 270 million parameter language model fine-tuned for function calling.
• Function Calling: The ability of a language model to identify and execute specific functions or API calls based on natural language input.
• Fine-tuning: The process of further training a pre-trained model on a specific dataset to improve its performance on a particular task.
• Google AI Edge: A platform for deploying AI models on edge devices (mobile, embedded systems).
• On-device AI: Running AI models directly on a device, offering benefits like privacy, offline access, and reduced latency.

Introduction of Function Gemma

The presentation introduces Function Gemma, a new model derived from the Gemma family, specifically designed for function calling. It’s a 270 million parameter model, making it the smallest in the Gemma lineup, but optimized to translate natural language into executable function calls and API actions. The core benefit highlighted is its ability to deliver performance comparable to much larger models, particularly after fine-tuning. Function Gemma is targeted towards developers aiming to create fast, private, and cost-effective applications.

Performance and Advantages

Despite its small size, Function Gemma demonstrates significant processing speed, even on resource-constrained hardware like embedded and mobile devices. Utilizing accelerators like GPUs and NPUs further enhances this speed. A key advantage is the potential for on-device execution, enabling applications to function offline and preserve user privacy, while also reducing reliance on cloud computing and associated costs. The developers observed improvements in accuracy when utilizing the new function calling format compared to simply prompting the base Gemma 3 27 model. Further accuracy gains were achieved through fine-tuning.

Mobile Actions Demo: On-Device Function Execution

A demonstration app, “Mobile Actions,” showcases Function Gemma’s capabilities. This app allows users to control device functions (e.g., creating calendar events, adding contacts, turning on the flashlight) using voice or text commands. The model is fine-tuned on a limited set of on-device functions, effectively acting as a specialized tool. The demo illustrates the model’s ability to parse natural language, identify the correct function, and execute the corresponding action. The sequence shows the model interpreting commands and requesting the appropriate function for execution.

Interactive Mini-Game Demo: Game Mechanics Control

Another demo features an interactive mini-game where players manage a virtual plot of land using voice commands. Commands like “plant sunflowers in the top row and water them” are interpreted by Function Gemma, which then selects the appropriate app functions (e.g., “plant crop,” “water crop”) along with specific grid coordinates. This demonstrates the model’s ability to translate free-form input into precise game logic execution.

Use Cases and Applications

Function Gemma unlocks a broad range of use cases beyond simple chat interactions. These include data lookup based on user queries, intelligent routing of queries to specialized sub-agents, and novel interaction modalities for games and applications. The ability to generate function calls and arguments from natural language input is central to these applications.

Fine-tuning and Accessibility

The model is explicitly designed for fine-tuning, allowing developers to create highly specialized function calling models tailored to their specific AI workflows. A step-by-step recipe for fine-tuning is publicly available. Function Gemma is accessible through common platforms like Hugging Face, Kaggle, and Vertex AI. It’s compatible with popular tools and frameworks including Hugging Face Transformers, O Lama, VLM, Llama CPP, Light RT, and MLX. Fine-tuning can be performed using tools like TRL, Unsloth, and Vertex AI. Developers are encouraged to consult the Gemma cookbook for guides and examples on function calling format and best practices.

Notable Quote

“Function Gemma is small enough to operate responsibly on consumer hardware, unlocking new use cases, as well as the usual benefits of running AI on device such as privacy, offline capabilities, and reduce cloud costs.” – Presenter, regarding the benefits of on-device AI.

Technical Terms

• Parameters: The adjustable variables within a machine learning model that are learned during training. A higher number of parameters generally indicates a more complex model. (Function Gemma has 270 million parameters).
• API (Application Programming Interface): A set of rules and specifications that allow different software applications to communicate with each other.
• NPU (Neural Processing Unit): A specialized hardware accelerator designed to accelerate machine learning tasks.
• GPU (Graphics Processing Unit): A specialized electronic circuit designed to rapidly manipulate and display computer graphics. Also used for accelerating machine learning tasks.
• TRL (Transformer Reinforcement Learning): A library for training language models with reinforcement learning.
• Unsloth: A framework designed for efficient fine-tuning of large language models.
• Vertex AI: Google Cloud’s machine learning platform.

Logical Connections

The presentation follows a logical progression: introduction of Function Gemma, explanation of its advantages (speed, privacy, cost), demonstration of its capabilities through real-world examples (Mobile Actions, mini-game), discussion of potential use cases, and finally, details on accessibility and fine-tuning. The demos serve to concretely illustrate the theoretical benefits outlined earlier.

Conclusion

Function Gemma represents a significant step towards bringing powerful function calling capabilities to edge devices. Its small size, combined with its fine-tuning potential, makes it a versatile tool for developers seeking to build fast, private, and cost-effective AI applications. The availability of resources and compatibility with popular frameworks further lowers the barrier to entry, encouraging experimentation and innovation in the field of on-device AI.