Build personal AI Assistants as Google Workspace Add-ons, in Gmail, Calendar, etc

Key Concepts

• Google Workspace Add-ons: Extensions that integrate custom functionality across multiple Workspace apps (Gmail, Calendar, Drive, Chat).
• AI Agents: Intelligent software components (built via Vertex AI Agent Engine) that automate workflows and provide context-aware assistance.
• Cloud Run: A managed compute platform used to host the HTTP endpoints for Workspace add-ons.
• Vertex AI Agent Engine: The platform used to build and manage the logic for the AI agents.
• Grounding: The process of connecting an AI model to external data sources (like Google Search) to provide accurate, verifiable information.
• Multimodality: The capability of models (like Gemini) to process and understand different types of data, such as text and images.

1. Overview of Workspace Add-ons

Google Workspace add-ons allow developers to create unified experiences that function across the entire Workspace ecosystem. A single add-on can be accessed via a sidebar in Gmail, Calendar, or Drive, and as a dedicated app within Google Chat. Administrators can deploy these add-ons across an entire organization, ensuring consistent access for all users.

2. Real-World Applications and Use Cases

The video categorizes Workspace add-ons into two primary deployment models:

• 2P (Second Party): Organizations build custom add-ons for internal use.
  • Example: An internal tool for investigating system incidents or an automated assistant that aggregates data to help employees respond to customer support emails more efficiently.
• 3P (Third Party): Businesses build add-ons to be distributed to other organizations.
  • Example: ServiceNow provides a virtual agent for resolving IT issues directly within Google Chat.
  • Example: Figma allows users to manage diagram comments, preview images, and receive notifications directly within Chat without switching contexts.

3. Technical Architecture and Development Process

The integration follows a specific architectural flow:

1. User Interaction: The user sends a message or triggers an action within a Workspace app (e.g., Gmail or Chat).
2. HTTP Request: Google sends an event to an HTTP endpoint hosted on Cloud Run.
3. Context Extraction: The Cloud Run service parses the event, extracts user identity, and retrieves relevant context (e.g., the body of an open email).
4. Agent Processing: The service forwards the prompt and context to an agent built on the Vertex AI Agent Engine.
5. Response: The agent processes the request (using Gemini for reasoning and Google Search for grounding) and returns the result to the user.

Deployment Steps:

• Develop: Write the backend logic in a service (e.g., Python/Node.js) and deploy it to Cloud Run.
• Configure: Use the Google Cloud Console to enable the Google Chat API, define the app’s metadata (name, avatar), and set the HTTP endpoint URL.
• Publish: For internal or public distribution, the add-on must go through a publishing process, often involving coordination with IT administrators.

4. Key Arguments and Perspectives

• Context Preservation: A major advantage of this framework is that the AI agent maintains state across different applications. A user can start a task in Gmail and finish it in Chat without losing the conversation history or context.
• Grounding and Accuracy: By utilizing Google Search for grounding, agents can provide citations and sources, which is critical for professional tasks like travel planning or compliance checks.
• Multimodal Capabilities: Leveraging models like Gemini allows agents to handle complex inputs, such as images, which significantly expands the utility of automated workflows.

5. Notable Quotes

• "It's a game-changer that we can now build add-ons that work across all those [Workspace apps] at once, including Chat." — Pierrick, Developer Relations Engineer.
• "The agent relies on Google Search for grounding, so it can display the related sources for further research." — Pierrick, regarding the importance of verifiable AI responses.

6. Synthesis and Main Takeaways

The integration of AI agents into Google Workspace via add-ons represents a shift toward "context-aware" automation. By utilizing Cloud Run and Vertex AI, developers can build sophisticated, cross-platform tools that reduce context switching for end-users.

Summary of Takeaways:

1. Productivity: Workspace add-ons extend the platform to streamline complex team workflows.
2. Flexibility: Developers can build for internal organizational needs (2P) or for the broader market (3P).
3. Unified Experience: Add-ons provide a seamless interface across Gmail, Chat, Calendar, and Drive.
4. Scalable Infrastructure: The combination of Cloud Run and Vertex AI provides a robust, scalable foundation for deploying intelligent, agentic applications.