Build AI agents with Cloud Run and Firebase Genkit

Key Concepts

AI Agents, APIs, Firebase Genkit, Tool Calling, Generative AI, Unified Interface, Model Integration, Observability, Zod, Schemas, Flows, Cloud Run, REST API, CORS, Deployment.

Genkit Overview

Genkit is a Firebase library designed to simplify the integration of generative AI into applications. It addresses the challenges of using multiple AI models from different providers and provides tools for defining and observing AI flows.

Key Features:

• Unified Interface: Genkit provides a unified interface for using various AI models, including Gemini, Gemma, and models from other providers via plugins like Ollama.
• Tool Definition: The defineTool() function allows developers to define tools that AI models can access, enabling them to read data or take actions.
• Observability: Genkit integrates with the Firebase console, providing observability tools to track the success and failure of AI flows.

Building a Weather App with Genkit: A Step-by-Step Example

The video demonstrates building a weather app using Genkit to illustrate its capabilities.

Step 1: Project Setup

• An Express application skeleton using TypeScript is used as the starting point.
• Genkit libraries are imported.
• A Genkit object is created, specifying the AI should run in the us-central1 region.

Step 2: Defining Input and Output Schemas

• inputSchema and outputSchema are defined using Zod syntax to specify the structure of the input and output data.
  • Zod: A schema declaration and validation library used to validate inputs and outputs.

Step 3: Implementing the AI Flow

• The ai.generate() function is called to configure the AI model.
  • Parameters include the model to use, temperature, system prompt, and user prompt.

Step 4: Local Testing

• The npx genkit start command launches a user interface for testing the AI flow.
• This allows developers to enter prompts and inspect the results without deploying the entire application.
• The dashboard provides a tight feedback loop for development.

Step 5: Defining a Tool for Weather Data

• The defineTool() function is used to define a tool for getting weather data for a given location.
  • Parameters include name, description, input schema, and output schema.
• A function is added to the tool that will be executed when Genkit invokes it.
  • In the example, the temperature is hardcoded to 68°F.

Step 6: Integrating the Tool into the AI Flow

• The tool is added to the tools array in the generate function.
• The AI will determine which tool to use based on the tool names and descriptions.

Step 7: Testing the Weather App

• The Genkit user interface is used to test the weather app.
• The AI is able to understand the prompt and call the weather tool to get the temperature.
• The "View Trace" feature allows developers to see the steps that were executed behind the scenes, including the calls to Gemini and the weather tool.

Deploying to Cloud Run

The video demonstrates how to deploy a Genkit AI agent to the cloud using Cloud Run.

Step 1: Installing the genkit-ai/express Package

• The genkit-ai/express package is installed to expose the flow as a public REST API endpoint.

Step 2: Importing and Calling startFlowServer()

• The startFlowServer() function is imported from the genkit-ai/express package.
• startFlowServer() is called to start the server and listen for incoming requests.
  • Parameters include the port to listen to, CORS options, and the flows to start.

Step 3: Configuring CORS

• CORS (Cross-Origin Resource Sharing) options are configured to allow requests from other domains.
  • In the example, CORS is opened up from all origins, but in a production app, it should be restricted to certain domains.

Step 4: Deploying to Cloud Run

• The gcloud run deploy command is used to deploy the application to Cloud Run.
  • The source code location is set to the current directory.
  • A name is chosen for the service (e.g., "serverless expeditions").
  • The region is selected (e.g., us-central1).
  • The service is opened up to anonymous users.

Step 5: Testing the Deployed Flow

• The deployed flow is tested using curl.
  • The request method is set to POST.
  • The URL is the URL where the code was deployed, followed by /mainflow.
  • The Content-Type header is set to application/json.
  • The payload includes a data object with a prompt containing the question.

Key Arguments and Perspectives

• Ease of Integration: Genkit simplifies the integration of generative AI into applications by providing a unified interface and tools for defining and observing AI flows.
• Rapid Development: The local development experience with Genkit allows for rapid testing and debugging of AI flows.
• Scalability and Cost-Effectiveness: Deploying Genkit AI agents to Cloud Run allows for scalability and cost-effectiveness, as you only pay when there is traffic.
• Declarative Code: The declarative nature of Genkit code makes it easier to get right.

Notable Quotes

• "Genkit is designed to bridge the gap between your AI models and real world applications." - Nohe
• "Declarative code is easier to get right." - Nohe

Technical Terms

• AI Agents: Software entities that can perceive their environment, make decisions, and take actions to achieve specific goals.
• APIs (Application Programming Interfaces): Sets of rules and specifications that software programs can follow to communicate with each other.
• Firebase Genkit: A Firebase library for integrating generative AI into applications.
• Tool Calling: The ability of an AI model to use external tools or APIs to gather information or perform actions.
• Generative AI: A type of artificial intelligence that can generate new content, such as text, images, or code.
• Unified Interface: A single interface for interacting with multiple AI models or tools.
• Observability: The ability to monitor and understand the behavior of a system.
• Zod: A schema declaration and validation library for TypeScript.
• Schemas: Structures that define the format and type of data.
• Flows: Sequences of steps or actions that an AI agent performs.
• Cloud Run: A fully managed compute platform for deploying and scaling containerized applications.
• REST API (Representational State Transfer Application Programming Interface): An architectural style for building web services.
• CORS (Cross-Origin Resource Sharing): A mechanism that allows restricted resources on a web page to be requested from another domain outside the domain from which the first resource was served.

Logical Connections

The video progresses logically from introducing Genkit and its features to demonstrating its use in building a weather app and deploying it to Cloud Run. Each step builds upon the previous one, providing a clear and comprehensive overview of the Genkit workflow.

Synthesis/Conclusion

Firebase Genkit simplifies the development and deployment of AI agents by providing a unified interface for multiple models, tools for defining actions, and observability features. The weather app example demonstrates how to use Genkit to build a practical application, and the deployment to Cloud Run shows how to make it accessible on the internet. Genkit empowers developers to integrate generative AI into their applications more easily and efficiently.