Brunch, automated: How to architect AI agents for everyday tasks

Key Concepts

• Agent Architecture
• Prompt Augmentation
• Long-Term and Session Memory
• Separation of Concerns
• Tool Usage
• Reservations Agent
• Menu Evaluation Tool
• Invite Tool
• Process Engineering

Agent Architecture for Brunch Planning

The video discusses the architecture of an AI agent designed to automate the process of planning a brunch with friends, considering their dietary restrictions and preferences. The agent leverages various tools and techniques to handle different aspects of the planning process, from determining the date to sending out invitations.

Inputs and Initial Prompt

The agent receives a text-based input, such as: "Let's go to brunch on Sunday. I want to invite my friends, Alice, Bob, and Charlie." This input is then broken down into its constituent parts.

Handling Date and Time

• Brunch: The agent assumes the LLM understands the concept and typical timing of brunch.
• Sunday: The agent needs to determine which Sunday is being referred to.
  • Prompt Augmentation: The current date is added to the prompt using date.now or date.new to ground the LLM in reality.
  • User Clarification: The agent is designed to engage in back-and-forth communication with the user to clarify ambiguous information, such as which Sunday.

Reservations Agent

• Functionality: Handles tasks related to making reservations, including checking availability, retrieving menus, making the reservation, and sharing details with the user.
• Self-Contained Nature: The reservations agent is designed as a self-contained unit with a specific job, making it reusable.
• Agent Definition: An agent is defined as code with a specific job and an LLM, if needed.

Dietary Restrictions and Menu Evaluation

• Memory and Context: The agent utilizes both long-term and session memory to store and retrieve information about friends' dietary restrictions.
• Eval Menu Tool: A dedicated tool that takes a menu and a set of dietary restrictions as input and outputs a score indicating how well the menu accommodates those restrictions. It also generates a description of suitable options for individuals with those restrictions.
• Separation of Concerns: The eval menu tool is separated from the reservations agent because the reservations agent only needs to know the number of people attending, not their specific dietary needs. The hosting agent is responsible for ensuring everyone has suitable options.

Invite Tool

• Functionality: Sends out invitations to friends based on their preferred communication methods (text, email, calendar invite).
• Long-Term Memory Access: The tool accesses long-term memory to retrieve contact preferences for each friend.
• Integration: The tool integrates with the user's mail and calendar to send out the invitations.

Technical Pieces Summary

• Prompt Augmentation: Adding the current date to the prompt.
• Memory and Context: Using long-term and session memory to improve results and ensure dietary needs are met.
• Agent Extraction: Creating a self-contained reservations agent as a reusable service.
• Separation of Concerns: Dividing responsibilities between agents and tools.
• Menu Evaluation: Using a dedicated tool to assess menu suitability.
• Invite Tool: Enabling the agent to interact with the world and send out invitations.

Process Engineering

The video emphasizes that building AI agents is similar to process engineering. The process involves:

1. Listing out all the steps involved in the task (e.g., planning a brunch).
2. Logically breaking down the task into agents and tools.
3. Coding up the agents and tools to automate the process.

Conclusion

The video provides a detailed overview of the architecture of an AI agent for brunch planning, highlighting the use of various tools, techniques, and design principles. The agent leverages prompt augmentation, memory, separation of concerns, and specialized tools to automate the complex task of coordinating a meal with friends while considering their individual needs and preferences. The key takeaway is that building effective AI agents involves a process-oriented approach, breaking down complex tasks into manageable components and implementing them using appropriate tools and techniques.