Multi-Agent AI, Open Protocols & Startup Acceleration with Saurabh Tiwary | AI Basics

Key Concepts

• AI Agents: Autonomous entities that can perform tasks, make decisions, and interact with other agents or humans.
• Agent Development Kit (ADK): Google's open-source framework for building AI agents, offering tools for debugging, tracing, and quality assurance.
• Agent-to-Agent (A2A) Protocol: A standard protocol enabling interoperability between agents built on different platforms and frameworks.
• LLMs (Large Language Models): AI models trained on vast amounts of text data, capable of generating human-like text, translating languages, and answering questions.
• Multimodality: The ability of AI models to process and reason across different data types, such as text, images, audio, and video.
• Human-in-the-Loop: A design approach where humans intervene in AI processes to provide feedback, make decisions, or ensure safety and accuracy.

AI Agents: The Paradigm Shift

The discussion centers on the evolution of AI agents from simple chatbots to sophisticated entities capable of performing complex, indeterminate tasks.

• Early Definitions: Initially, agents were seen as LLMs wrapped with prompts or workflows codified through LLMs.
• Current Definition: Agents are now expected to make decisions and take actions based on the output of previous actions, enabling them to handle ambiguous tasks.
• Example: An agent could analyze emails, identify important ones, extract tasks, and execute them autonomously.

The Emergence of the Agent Future

The conversation explores when AI agents will become commonplace in everyday tasks.

• Early Foundations: The foundations are being laid as model quality improves, allowing for more complex tasks.
• Quality Threshold: Agents need to achieve a high level of accuracy (80-90%) before users will trust them with important tasks.
• Increasing Ambition: As agents become more reliable, users will delegate more complex tasks, driving further development.
• Real-World Applications: Companies are already deploying agents in production environments for tasks like normalizing founder updates for investors.

Building Trust and Infrastructure: ADK and A2A

The discussion shifts to the infrastructure and tools needed to build and deploy trustworthy AI agents.

• ADK (Agent Development Kit):
  • An open-source framework for building multi-agent systems.
  • Provides tools for debugging, tracing, and ensuring quality.
  • Not tied to Google; can be used with any LLM and deployed on any cloud or on-premise environment.
• A2A (Agent-to-Agent) Protocol:
  • Enables interoperability between agents built on different platforms.
  • Allows agents to communicate and collaborate regardless of their underlying technology.
  • Donated to the Linux Foundation for open governance.
  • Supported by major companies like Salesforce, Microsoft, and Amazon.
• Example: A billing agent could use the A2A protocol to communicate with Salesforce to identify the account executive for an unpaid invoice and then contact the customer's agent to verify payment status.

Human-in-the-Loop and Security Considerations

The conversation addresses the importance of human oversight and security in AI agent systems.

• Human Intervention: The ADK includes capabilities for human-in-the-loop, allowing users to intervene and provide feedback.
• Security Challenges: As agents gain more capabilities, security concerns similar to those in app ecosystems will arise.
• Permissions and Controls: It's crucial to establish permissions and controls to limit the actions agents can perform, even if they are considered trustworthy.
• Example: An agent should not be able to change an employee's pay slip or social security number.

The Future of AI Agents: Success Metrics and Multimodality

The discussion concludes with a vision of the future of AI agents and the key factors that will drive their success.

• Success Metrics: Success will be measured by the complexity of tasks agents can handle and their ability to orchestrate across multiple systems.
• Multimodality: The ability to process and reason across different data types (text, images, audio, video) will be crucial for agent effectiveness.
• Example: An agent could analyze a receipt image for expense approval to verify its authenticity.
• Real-World Example: Shopify is using multimodality to guide users through their storefront setup process by analyzing their screen and providing contextual instructions.
• Iterative Design: Providing agents with the ability to iteratively design and improve their work will unlock new opportunities.

Synthesis/Conclusion

The future of AI agents is rapidly evolving, driven by advancements in LLMs, open-source frameworks like ADK, and interoperability standards like the A2A protocol. As agents become more capable and trustworthy, they will automate increasingly complex tasks, freeing up humans to focus on higher-level activities. Multimodality and human-in-the-loop mechanisms will be essential for ensuring agent effectiveness, security, and alignment with human values. The journey towards fully autonomous agents will be iterative, with continuous improvements in model quality, infrastructure, and security protocols.