Stanford CS25: V5 I The Advent of AGI, Div Garg

Key Concepts

• AGI (Artificial General Intelligence): The concept of AI that can perform any intellectual task that a human being can.
• AI Agents: Software entities that can perceive their environment, reason, plan, and act autonomously to achieve specific goals.
• Human-like Agents vs. API Agents: Agents that interact with interfaces like humans versus agents that use direct API calls.
• Levels of Autonomy (L1-L5): A scale representing the degree of human involvement in agent operation, from co-pilot to fully autonomous.
• Agent Evaluation: Benchmarking and assessing agent performance in real-world scenarios.
• Agent Training: Using techniques like reinforcement learning to improve agent capabilities.
• Agent Communication: Protocols and methods for agents to interact and collaborate.
• Memory (Short-term, Long-term): The ability of agents to store and retrieve information for context and personalization.
• Personalization: Tailoring agent behavior to individual user preferences and needs.
• Model Context Protocol (MCP): A framework for standardizing API interactions to facilitate agent communication.
• Reliability: The consistency and trustworthiness of agent actions.
• Looping: When agents get stuck in repetitive, unproductive actions.

1. What Does AGI Look Like?

• AGI is an abstract concept without a clear definition or visualization.
• Questions to consider: Is it a supercomputer, an enhanced chatbot, a personal companion, or something embedded in daily life?

2. AI Agent Architecture

• Based on Lilian Wang's architecture, agents can be broken down into subpaths.
• Memory:
  • Short-term memory (e.g., chat window).
  • Long-term memory (e.g., user history, preferences).
• Tools:
  • Access to calculators, calendars, web search, coding environments, etc.
• Advanced Planning:
  • Reflection (failover mechanisms, error correction).
  • Self-criticism.
  • Decomposition (chains of thought, breaking down complex tasks).
• Actions:
  • Ability to act on the user's behalf.

3. AGI Inc. and Real-World Applications

• AGI Inc. is an AI lab focused on bringing AGI into everyday life.
• Example: An AI agent passing a real California DMV driving test.
  • The agent took the online test, answering 40 questions correctly.
  • The DMV was screen recording and monitoring the person on camera.
  • The agent successfully passed the exam, and the DMV sent a driving license afterward.
  • This was a white-hacking attempt, and the DMV was informed afterward.

4. Agent Evaluation, Training, and Communication

• Agent Evaluation: Establishing standards and benchmarks to assess agent performance on websites and use cases.
• Agent Training: Using reinforcement learning and other techniques to enable advanced planning, self-correction, and improvement.
• Agent Communication: Developing protocols for agents to communicate with each other.
  • Model Context Protocol (MCP).
  • Google's agent-to-agent communication protocol (A2A).
  • Open-source projects like Agent Protocol.

5. Why Build Agents?

• Agents are more efficient at interfacing with computers and the digital world than humans.
• Vision: An army of virtual assistants that can perform tasks on your behalf through a human interface.
• Large language models (LLMs) alone are not sufficient; action capabilities are needed.
• Building blocks include memory, actions, personalization, and internet access.

6. Human-like Agents vs. API Agents: Pros and Cons

• Human-like Agents:
  • Can operate interfaces designed for humans (web, keyboard, mouse).
  • Can become a digital extension of the user, learning their context and preferences.
  • Handle logins, payments, and interact with services without API restrictions.
  • Simple action space (click and type) allows for generalization and improvement.
• API Agents:
  • Easier to build, more controllable, and safer.
  • High API variability requires different agents for each API.
  • APIs can change, making agents unreliable.
• Direct Computer Control Agents:
  • Easier to take actions, more free-form interactions.
  • Hard to provide guarantees on behavior.

7. Levels of Autonomy (L1-L5)

• L1-L2 (Human in Control): Agent acts as a co-pilot (e.g., code editor like Cursor).
• L3 (Agent in Control, Human Fallback): Agent writes most of the code, human monitors and provides feedback (e.g., Cursor Composer).
• L4 (No Human in the Loop, Automated Fallback): Agent operates autonomously with remote monitoring (e.g., Waymo in SF).
• L5 (Fully Autonomous): No human involvement or monitoring.

8. Agent Evaluation: The Miniature Internet

• A miniature version of the internet with clones of the top 20 websites (really.xyz).
• Agents navigate interfaces on predefined tasks, and their performance is scored.
• Evaluation results for GPT-4o: Only 14% successful accuracy.
• Comparison of open-source frameworks:
  • Open Computer Use Model (Operator): Maximum 20% accuracy on some environments.
  • Stagehand, Agent Zero: Up to 50% success rate.
• Benchmark of different models:
  • Claude 3.5 Sonnet: Maximum 40% accuracy.
  • Gemini 1.5 Pro and 1.0 Ultra follow closely.
• Conclusion: Most models are not ready for real-world deployment due to low success rates.

9. Agent Training: Agent Q

• Agent Q is a self-improving agent that learns from corrections and planning.
• It combines Monte Carlo Tree Search (MCTS), self-critic mechanisms, and reinforcement learning (RL) techniques like Direct Preference Optimization (DPO).
• Monte Carlo Tree Search: Plans over the search space of tasks.
• Self-Critic Mechanisms: Self-verifies and gets feedback on mistakes.
• RLHF (Reinforcement Learning from Human Feedback): Uses DPO to improve the agent using preference data.
• Agent Q is available on arXiv as a research paper.
• Example: Booking a restaurant reservation on OpenTable.
  • The agent proposes actions, and a critic network ranks them.
  • The system optimizes to take the correct actions and improve.
• The agent backtracks and recovers from mistakes.
• Agent Q was tested in real-world scenarios, including OpenTable reservations.
• Results:
  • GPT-4: 62.6% accuracy.
  • DPO: 71% accuracy.
  • Agent Q (without MCTS): 81% accuracy.
  • Agent Q (with MCTS and DPO): 95.44% accuracy.
• Training takes less than one day to improve from 20% to 95.44% accuracy.

10. Memory and Personalization

• AI models can be viewed as processors that operate over natural language.
• The transformer model takes input prompts and outputs new language tokens.
• Memory systems (file disk, RAM) are needed to save and process information over time.
• Long-term memory is similar to a disk, where user context is saved and loaded on the fly.
• Mechanisms for long-term memory: Embeddings and retrieval models.
• Open questions: Hierarchy of memory, temporal persistence, adaptability.
• Personalization: Tailoring agents to user preferences.
• Explicit personalization: Actively collected information (e.g., allergies, favorite dishes).
• Implicit preferences: Learned from user behavior (e.g., brand preferences).
• Challenges: User privacy and trust.
• Methods for collecting information: Active learning (explicitly asking) and passive learning (recording user behavior).
• Learning to personalize: Supervised fine-tuning and human feedback (thumbs up/down).

11. Agent-to-Agent Communication

• Multi-agent architectures and systems involve coordinated communication between agents.
• Reasons for building multi-agent systems:
  • Paralization: Dividing tasks into smaller chunks.
  • Specialization: Routing tasks to specialized agents.
• Challenges:
  • Lossy communication: Information loss during agent-to-agent communication.
  • Building the right system of hierarchies.
  • Exchanging communication effectively.
  • Building syncing primitives.
• Frameworks:
  • Model Context Protocol (MCP): A simple wrapper around APIs for standardized communication.
  • Agent to Agent Protocol (Google): Allows agents to co-compose other agents.
• MCP abstracts APIs and makes them modular, allowing for dynamic tool discovery.

12. Issues with Agent Existence

• Key problems that need to be solved for agents to be practical:
  • Reliability: Systems need to be highly reliable (close to 99.9%).
  • Looping: Agents can get stuck in repetitive actions.
• Solutions:
  • Testing and benchmarking agents in real-world use cases.
  • Monitoring agents online.
  • Safety mechanisms: Audit trails and human overrides.

13. Q&A Highlights

• Accuracy: Reaching 99.9% accuracy is possible with reinforcement learning and training on agentic interfaces.
• Captchas: AI agents can solve captchas, leading to a cat-and-mouse game. Biometrics may be a better method of identity verification.
• Automating AI Agent Creation: Possible to automate the process of creating AI agents, especially for niche-specific tasks.
• Hallucination: Bigger models and more data can reduce hallucinations. Domain-specific testing and evaluation are important.
• Small vs. Large Language Models: Smaller models with better reasoning and processing can be effective for agents.
• Memory Implementation: Implementing a computer-like memory system (RAM, ROM, hard drive) for AI agents is an interesting question.

Synthesis/Conclusion

The lecture explores the current state and future potential of AI agents, emphasizing the need for reliable, trainable, and communicative agents that can effectively interact with the digital world. While significant progress has been made in areas like agent architecture, training methodologies, and communication protocols, challenges remain in achieving high reliability, preventing looping behavior, and ensuring user trust. The development of robust evaluation frameworks, domain-specific testing, and innovative memory systems will be crucial for realizing the full potential of AI agents and integrating them into everyday life.