Don’t Build Multi-Agents, Here’s Why

Key Concepts

Multi-agent systems, AI agents, reliability, context engineering, prompt engineering, linear agents, context compression, OpenAI Swarm, Microsoft Autogen, CrewAI, Devon, Vectal, single-threaded agents, parallel processing, implicit decisions, context overflow, code sub-agents, anthropomorphizing agents.

Why Multi-Agent Systems Are Often a Trap

The video discusses why building multi-agent systems, despite their initial appeal, often leads to unreliable AI applications. It references an article by Cognition AI, the company behind the AI agent Devon, titled "Don't Build Multi-Agents." The core argument is that the complexity introduced by multiple agents working in parallel often reduces the overall reliability of the system. The video highlights that frameworks like OpenAI Swarm and Microsoft Autogen, while tempting, can lead developers down a problematic path.

Example: The video uses the example of building a "Flappy Bird" clone to illustrate the problem. If the task is divided between two agents – one building the background and the other the bird – they might create assets with conflicting visual styles, leading to a disjointed final product.

Two Key Principles for Building Reliable AI Agents

The Cognition AI article emphasizes two crucial principles:

1. Share Context: All agents must have access to the full context of the task, including the history of interactions and decisions made by other agents.
2. Actions Carry Implicit Decisions: Every action taken by an agent implies a decision, and conflicting decisions lead to poor results.

Analogy: The video draws an analogy to the early days of web development, comparing the current state of AI agent development to the era of HTML and CSS before the emergence of frameworks like React. React's success is attributed to its philosophical approach to front-end development, something currently lacking in the AI agent space.

Unreliable Multi-Agent Architectures (with Code Examples)

The video presents two common but unreliable multi-agent architectures, illustrated with code examples from a GitHub repository (linked in the video description):

1. Parallel Sub-Agents with Limited Context: The main agent breaks down the task into subtasks and delegates them to sub-agents that operate in parallel with minimal or no shared context. The main agent then attempts to combine the results. This is highly fragile because sub-agents can make conflicting assumptions.
   • Code Example: The video shows a Python code snippet where a task generation agent creates two subtasks (develop a storyline and create a visual style for a mobile game). Two sub-agents then execute these tasks in parallel, resulting in inconsistent outputs (a vibrant futuristic city vs. dark gritty tones for a horror game).
2. Parallel Sub-Agents with Shared Context (Partial Improvement): This architecture improves upon the first by providing each sub-agent with the context of the conversation and actions so far. However, the sub-agents still operate in parallel and are unaware of each other's reasoning and actions, leading to potential inconsistencies.

A Reliable Multi-Agent Architecture: Linear Agents

The video advocates for a single-threaded linear agent architecture as the most reliable approach.

Process:

1. A task is received by the main agent.
2. The main agent calls the first sub-agent, providing the full context.
3. The first sub-agent performs its task and adds its context to the overall context.
4. The main agent then calls the second sub-agent, providing the original context plus the context from the first sub-agent.
5. This process continues linearly, ensuring that each agent has complete awareness of the previous agents' actions and reasoning.
6. Finally, the main agent combines the results from all sub-agents.

Code Example: The video demonstrates a Python code snippet where the first sub-agent is executed, and then the second sub-agent is executed after the first one has completed, ensuring that the second agent has access to the first agent's output.

Reliable Architecture for Longer Tasks: Context Compression

For very large tasks where the context window might overflow, the video introduces a more advanced architecture that incorporates context compression.

Process:

1. The main agent breaks down the task.
2. A context compression LLM runs in the background, summarizing the conversation in real-time.
3. Each sub-agent receives the compressed conversation and compressed actions of the previous agents.

Benefits: This approach allows for much longer-running and more complex tasks by reducing the amount of context that each agent needs to process.

Caveats: The video emphasizes that this approach is significantly harder to implement correctly and should only be considered for tasks that truly require it.

Code Example: The video shows a Python code snippet that includes a "compressor agent" that summarizes the conversation to less than 200 words.

Applying the Principles: Clothe Code Example

The video cites Clothe Code, an AI agent from Anthropic, as an example of a system that adheres to these principles. Clothe Code uses sub-agents, but they never work in parallel. The sub-agents are typically tasked with answering specific questions, and only the main agent writes code. This approach ensures that the sub-agents' investigative work does not clutter the main agent's conversation history.

Why Agents Aren't Humans

The video cautions against anthropomorphizing AI agents. While humans can effectively communicate and resolve disagreements through dialogue, current LLMs are not as capable in this area. Running multiple agents in collaboration can lead to fragile systems where decision-making is too dispersed and context is not adequately shared.

Conclusion

The video concludes by reiterating the importance of choosing the right agent architecture and emphasizes that a simple, linear architecture is often the most reliable choice. It encourages viewers to prioritize context sharing and avoid complex parallel architectures unless absolutely necessary. The video also promotes the idea of building AI startups, highlighting the opportunities available in the current AI landscape.