The Multi Agent Debate, Who is Right?

Key Concepts

Multi-agent systems, single-agent systems, orchestrator, sub-agents, aggregator, context sharing, handoff principle, context overflow, compression LLM, context engineering, prompt engineering, tool design, evaluation, LLM as a judge, emergent behaviors, rainbow deployment, synchronous execution.

Multi-Agent vs. Single-Agent Systems: Two Perspectives

The video discusses two contrasting approaches to building agentic systems, highlighting the debate between multi-agent and single-agent architectures. It summarizes two articles: one from Anthropic advocating for multi-agent systems in specific contexts, and another from Cognition Labs (creators of Devon) suggesting that single-agent systems might be more effective for certain tasks.

Cognition Labs: The Case Against Multi-Agent Systems

Cognition Labs argues that the traditional multi-agent approach, involving an orchestrator dividing tasks among independent sub-agents and an aggregator combining results, suffers from two key issues:

1. Lack of Inter-Agent Visibility: Sub-agents operate in isolation, unaware of each other's progress or context.
2. Error Propagation: Misunderstandings by one sub-agent can negatively impact the aggregator's ability to produce coherent results.

Example: Building a Flappy Bird clone with independent sub-agents potentially leading to visually inconsistent elements.

To address these issues, Cognition Labs proposes a sequential approach where a single agent breaks down the task into subtasks and executes them sequentially, updating its memory after each subtask. This ensures context consistency and avoids the coordination challenges of multi-agent systems.

Addressing Context Overflow: For long-running tasks, they suggest using a "compression LLM" running in parallel to compress the chat history and memory, providing a concise context for the agent.

Context Engineering: They introduce the concept of "context engineering," an extension of prompt engineering focused on ensuring agents have access to the necessary context of past actions and responses.

Cloud Code Example: They cite Anthropic's Cloud Code as an example of a system that creates subtasks for itself but executes them sequentially within a single agent.

Coding Task Example: They found that a single model making decisions and applying code edits in one action was better than using separate models for proposing changes and applying them.

Anthropic: The Case for Multi-Agent Systems

Anthropic, in their article, presents a case for multi-agent systems, particularly within their research system available in Claude. They found that a multi-agent system with Claude Opus 4 as the lead agent and Claude Solid 4 sub-agents outperformed a single-agent Claude Opus 4 by 90% on their internal research evolves.

Architecture: Their system involves a "lead researcher" (orchestrator) that generates a plan based on the user query, and sub-agents with memory that execute the plan using various tools.

Search Application: They argue that multi-agent systems are well-suited for search applications because sub-agents can explore different research surfaces concurrently.

Performance Factors: They identified three factors contributing to the performance boost:

• Token Usage: Explained 80% of the variance.
• Number of Tool Calls
• Model Choices

Limitations: Anthropic acknowledges that multi-agent systems are not suitable for all tasks, particularly those requiring extensive context sharing or involving many dependencies between agents, such as coding tasks.

Practical Considerations for Building Agentic Systems

The video then delves into practical aspects of building agentic systems, drawing from Anthropic's insights.

Prompt Engineering

• Think Like Your Agent: Understand the task execution process from a human perspective to provide clear instructions.
• Teach Your Orchestrator How to Delegate: Provide detailed task descriptions to avoid task duplication.
• Scale Effort to Query Complexity: Allocate appropriate compute resources based on task complexity.
• Tool Design and Selection are Critical: Design tools with complexity in mind and provide clear descriptions of their inputs and outputs.
• Let Agent Self-Improve: Allow agents to modify tool descriptions or available tools based on their experiences.
• Search Specifics: Start wide and then narrow down, guiding the thinking process.
• Parallel Tool Execution and Sub-agents: Use parallelization to reduce research time, but be mindful of increased token usage.

Evaluation

• Start Small: Begin with a small set of representative queries (e.g., 20) to get early feedback.
• LLM as a Judge: Use a single LLM call with a simple scoring system (0-1 and pass/fail) for consistent evaluation.
• Human in the Loop: Human evaluators are crucial for catching issues that automated evaluation might miss, such as reward hacking.
• Emergent Behaviors: Be aware that small changes to one component can have unpredictable downstream effects on the entire system, requiring end-to-end evaluation.

Production Considerations

• Agents are Stateful and Errors Compound: Implement tracing and observability to identify and address errors.
• Rainbow Deployment Strategies: Use gradual deployment strategies to minimize disruption during updates.
• Synchronous Execution: Anthropic's system executes sub-agents synchronously to facilitate result aggregation.

Synthesis/Conclusion

The video presents a nuanced view of building agentic systems, highlighting the trade-offs between multi-agent and single-agent architectures. While Cognition Labs advocates for single-agent systems with sequential task execution and context compression, Anthropic demonstrates the effectiveness of multi-agent systems for specific applications like search. The key takeaway is that the optimal approach depends on the nature of the task, the level of context sharing required, and the complexity of coordination. Regardless of the architecture chosen, careful prompt engineering, rigorous evaluation, and robust production practices are essential for building reliable and effective agentic systems.