5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems

Key Concepts

• Augmented LLM: A single LLM API call with structured output.
• DAG (Directed Acyclic Graph): A deterministic workflow pattern used for routing and process automation.
• Tool Calling: Enabling an LLM to interact with external databases, APIs, or policies.
• Agent Harness: A runtime environment (e.g., Claude Code SDK) that provides agents with file system access, bash execution, and web search capabilities.
• Multi-Agent Orchestration: A system where an orchestrator delegates tasks to sub-agents, each with its own isolated context window.
• Edge Nodes: Specific points in a workflow where complex reasoning or tool usage is required.

---

1. The Five Levels of AI Agent Complexity

Dave Abalar categorizes AI systems into five progressive levels of complexity, emphasizing that engineers should always choose the simplest level that solves the problem.

1. Level 1: Augmented LLM: The simplest form, involving a single API call with structured output.
2. Level 2: Prompt Chaining & Routing (DAGs): Using deterministic "if-else" logic to route inputs (e.g., classifying customer support tickets) to specific workflows.
3. Level 3: Tool Calling Agents: The LLM is given a set of tools to perform actions in a loop, such as querying databases or checking company policies.
4. Level 4: Agent Harnesses: Systems that provide a full runtime environment (file system access, bash, web search). This is described as "experimental" and powerful, requiring careful permission management.
5. Level 5: Multi-Agent Orchestration: An orchestrator manages sub-agents, each with its own context window, preventing context bloat during long-horizon tasks.

2. Production Insights and Methodology

Abalar highlights that while "agentic" systems are popular, DAGs remain the "bread and butter" of reliable B2B production systems.

• The Hybrid Approach: The most effective production systems combine deterministic DAGs with agentic "edge nodes."
• Monitoring: Using tools like Langfuse is essential for tracing complex workflows and debugging tool calls within production environments.
• Evolution of Systems: Abalar’s team typically starts with simple DAGs and introduces tool-calling agents only at specific edge nodes where the LLM needs to retrieve external information (e.g., product rules or missing customer data).

3. Technical Frameworks and Tools

• Claude Code SDK: A primary example of an "Agent Harness." It allows developers to build environments where agents can execute bash commands, search files, and browse the internet.
• Context Management: In multi-agent systems, the orchestrator pattern is used to keep the main context window clean by spawning separate, isolated context windows for sub-agents.
• Alternative Frameworks: While the Claude SDK is highlighted, similar results can be achieved using LangGraph or PydanticAI.

4. Key Arguments and Perspectives

• Complexity Management: Abalar warns against "Frankenstein DAGs"—overly complex graphs that become difficult to maintain. He suggests decomposition or microservices as the system scales.
• The "YOLO" Risk: He cautions that giving agents full file system and internet access (Level 4) is dangerous in production without strict containerization and permission constraints.
• Demystifying Hype: Abalar argues that new trends (like OpenClaw) are often just new "agent harnesses" built on foundational principles (LLMs + Tools + System Prompts). Understanding these abstractions allows engineers to see through marketing hype.

5. Notable Quotes

• "Use the simplest level that gets the job done and combine them."
• "A deterministic, almost deterministic-like DAG is always easier to maintain and create unit tests around than an LLM node with five tool calls."
• "The agent harness around [the model] is so, so important."

6. Synthesis and Conclusion

The core takeaway for AI engineers is to prioritize reliability and maintainability. Start with deterministic workflows (DAGs) and only introduce agentic complexity (tool calling or harnesses) at the edges where it is strictly necessary. As AI technology evolves, the underlying principles—LLMs, structured prompts, and tool integration—remain constant. Success in production comes from strategically combining these levels of complexity rather than relying on a single "magic" agent framework.