OpenAI Just Confused Everyone... Again

Key Concepts

• AI Agent (OpenAI's Definition): A system that can do work independently on behalf of the user (vague).
• AI Agent (Anthropic/Industry Definition): Systems where LLMs dynamically direct their own processes and tool usage, maintaining control over task accomplishment.
• Workflow (OpenAI's "Agent Builder"): A well-defined, declarative graph where users/developers explicitly define every step, branch, loop, and conditional upfront, orchestrating LLMs and tools through predefined code paths.
• Workflow (Anthropic/Industry Definition): Systems where LLMs and tools are orchestrated through predefined code paths.
• Declarative Graph: A visual or explicit representation where every step, branch, loop, and conditional in a workflow is defined upfront.
• Code-First Approach: Expressing workflow logic directly using programming constructs without needing to predefine the entire graph, enabling more dynamic and adaptable orchestration.
• Agent SDK: OpenAI's Software Development Kit for building agents, used in the backend for code generation.
• Autonomous Agent: An agent with the ability to plan, take actions, and modify its plan based on feedback from the environment or results of actions.
• LLM Calls: Interactions with Large Language Models.
• Conditional Logic Gate: A mechanism that directs the flow of operations based on specific conditions.
• MCPs (Multi-Cloud Providers/Integration Points): Connectors or integrations that allow bringing external services (like Zapier) into OpenAI's Agent Kit.
• Vendor Lock-in: Dependence on a specific vendor's products or services, making it difficult to switch to another vendor.

---

OpenAI's Confusing Agent Terminology

The video highlights significant confusion regarding OpenAI's definition of an "agent" and its "agent builder." OpenAI's "agent builder" visually resembles what the industry, including Anthropic, commonly refers to as workflows. These are characterized by multiple LLM calls chained together through conditional logic gates, forming a predefined, declarative graph where every step is explicitly defined by the user or developer.

OpenAI's new definition of an AI agent is "a system that can do work independently on behalf of the user," which is criticized as extremely vague and potentially intentional. In contrast, Anthropic defines agents as "systems where LLMs dynamically direct their own processes and tool usage maintaining control over how they can accomplish tasks." This distinction is crucial: OpenAI's "agent builder" creates predefined workflows, not dynamically directing, autonomous agents as understood by the broader industry. The blog post's statement, "design workflows with agent builder," further exemplifies this terminological ambiguity.

Declarative vs. Code-First Approaches: OpenAI's Shifting Stance

Ironically, a few months prior to releasing the "agent builder," OpenAI published "A practical guide to building agents" which critiqued declarative graphs. In this guide, they stated that frameworks requiring developers to "explicitly define every branch loop and conditional in the workflow upfront through graphs" (likely referring to tools like LangGraph) could "quickly become cumbersome and challenging as workflows grow more dynamic and complex, often necessitating the learning of specialized domain specific languages."

The video points out that OpenAI's new "agent builder" is precisely this type of declarative graph, contradicting their earlier criticism. The guide then advocated for a "more flexible code-first approach" using the Agent SDK, allowing developers to "directly express workflow logic using familiar programming constructs without needing to predefine the entire graph upfront." While the "agent builder" can generate code using the Agent SDK, the core visual interface remains a declarative workflow builder.

Understanding Workflows

A workflow is essentially a declarative decision graph that translates how a user solves a problem into a series of predefined actions and conditions. The user defines these actions and conditions, and an LLM can make decisions at different points based on available tools. The speaker believes that for most business applications, this predefined workflow approach is exactly what is desired. Anthropic's blog post, for instance, details six or seven different configurations for building workflows to solve specific problems.

Understanding Autonomous Agents

In contrast to workflows, autonomous agents possess the ability to plan, take actions, and crucially, modify their plan based on feedback from the environment or the results of those actions. The process involves:

1. Tool Selection: The model selects a specific tool based on its needs, which goes into its context.
2. Action: The model takes action, and the tool call generates results.
3. Observation & Plan Update: The results go into the model's history, and based on these observations, the LLM updates its plan and takes further actions.

This iterative feedback loop provides significantly more flexibility than a predefined workflow.

Criticisms of Visual Workflow Builders

Harrison Chase, CEO of Langchain, highlighted several issues with visual workflow builders in a blog post:

1. Not Low Barrier to Entry: Despite being built for a mass audience, they are not easy for average non-technical users, echoing OpenAI's own past criticism.
2. Complexity Management: Complex tasks quickly become too complicated to manage effectively within a visual builder.

While Chase's arguments are made in the context of advocating for LangGraph, the criticisms of visual builders are considered valid.

OpenAI Agent Kit's Impact and Limitations

The speaker disagrees with the notion that OpenAI's Agent Builder will "kill" other visual builders like Zapier, Make, or N8N. Instead, the Agent Kit offers powerful integration capabilities through MCPs (Multi-Cloud Providers/Integration Points), allowing users to bring in connectors and integrations from services like Zapier directly into OpenAI's environment. This theoretically expands the kit's capabilities beyond its current list of supported integrations.

While advertised to developers, the speaker suggests that developers might prefer more control than a visual builder offers. However, the Agent Kit does include useful features like evaluation (evals) and smooth integration.

A significant limitation is vendor lock-in: being an OpenAI product, it primarily limits users to OpenAI models. Although MCP support theoretically allows integrating other models, they might not drive the overall flow or structure. The speaker speculates that since the kit is built on the Agent SDK, there might be a workaround to build and test workflows within the kit, and then potentially replace the underlying models with alternatives, though this requires further testing.

Synthesis and Conclusion

The core issue lies in OpenAI's ambiguous and potentially misleading terminology, conflating what the industry calls "workflows" with "agents." While OpenAI's "agent builder" offers a powerful, declarative way to construct predefined sequences of LLM calls and tool usage (workflows), it does not align with the industry's understanding of autonomous agents that dynamically plan and adapt. Despite its utility for many business applications and its integration capabilities via MCPs, the Agent Kit faces criticisms regarding its complexity for non-technical users and its inherent vendor lock-in. The distinction between predefined, user-directed workflows and truly autonomous, self-directing agents remains a critical point of differentiation.