OpenAI's NEW Agent Builder and ChatKit are INSANE

Key Concepts

• Agent Builder: A visual, no-code/low-code interface by OpenAI for designing and orchestrating multi-agent AI workflows.
• Chat Kit SDK: A Software Development Kit that allows developers to easily integrate Agent Builder workflows into web applications as interactive chatbots.
• Widgets: Dynamic UI components that can be embedded within chat interfaces to display structured data or interactive elements.
• Multi-Agent Workflows: Systems where multiple specialized AI agents collaborate sequentially or in parallel to achieve a complex task.
• Vector Store: A database used to store and retrieve embeddings of documents or data, providing context for AI agents (a form of Retrieval Augmented Generation - RAG).
• MCP (Model Context Protocol): A new interface enabling Large Language Models (LLMs) to interact with external tools and APIs (e.g., CRM, e-commerce platforms, communication tools).
• Guardrails: Configurable safety and quality mechanisms within Agent Builder to moderate responses, prevent hallucination, hide personal information, and ensure desired output structure.
• AI Fluency: The understanding and ability to effectively prompt, provide context, take responsibility for AI outputs, and iterate on agent performance.
• Reasoning Levels: A configurable setting for agents to determine the complexity of thinking required for a given task (minimal for simple execution, high for complex problem-solving).

---

Overview of OpenAI Dev Day Updates

The discussion highlights three major announcements from OpenAI's Dev Day: Agent Builder, Chat Kit, and Widgets. These tools aim to simplify the creation and deployment of sophisticated AI applications, particularly multi-agent workflows, by providing visual interfaces and SDKs, thereby lowering the barrier for non-technical users.

---

Deep Dive into Agent Builder

Purpose and Functionality: Traditionally, building multi-agent workflows required custom code to orchestrate parallel or sequential interactions between assistants. OpenAI's Agent Builder introduces a visual interface where users can design these workflows using a drag-and-drop system of interconnected nodes. Each node represents a specific action or logic.

Key Features:

• Visual Workflow Creation: Users can visually build workflows, including parallel or sequential steps, without writing code.
• Tool Integration: Agents can call external tools, perform web searches, or access files (e.g., from a vector store).
• Context Management: Data from a vector store can be held as context for agents.
• Response Evaluation and Refinement: Tools for evaluating and refining agent responses are available.
• Guardrails: Implement safety and quality controls to prevent issues like hallucination, moderate harmful content, or hide personal information. This is crucial for building trust, especially for users new to AI.
• Logic and Data Transformation: Define how agents proceed based on conditions and transform data formats (e.g., to JSON with a specific schema).
• Reasoning Levels: Configure individual agents for "minimal" or "high" reasoning based on task complexity, impacting speed and cost.

Reducing Technical Barrier: A core argument is that Agent Builder significantly reduces the barrier for non-technical people to build multi-agent workflows, moving AI development beyond command-line interfaces (CLI) to a more accessible graphical user interface (GUI), akin to the shift from MS-DOS to Windows for personal computing.

---

Building a Demo Chatbot Workflow

The video demonstrates building a specific workflow in Agent Builder designed to handle customer interactions on a website:

1. Start Input: Receives user input as text (e.g., a message from a website visitor).
2. Classifier Agent:
   • Prompt: "Look at the inquiry and tell us if this is an existing customer with a support ticket or a new lead. Analyze the message and determine how you get to that conclusion."
   • Examples: Provided examples of what a "new lead" or an "existing customer" inquiry would look like to train the classifier.
3. Logic Node: Based on the classifier's output, it directs the interaction to one of two specialized agents:
   • If "Existing Customer": Passes to the Customer Support Agent.
   • If "New Lead": Passes to the Sales Lead Agent.
4. Customer Support Agent:
   • Training Data: Trained on the company's knowledge base (scraped product data, stored in a vector store).
   • Role: Answers support tickets and troubleshoots questions using the provided context.
   • Reasoning: Could be set to "medium" reasoning for troubleshooting.
5. Sales Lead Agent:
   • Role: Captures information about new leads.
   • Questions: Asks for details like website URL, company name, email, monthly visits, and current tools used (e.g., "What's your website URL?", "What's your company name?", "What's your email?", "How many visits do you get per month?", "What are you currently using?").
   • Reasoning: Could be set to "minimal" reasoning as it primarily involves data collection.
   • Next Step (MCP Integration): The gathered data can then be pushed to a database, a Slack notification, or a CRM (e.g., HubSpot) using an MCP.

Prompt Generation and Enhancement: The speaker notes that prompts for agents can be written manually, generated using ChatGPT (acting as a "prompt generator"), or enhanced directly within the Agent Builder interface to refine structure, tone, or specific response formats.

---

Integrating with Chat Kit and Widgets

Once a workflow is built and published in Agent Builder, it can be deployed:

• Chat Kit SDK: This new capability allows users to connect their Agent Builder workflow to a frontend. By pasting the workflow ID and API keys, a custom chatbot can be embedded directly onto a website.
• Developer Independence: A significant advantage is that non-technical teams (e.g., customer support) can build and modify these agent workflows in Agent Builder, publish them, and see changes live on the website without requiring engineering team deployment.
• Cost: The cost of running the chatbot is primarily based on OpenAI API token usage.
• Widgets: Dynamic components can be added to chat interfaces. For example, if an agent is connected to a Shopify store via an MCP, a widget could display order details, estimated delivery times, or product information directly within the chat.
• Customization: The chat widget's appearance and disclaimers are fully customizable via a playground.

Demo Walkthrough: The demo shows a user interacting with the embedded chatbot:

• Lead Scenario: User expresses interest in a demo ("Hi, I'm interested in a Humbolty demo."). The classifier identifies them as a new lead, and the sales agent asks for business details (website, company, email, monthly visits, current tools). The agent then recommends a plan and prompts for a demo booking.
• Customer Support Scenario: User identifies as an existing customer ("I'm an existing customer. Help me add a web flow site to track."). The classifier identifies them as an existing customer, and the support agent provides instructions based on the knowledge base.

---

Benefits and Customization

• Internal Use Cases: Multi-agent orchestration with MCPs (e.g., sending Slack notifications) is valuable for backend automations.
• Cost and Time Savings: For startups or mid-sized companies with engineering capabilities, building custom solutions with Agent Builder and Chat Kit can lead to significant long-term cost and time savings compared to third-party SaaS products like Intercom.
• Full Control and Customization: Users have complete control over the workflow, data, and agent behavior, allowing for highly tailored solutions.
• Ownership: Users essentially "own" their workflow and system.
• Guardrails for Trust: The ability to implement guardrails helps build trust in AI outputs, addressing common issues where users lose faith if an agent makes a single mistake.

---

Comparison and Future Outlook

• MCP Capabilities: While OpenAI has official connectors and third-party options, the speaker notes that Claude is currently ahead in MCP capabilities, having invented the Model Context Protocol and offering a more extensive directory and features. OpenAI needs to enhance its MCP offerings.
• Target Audience: The Agent Builder caters to "knowledge workers" and non-technical people, addressing a broader use case beyond just engineering and coding.
• Evolution: The speaker is curious to see how MCP capabilities evolve over time.

---

Getting Started & Opportunities for Founders

How to Get Started with Agent Builder:

1. Access: Available at platform.openai.com.
2. Define Use Case: Start by identifying a specific problem or goal (e.g., custom customer support, lead capture).
3. Workflow Design: Work backward from the goal to design a multi-agent workflow with specialized agents.
4. Data Context: Build and clean your data context, store it in a vector store, and reference it in Agent Builder.
5. Optimize Context: Use as little context as possible to maximize performance, as excessive context can degrade agent performance.
6. Specify Roles: Clearly define roles for each agent (e.g., classifier, sales lead, customer support).
7. External Tools: Determine if external tools, MCPs, or web search are needed.

Opportunities for Founders:

• ChatGPT as a Distribution Channel: Leverage OpenAI's new "app capabilities" within ChatGPT as a new interface and distribution layer for your applications.
• Empower Non-Technical Teams: Provide Agent Builder and Chat Kit to product managers, customer support, and sales teams. With engineering support for MCPs and server setup, these teams can build custom workflows, save time, and innovate.
• Custom Solutions: Build highly customized, owned solutions that offer greater control and potential cost savings compared to off-the-shelf SaaS products.