Ship Your First Vertical AI Agent (& Make It Reusable)

Key Concepts

• Vertical AI Agents: AI agents designed for a specific role and reusable across multiple businesses. The IP is owned by the developer, who charges for usage.
• Horizontal AI Agents: AI agents built for a specific business and not reusable. The client owns the IP, and the developer charges per hour or service.
• Product-Based Business: Focuses on scalable, reusable products (Vertical AI Agents).
• Service-Based Business: Focuses on custom solutions for individual clients (Horizontal AI Agents).
• Playbook for Vertical AI Agents: A step-by-step guide to productizing and scaling AI agents.
• Customization Components: Instructions, Knowledge, and Actions are the primary parts of an agent that need customization for different businesses.
• Onboarding Flow: A process to collect specific business information needed to customize an agent.
• Evals (Evaluations): A system for continuously improving AI agents by analyzing their performance on past interactions.
• Cursor: An AI-powered coding assistant and framework used in the demonstration.
• Licensing Model: A pricing strategy where clients pay a recurring fee for the agent's IP.

Vertical AI Agents: The Playbook for Productization and Scaling

This video provides a comprehensive guide to building and scaling vertical AI agents, building upon a previous popular video on the topic. The core aim is to offer a practical, step-by-step playbook for productizing AI agents, illustrated with a real-world example of a customer support agent for a SaaS platform.

Why Build Vertical AI Agents?

The primary motivation for building vertical AI agents is to 100x your impact by leveraging your expertise across an entire industry. Instead of being limited by your individual capacity, an AI agent trained on your knowledge can serve clients without your direct involvement. Furthermore, building vertical AI agents is simpler than traditional SaaS as it requires less complex UIs and backends, often allowing clients to interact via platforms like Slack.

Vertical vs. Horizontal AI Agents: A Business Model Distinction

A crucial distinction is made between vertical and horizontal AI agents, which fundamentally impacts the business model:

• Vertical AI Agents:
  • Target: Specific roles, reusable across businesses.
  • IP Ownership: Developer owns the Intellectual Property.
  • Revenue Model: Charge clients for usage (e.g., licensing).
  • Business Model: Product-based, emphasizing scalability.
  • Upfront Investment: Higher time and effort for productization.
• Horizontal AI Agents:
  • Target: Specific businesses, custom solutions.
  • IP Ownership: Client owns the Intellectual Property.
  • Revenue Model: Charge per hour or for services delivered.
  • Business Model: Service-based, less scalable.
  • Upfront Investment: Lower, as development is client-specific.

The speaker's perspective has shifted, recommending productizing as soon as possible (even after 1-2 agents) rather than waiting to build multiple horizontal agents first. This is due to a market trend where clients prefer pre-defined solutions over custom ones.

Identifying Valuable Opportunities for Productization

Several avenues exist for finding opportunities to productize AI agents:

1. Productize Your Own Expertise: Leverage your existing knowledge from education, work experience, or specific niches. Even a basic understanding of a general process can be a starting point.
2. Buy a High-End Course: Expensive courses (thousands of dollars) often encapsulate well-documented business processes that can be used to train an AI agent. Avoid low-cost online communities that offer little beyond what ChatGPT provides.
3. Partner with an Industry Expert: This model, common among YC companies, involves a CTO handling the agent layer and technology stack, while an industry expert trains the agent on specific processes.

The speaker highlights that the technology stack for building agents is largely standardized, and their open-source framework allows for building without significant upfront investment.

Pricing Vertical AI Agents

The recommended pricing model for vertical AI agents is licensing. Clients pay a recurring monthly fee for the agent's IP. While usage-based and performance-based models are being developed, licensing is currently the most straightforward. Under this model, clients also cover their own API token costs, mitigating concerns about excessive usage. Offering an onboarding flow, ongoing support (e.g., Slack), and a 7-day free trial is strongly advised to improve customer conversion.

The Vertical AI Agent Playbook: Step-by-Step

The core idea behind building vertical AI agents is to create templates. The process involves productizing only the customizable parts of an agent while keeping the rest standardized.

1. Start with a Solution for a Specific Client: Develop an initial agent for a real-world use case.
2. Identify Customization Needs: Determine which parts of the agent require modification for different businesses. The primary areas are:
   • Instructions: The core directives and persona of the agent.
   • Knowledge: The data and information the agent can access.
   • Actions: The tools and functionalities the agent can perform (e.g., API calls).
3. Create an Onboarding Flow: Design a user-friendly process to collect the specific variables needed for customization. This is typically implemented using Pydantic models, where each field has a type and description.
4. Deploy the Agent: Make the agent accessible.
5. Set Up Evals (Evaluations): Implement a system for continuous improvement. Evals allow agents to learn and adapt from every new client interaction.
6. Iterate and Improve: Refine the agent based on feedback and eval results.
7. Scale: Increase marketing efforts or rely on client referrals if the solution is compelling.

Practical Implementation: Productizing a Customer Support Agent

The video demonstrates productizing an existing customer support agent using the Cursor framework.

• Template Repository: A starter template is provided, including a productize command that guides the agent through creating vertical templates.
• Customization Example (Customer Support Agent):
  • Instructions: Variables like email addresses for support teams need to be customized.
  • Actions: The agent's schema for submitting support requests needs to be adaptable.
  • Knowledge: The folder for resolved customer support issues will contain client-specific data.
• Using Cursor's productize Command:
  1. Tag the productize command in Cursor.
  2. Cursor searches documentation and analyzes existing agent files.
  3. It suggests potential customizable fields (e.g., email addresses, API tokens).
  4. For advanced features not recognized by Cursor, manual documentation links can be provided.
  5. The agent automatically inserts these variables into instruction templates using Python's format command.
  6. Environment variables (e.g., API keys) are added to .env.template files, which are then presented to the user in the onboarding form.
• Deployment:
  1. Commit changes to GitHub.
  2. Publish the agent on the platform's marketplace (or keep it private).
  3. The platform builds the agent, checks for errors, and makes it live.
  4. The onboarding form is automatically generated based on the defined Pydantic model.
• End-User Experience:
  1. Clients fill out a simple onboarding form with specific details (e.g., business overview, accounting frequency, software used).
  2. The agent is automatically deployed with the customized settings.
  3. Updates to the agent are automatically pushed to client instances.
  4. Changes to the onboarding form only require clients to provide new fields.
• Website Widget Integration: The deployed agent can be embedded into a website as a widget.
• Real-World Test: The customer support agent successfully processed a support ticket by searching provided files and executing an API call, demonstrating its functionality.

Supercharging Agents with Evals

Evals are a critical feature for creating a strong moat around AI agent services.

• Enabling Evals: Evals are enabled by default on the platform. Users can access logs and traces on platform.openai.com.
• Running an Eval:
  1. Navigate to the "Evals" section.
  2. Select the relevant agent (e.g., "customer support agency").
  3. Describe the evaluation criteria (e.g., "if the issue has been resolved correctly").
  4. Choose a model (e.g., GPT-4 Mini).
  5. Run the evaluation on a selected number of traces (e.g., last 5, 10, or all).
• Interpreting Results:
  • Evals run asynchronously and can take minutes to hours.
  • The results provide metrics like "resolution correctness" (e.g., 60%).
  • Users can open failing traces to understand the specific issues, analyze the reasoning, and identify areas for improvement.
  • Successful traces can also be reviewed to understand common customer queries.
• Continuous Improvement: The more customers an agent serves, the more data is generated for evals, leading to continuous improvement and a stronger competitive advantage.

Conclusion

The video concludes by emphasizing the power of productizing AI agents into scalable vertical templates. The demonstrated process, facilitated by tools like Cursor, simplifies the creation of customizable AI solutions. The integration of evals provides a robust mechanism for ongoing improvement, making vertical AI agents a highly scalable and defensible product offering. The speaker encourages viewers to explore further resources and subscribe for more advanced AI development content.