Rise of the AI Architect — Clay Bavor, Cofounder, Sierra w/ Alessio Fanelli

Key Concepts

AI Architect, Customer Experience (CX), Agent OS, Agent Development Lifecycle, Build vs. Buy, Large Language Models (LLMs), Model Capabilities, AI Strategy, Customer Support, AI Agents, Voice AI, Wearable AI, Generative AI, User Simulation Testing, Regression Testing, Model Migration, Model Upgrades.

Sierra Overview and Mission

Sierra aims to bridge the gap between businesses' desire for excellent customer service and the cost-related challenges of providing it. They build AI agents to improve customer experiences. They have hundreds of customers and will serve hundreds of millions of consumers this year. Clients include ADT (home security) and SiriusXM (AI agent named Harmony that can send satellite signals). Sierra believes every company will have its own branded, customer-facing AI agent, succeeding websites and mobile apps.

The AI Architect Role

The AI Architect is the AI-era equivalent of a webmaster, responsible for a company's AI agent. This role involves:

1. Technology Understanding: Having a feel for what AI agents can do, without necessarily being a deep expert in model training or coding.
2. Brand Ambassador: Ensuring the agent reflects the company's voice, values, and tone, creating a connection with the customer. Example: Chubbies' agent, Duncan Smothers, tells irreverent jokes.
3. Business Outcomes: Aligning the agent's interactions with desired business results.

This role requires a blend of technology, aesthetics/design, and business acumen. It is expected to be one of the fastest-growing job types in the next 5 years. AI Architects are emerging from customer experience, engineering, and other teams, with those from CX often being the most effective.

Developing an AI Strategy

Successful AI strategies involve:

1. Embracing Risk and Exploration: Being willing to experiment with probabilistic AI software and not letting "perfect be the enemy of the good."
2. Focusing on Concrete Problems: Starting with a narrow, valuable problem to solve, rather than broadly applying AI. Example: Successfully processing a single return, including drop-shipping a new pair of shoes and providing a shipping label.
3. Re-architecting Teams: Restructuring customer experience teams to support and improve the AI agent. Example: Creating a team to review conversations and coach the agent on empathy, judgment, and decision-making.

Build vs. Buy Fallacy

Companies often underestimate the complexity of building their own AI agents. The "agent iceberg" illustrates this:

• Above the Surface: Choosing a language model, embeddings model, and vector database.
• Below the Surface: Hundreds of hidden complexities, including regression testing, unit testing, model migration, model upgrades, and handling voice-specific challenges (e.g., separating primary and secondary speakers).

Sierra offers "Agent OS," a platform with both in-code tools for building sophisticated agents and no-code tools for non-technical users to refine and coach the agent. Companies that attempt to build their own often return after realizing the depth of the challenges.

Agent Building Iteration Process

Sierra has developed an "agent development lifecycle" to address the non-deterministic nature of AI software. Key aspects include:

1. User Simulation Testing: Using AI to test AI. Creating dozens of personas with simulated accounts and devices to have tens or hundreds of thousands of conversations with the agent before launch.
2. Modeling Customer Journeys: Expressively modeling key customer journeys in code, allowing the agent to handle both common scenarios and unexpected curveballs.
3. Closed-Loop Learning: Providing CX and engineering teams with insights into when the agent struggles and enabling the agent to learn from its mistakes through coaching and improvement.

Staying Up-to-Date on Model Capabilities

Keeping pace with rapidly evolving AI models requires:

1. Immersion: Staying informed through Twitter/X, research papers, and adjacent fields (even if not directly applicable).
2. Hands-On Experience: Regularly using the tools and models to understand their capabilities.
3. Focus on the First Derivative: Understanding where things are going is more important than knowing the current state.
4. Tracking Model Improvements: Maintaining a record of problems that were previously unsolvable and periodically re-testing them with newer models.
5. Anticipating the Frontier: Building products that anticipate future model capabilities and performance improvements.

The Future of AI Interfaces

AI interaction will evolve beyond chat interfaces and voice calls. Agents will be "shape shifters" capable of using text, voice, video, imagery, and user interfaces. Wearable devices, particularly glasses, will be the ultimate vehicle for trusted personal AI. These devices can see and hear what the user does, providing real-time assistance and guidance. The goal is an omnipresent, omnicapable AI assistant that enhances our lives.

Conclusion

The development and deployment of AI agents for customer service is a complex undertaking that requires a strategic approach. The AI Architect role is crucial for aligning technology, brand, and business goals. Companies should focus on solving concrete problems, re-architecting teams to support AI, and understanding the full scope of the "agent iceberg" before deciding to build or buy. Staying ahead of the curve in AI requires continuous learning, experimentation, and a focus on anticipating future model capabilities. The future of AI interfaces will likely involve wearable devices that provide seamless, personalized assistance.