Ensure AI Agents Work: Evaluation Frameworks for Scaling Success — Aparna Dhinkaran, CEO Arize

Key Concepts

• AI Agents & Assistants Evaluation
• Routers, Skills, and Memory (Agent Components)
• Control Flow
• Evals (Evaluation Metrics)
• Convergence (Agent Path Efficiency)
• Multimodal Agents (Voice AI)
• Traces (Agent Inner Workings)

Evaluating AI Agents and Assistants

Introduction

The talk focuses on the critical aspect of evaluating AI agents and assistants, particularly after they are deployed into production. It emphasizes the importance of ensuring these agents function correctly in real-world scenarios. The speaker, Apta, highlights that while many discussions revolve around building agents, less attention is given to their evaluation.

Components of an Agent

Apta breaks down an agent into three core components:

1. Router: The "boss" that decides the next step for the agent. It determines which skill to call based on the user's query.
2. Skills: The logical chains that perform the actual work or tasks requested by the user.
3. Memory: Stores the conversation history to maintain context in multi-turn interactions.

These components are common across different agent frameworks like LangChain, CrewAI, and LlamaIndex, regardless of the specific implementation.

Router Evaluation

The primary concern with routers is ensuring they call the correct skill. For example, if a user asks about leggings, the router should direct the query to a product search skill, not customer service or discount offers. Evaluation involves verifying:

• The control flow of the agent.
• Whether the router is correctly calling the appropriate skill (A, B, or C).
• If the router is passing the correct parameters to the skill based on the user's request (e.g., material type, price range).

Skill Evaluation

Evaluating skills is more complex due to the various components involved. Key aspects include:

• Relevance: For Retrieval-Augmented Generation (RAG) skills, assessing the relevance of the retrieved chunks.
• Correctness: Evaluating the accuracy of the generated answer.
• Path/Convergence: Analyzing the number of steps the agent takes to complete a task. Ideally, the agent should consistently take a similar number of steps for the same task.

The speaker notes that convergence is one of the most challenging aspects to evaluate. Different LLMs (e.g., OpenAI vs. Anthropic) can result in significantly different path lengths for the same skill.

Multimodal Agent Evaluation (Voice AI)

Voice AI agents, which are increasingly prevalent in call centers, require additional evaluation considerations beyond text-based agents. These include:

• Evaluating the audio chunks in addition to the generated transcript.
• Assessing user sentiment from the audio.
• Measuring the accuracy of speech-to-text transcription.
• Ensuring the tone is consistent throughout the conversation.

Case Study: Arise AI's Co-pilot

Apta shares an example of how Arise AI evaluates its own co-pilot, which is integrated into their product. They use "traces" to monitor the co-pilot's inner workings and run evaluations at every step. These evaluations include:

• Overall response correctness (for search questions).
• Router accuracy in selecting the right skill.
• Parameter accuracy passed to the router.
• Correctness of task completion.

The key takeaway is to implement evaluations throughout the agent's application to facilitate debugging and identify the source of errors (router, skill, or other components).

Conclusion

Evaluating AI agents and assistants is crucial for ensuring their effectiveness in real-world applications. This involves assessing the performance of individual components like routers and skills, as well as considering additional factors for multimodal agents like voice AI. Implementing evaluations at multiple layers of the agent's architecture enables comprehensive monitoring and debugging.