Scaling AI Agents Without Breaking Reliability — Preeti Somal, Temporal

Key Concepts:

• Agentic AI Applications
• Workflows (orchestration of processes, state management, human interaction, parallel execution)
• Reliability and Scalability
• Temporal (platform for building reliable and scalable distributed applications)
• Activities (wrapping tools and LLMs)
• Signals and Queries (input and data retrieval for workflows)
• Temporal Cloud (manages execution state, failures, retries)
• Workers (developer's code, runs in their environment)
• Code Exchange (open-source repository of Temporal code examples)

Agentic AI Applications as Complex Distributed Systems:

• Agents are more than just simple software; they are complex distributed systems that require careful orchestration.
• These systems must handle interactions with Large Language Models (LLMs), scale efficiently, and provide durability and reliability.
• The core of Agentic AI applications involves complicated workflows that orchestrate multiple processes, manage state over time, handle human interaction, and run in parallel.
• The speaker emphasizes that building agents is not a simple task, as it involves managing numerous interactions and ensuring reliable execution.
• LLMs are inherently unreliable, making debugging and testing difficult.

Temporal's Solution for Reliability and Scalability:

• Temporal is designed to outsource the reliability and scalability aspects of building complex distributed applications.
• It provides language-idiomatic SDKs (Python being a popular choice) and handles the plumbing code to ensure reliable process execution.
• Temporal has been in production for over a decade and is used in mission-critical applications by various customers.
• Using Temporal allows developers to focus on writing business logic without worrying about reliability issues.

Customer Success and Use Cases:

• Customers like Dust are building their agents on top of Temporal.
• Gorgeous, a customer service provider for brands like Reebok and Glossier, uses AI agents built on Temporal in production.
• These use cases demonstrate that Temporal enables agility and speed by allowing developers to focus on business logic.
• The payments example highlights the mission-critical nature of workloads running on Temporal.

Architecture and Code:

• Before Temporal, developers had to code a lot of interaction and error handling.
• Temporal abstracts this complexity into workflows, which are written as code using Temporal SDKs.
• Workflows orchestrate interactions between LLMs, chat history databases, and tools.
• The key abstractions include workflows, signals (input to workflows), and queries (data retrieval).
• Temporal handles plumbing code, such as retries, allowing developers to focus on business logic.
• Temporal stores workflow history for visibility and debugging.

Impact on Engineering Teams:

• Temporal accelerates development, enabling teams to put applications into production in weeks.
• Customers have reported feature delivery velocity improvements of over 6x after adopting Temporal.
• Temporal Cloud handles scaling, allowing applications to scale with events without managing scale logic.
• Reliable applications lead to happier customers and reduced stress for engineers.

Ticket Booking Agent Example:

• The speaker presents a ticket booking agent demo to illustrate how Temporal works.
• The architecture involves the user, Temporal, AI language models, goals, and tools.
• The workflow defines the application flow and is written as code.
• Activities wrap the tools used by the agent.
• Temporal manages interactions through signals and queries and stores them in the workflow history.
• The workflow history can be exported for compliance or debugging purposes.
• Temporal Cloud handles reliability and scalability, while the developer's code runs in their environment.

Temporal Cloud and Worker Code:

• The worker is the developer's code, which runs in their environment and integrates with their CI/CD practices.
• Temporal aims to meet developers where they are, allowing them to write code without changing their existing practices.
• Temporal Cloud is available for sign-up, and credits are provided to get started.

Code Exchange and Open Source:

• Temporal is an open-source product, and code examples are available on the Code Exchange.
• Developers can explore the code, run examples in their local environment, and deploy to Temporal Cloud.

Conclusion:

Temporal provides a robust platform for building reliable and scalable Agentic AI applications. By abstracting away the complexities of distributed systems, Temporal allows developers to focus on writing business logic and delivering value to their customers. The platform's architecture, SDKs, and cloud infrastructure provide the necessary tools and services to build and deploy complex AI agents with confidence.