Why long-running AI agents break on HTTP and how Ably is fixing it

Key Concepts

• Durable Sessions: A communication layer that maintains state and continuity for long-running AI agent interactions, allowing for reconnections and device switching without losing context.
• Pub/Sub (Publish/Subscribe): A messaging pattern where senders (publishers) categorize messages into classes without knowledge of the receivers (subscribers).
• AI Transport: A specialized infrastructure layer designed to handle the unique requirements of AI agents, such as token streaming, tool calls, and state synchronization.
• Live Objects: A primitive for collaborative object storage that keeps state synchronized in real-time between clients and agents.
• Idempotency: The property where an operation can be applied multiple times without changing the result beyond the initial application, crucial for reliable agent execution.
• CRUD (Create, Read, Update, Delete): The basic operations of persistent storage, often used to describe standard web application architectures.

---

1. Main Topics and Key Points

• Evolution of Infrastructure: Ably transitioned from a general-purpose real-time pub/sub platform to providing specialized infrastructure for AI agents.
• The Shift to Long-Running Agents: Modern AI applications have moved from simple request/response models to long-running, stateful agentic workflows. This shift renders standard HTTP transport inadequate due to connection drops and the need for persistent state.
• The "Durable" Requirement: As agents become more complex, they require "durable" layers—durable memory, durable execution (e.g., Temporal), and durable communication (Ably).
• Decoupling: A core argument is that the transport layer should be decoupled from the application logic. Developers should not have to build custom plumbing for presence, reconnection, or state synchronization.

2. Real-World Applications

• Intercom: Adopted Ably to handle the complexities of long-running conversations, allowing their engineers to focus on UX rather than infrastructure plumbing.
• HubSpot: Utilizes "Live Objects" to maintain synchronized state between users and AI co-pilots, ensuring that if a user moves between tabs or devices, the agent remains aware of the current state of the user's work (e.g., editing a spreadsheet).
• Claude Code: Cited as an example of an advanced experience where users can "double text" or interrupt an agent mid-process, requiring a bidirectional, durable transport layer.

3. Methodologies and Frameworks

• The "Drop-in" Approach: Ably’s AI Transport is designed as a plugin for existing AI SDKs (like the Vercel AI SDK). It allows developers to add durability without rewriting their entire architecture.
• The Turn-Based Model: The platform organizes communication into "turns" (a request followed by tool calls and responses) that sit on top of a persistent session layer.
• Implementation Process:
  1. Client sends a request via HTTP.
  2. The agent acknowledges the request.
  3. The agent publishes updates to a globally distributed channel (using Ably’s infrastructure).
  4. The client consumes these updates, maintaining state even if the connection drops or the user switches devices.

4. Key Arguments

• HTTP Limitations: While HTTP is sufficient for simple request/response, it fails in complex, long-running agentic interactions where bidirectional communication and state persistence are required.
• Infrastructure vs. Intelligence: Companies are increasingly solving the "intelligence" problem (getting good answers from LLMs) and are now shifting focus to the "user experience" problem (how that intelligence is delivered and maintained).
• Presence and Continuity: A critical requirement for modern AI is knowing if the user or agent is "present" and ensuring that if a connection is lost, the session can be rehydrated seamlessly.

5. Notable Quotes

• "We don't have an AI story as such. We just are some of the plumbing that people are using for their AI applications." — Matt O'Connell, CEO of Ably.
• "The next frontier for them [mature AI companies] is the user experience of those agents."
• "Durable sessions is a better terminology [than durable streams]... it's the presence state, it's the collaborative state... it's a much [more comprehensive] layer."

6. Technical Infrastructure

• Storage Layer: Ably utilizes Cassandra as its underlying distributed storage layer to ensure data persistence and reliability.
• Global Distribution: The platform replicates channels across every region where an active client exists to ensure low latency and high availability.

7. Synthesis and Conclusion

The transition from simple CRUD-based web applications to complex, agentic AI systems has created a "plumbing" crisis. Developers are hitting the limits of HTTP for long-running, stateful interactions. Ably’s solution is to provide a "durable session layer" that abstracts away the complexities of connection management, presence, and state synchronization. By integrating this as a drop-in transport layer, developers can focus on building intelligent AI experiences rather than reinventing the infrastructure required to keep those agents connected and consistent.