Why Eval++ Is the Next Great Compute Primitive — Sunil Pai & Matt Carrie, Cloudflare

Key Concepts

• Durable Objects (DO): A stateful serverless primitive that allows for persistent state, WebSocket connections, and coordination across a global network.
• AI Agents: Autonomous or semi-autonomous software entities that perform tasks, maintain state, and interact with external APIs.
• Dynamic Workers: A feature allowing the execution of user-generated or LLM-generated code in isolated, sandboxed environments on-demand.
• MCP (Model Context Protocol): A standard for connecting AI assistants to systems; Cloudflare uses DOs to maintain the stateful connections required by MCP.
• Code Mode: A methodology for AI agents to generate and execute code dynamically to interact with complex APIs (e.g., Cloudflare’s 2,600+ endpoints) efficiently.
• Isolates: The underlying technology for Cloudflare Workers, providing lightweight, secure execution environments without the overhead of VMs or containers.

---

1. Main Topics and Technical Infrastructure

The speakers, Matt and Sunil, discuss the evolution of building AI agents on Cloudflare’s infrastructure.

• Stateful Serverless: Traditional serverless functions are stateless, requiring external databases for state. Durable Objects solve this by providing a persistent, addressable compute unit that maintains state and handles WebSocket connections globally with low latency (e.g., ~15ms).
• The Agents SDK: A framework that simplifies building agents by providing React hooks, background scheduling, and persistence. It allows developers to define "callables" that can be triggered by clients or background tasks.
• Dynamic Workers: This allows developers to take a string of code (generated by an LLM or user) and execute it in a secure, isolated sandbox. Unlike traditional sandboxes that wrap VMs in security, this approach starts with a "deny-all" posture, where the code has no access to APIs unless explicitly granted.

2. Real-World Applications and Use Cases

• Real-time Collaboration: Using Durable Objects to enable multi-user, synchronized experiences (e.g., shared screens or collaborative AI chat).
• Resumable Streaming: Because DOs maintain state, if a user refreshes their browser during an LLM stream, the agent can resume the stream seamlessly without needing complex database replication or sticky sessions.
• Code Mode: Used to interact with massive API surfaces. By generating code to perform specific tasks, an agent can interact with thousands of endpoints using minimal tokens compared to traditional prompt-based API interaction.
• M- (New CMS): A recently launched CMS built on Workers and Durable Objects, utilizing Dynamic Workers for a secure, sandboxed plugin system.

3. Methodologies and Frameworks

• The "Agent Loop": The team advocates for building agents that are persistent and addressable. By moving the "agent logic" into a Durable Object, the agent becomes a "nexus" that can connect to various clients (iOS, Web, Terminal) while keeping state synced across all of them.
• Security-First Sandboxing: Instead of adding security layers to a container, Cloudflare’s approach is to provide an environment with zero capabilities by default, then explicitly "punching holes" for specific allowed actions (e.g., outgoing fetches to specific domains).
• Worker Bundler: A tool that runs inside a worker to pull dependencies from NPM, strip types/JSX, and generate executable code, leveraging Cloudflare’s cache to avoid NPM downtime.

4. Key Arguments and Perspectives

• "AI as a Multiplayer Game": The speakers argue that AI should be inherently collaborative. Current platforms (like ChatGPT) often lack native multi-user support; Cloudflare’s infrastructure makes this "someone else's problem" (the platform's) rather than the developer's.
• Revisiting eval: The speakers suggest that "Dynamic Workers" are essentially a safe, fast, and cheap version of eval. They argue that the industry has avoided dynamic code execution for 30 years due to security risks, but modern isolation technology makes this a powerful, untapped "branch of the tech tree."
• Infrastructure Philosophy: Cloudflare’s low cost is attributed to their unique network architecture—placing hardware directly at ISP locations and bulk-buying bandwidth—rather than traditional data center models.

5. Notable Quotes

• "We call this stateful serverless, which is the most confusing thing, because servers are stateful. But it works like serverless in that it scales across the planet." — Sunil
• "I think about it how there's an entire branch of the tech tree that we haven't explored in like 30 years. And now we are giving you a fast, secure, and cheap way [to run dynamic code]." — Matt
• "Why are you generating JSON? Why don't you just generate HTML or even generate React and then just render that?" — Matt (on the benefits of generative UI).

6. Synthesis and Conclusion

The core takeaway is that Cloudflare is positioning its infrastructure—specifically Durable Objects and Dynamic Workers—as the ideal backend for the next generation of AI agents. By providing a stateful, globally distributed, and secure execution environment, they aim to abstract away the complexities of distributed systems (sync, persistence, security) so developers can focus on building agentic capabilities. The team emphasizes that their tools are "first-class" for JavaScript and Python, with support for WASM (Zig, Rust, Go) for high-performance needs.