Code Mode: Let the Code do the Talking - Sunil Pai, Cloudflare

Key Concepts

• Code Mode: A paradigm shift where LLMs generate and execute code (e.g., JavaScript) to interact with systems, rather than relying on traditional JSON-based tool calling.
• Capability-Based Security: A security model where an execution environment starts with zero permissions and is explicitly granted specific capabilities (APIs) to perform tasks.
• V8 Isolates: Lightweight, secure execution environments used to run code with minimal startup time and high performance.
• Harness: A software architecture that provides a safe, sandboxed environment for AI agents to execute generated code.
• Generative UI: The ability for AI to dynamically create custom user interfaces tailored to individual user needs on the fly.

---

1. The Problem with Traditional Tool Calling

Sunil Pai argues that traditional "JSON back-and-forth" tool calling breaks down at scale. When an application requires hundreds of tools (e.g., integrating Jira, Google Services, and internal wikis), the context window becomes cluttered, leading to slow performance and high token usage.

• The Cloudflare Case Study: Managing 2,600 API endpoints via standard tool calling would require ~1.2 million tokens per call. By switching to a "Code Mode" approach—where the model generates code to interact with the API—they reduced token usage to 1,000 tokens (a 99.9% reduction).
• Efficiency: Instead of multiple round-trips, the model generates a single script that executes all necessary logic in one shot.

2. Methodology: The "Harness" Architecture

The proposed architecture moves away from static tool definitions toward a dynamic execution environment.

• The Sandbox: A secure, ephemeral environment (using V8 isolates) that starts with no network access or API permissions.
• Explicit Capabilities: Developers expose specific functions as APIs to the sandbox. The agent can only interact with the system through these pre-defined, safe interfaces.
• Observability: Because the agent executes code, developers can log and audit the exact script that was run, providing full transparency into why a specific action was taken.

3. Emergent Behavior and State Machines

Pai highlights a shift from "generating a program" to "inhabiting a state machine."

• Example: In a collaborative canvas application (like Excalidraw), instead of the model generating a new app to play Tic-Tac-Toe, it was given access to the existing state (an array of strokes). The model recognized the state as a game board and interacted with it directly by adding new strokes.
• Key Insight: The model didn't need "Tic-Tac-Toe code"; it understood the system's state and manipulated it, demonstrating that LLMs can act as intelligent operators within existing software environments.

4. Future Implications

• Democratization of Power: Historically, only programmers could write scripts to automate tasks. LLMs now allow non-technical users to "program" their environment by describing tasks in natural language, which the model then converts into executable code.
• Hyper-Personalized UI: Rather than building "lowest common denominator" interfaces, developers can use LLMs to generate custom UIs for every user based on their specific context, history, and needs.
• Developer Experience (DX) for Agents: As agents become the primary users of systems, developers must focus on "agent-friendly" design: clear documentation (Markdown), descriptive error messages, and discoverable APIs.

5. Notable Quotes

• "We stopped generating a program and instead started inhabiting the state machine."
• "Your next billion users are these little robots that are generating code for you."
• "For the longest time, programmers... had infinite power to interact with any system... everyone else got buttons and forms. That distinction is breaking."

Conclusion

The transition to "Code Mode" represents a fundamental change in software architecture. By treating LLMs as agents that execute code within secure, capability-restricted sandboxes, developers can build systems that are more efficient, highly observable, and capable of delivering deeply personalized user experiences. The future of software development lies in building robust "harnesses" that allow these agents to safely interact with the vast, complex systems we build.