Introducing TanStack AI Code Mode

Key Concepts

• Code Mode: A framework by TanStack that allows LLMs to write and execute TypeScript code within an isolated environment instead of relying on traditional, multi-step tool calling.
• Isolate: A secure, sandboxed environment (e.g., QuickJS, Node, or Cloudflare Workers) where the generated TypeScript code is executed.
• N+1 API Pattern: An inefficient data-fetching pattern where an LLM makes multiple sequential requests to retrieve related data, leading to high latency and context bloat.
• Code Mode Skills: A feature that allows the LLM to persist and reuse generated TypeScript code as a reusable "skill" or tool, significantly reducing future processing time and context usage.
• Dynamic UI Generation: The ability for Code Mode to generate code that renders UI components directly, moving beyond simple JSON-based UI specifications.

---

1. The Problem: Inefficiency of Traditional Tool Calling

Traditional LLM tool calling often suffers from the "N+1" problem. When an agent needs to perform a task like calculating the average price of products, it must:

• Request a tool call.
• Wait for the server to execute the tool.
• Send the result back to the LLM.
• Repeat this cycle for every data point.

Evidence: In the provided example, calculating the average shoe price required four round trips to the server, consuming 9.8 KB of context and taking 27 seconds. Furthermore, LLMs often struggle with mathematical accuracy, leading to incorrect calculations (e.g., 134.50 vs. the actual 137.75).

2. The Solution: Code Mode

Code Mode shifts the paradigm by asking the LLM to write a single TypeScript script that performs the logic (using map, reduce, promise.all, etc.) and executes it in an isolated environment.

• Performance Gains: The same task was completed in two LLM calls, consuming only 1.7 KB of context and taking 8 seconds.
• Accuracy: Because the calculation is performed by the TypeScript engine rather than the LLM’s internal reasoning, the mathematical output is precise.

3. Implementation Framework

TanStack AI models Code Mode as a standard tool. The process involves:

1. Isolate Driver: Creating a driver (QuickJS, Node, or Cloudflare Worker) to run the code.
2. Tool Injection: Injecting specific tools into the isolate so the generated code has access to necessary APIs (e.g., getProductList, getProductById).
3. System Prompting: The framework automatically generates a system prompt containing TypeScript typings for the injected tools and instructions on how to invoke the executeTypeScript function.

4. Advanced Features: Skills and Dynamic UI

• Code Mode Skills: Once the LLM generates a successful script, it can be saved as a "Skill." When the user asks the same question again, the agent retrieves the pre-written code, reducing the process to 0.5 KB of context and 3 seconds of execution time.
• Dynamic UI: Code Mode can generate code that renders UI components directly. Instead of relying on rigid JSON schemas, the LLM writes code to build the interface, allowing for highly flexible and complex data reporting.

5. Notable Quotes

• "We know that there are things that they're [LLMs] very efficient at and other things that they're not so efficient at. And one of the things that they're not so efficient at is tool calling."
• "I've seen a lot of JSON specified UI stuff out there for LLMs but why not just use code? This works really, really well."

6. Synthesis and Conclusion

The TanStack Code Mode framework represents a significant evolution in agentic workflows. By moving from sequential, LLM-driven tool orchestration to code-driven execution, developers can bypass the inherent latency and mathematical limitations of LLMs. The ability to persist these scripts as "Skills" and use them to generate dynamic UI components provides a robust, scalable, and highly efficient architecture for building complex AI applications. Developers are encouraged to explore the TanStack AI monorepo to implement these patterns in their own data-heavy environments.