TanStack AI: What You Need To Know To Start

TanStack AI: A Deep Dive into the Library and its Functionality

Key Concepts:

• TanStack AI: A library for building AI-powered applications, separating client and server components.
• AI Applications: Applications integrating AI as a core customer feature (e.g., AI chat).
• Adapters: Interfaces that normalize communication with different AI providers (OpenAI, Anthropic, Gemini, Ollama, etc.).
• Tool Calls: Mechanisms allowing the LLM to interact with external functions (server or client-side) to perform actions.
• Agents: LLMs equipped with tool calls, enabling them to take actions and interact with the real world.
• AG-UI Token Format: A standardized format for tokens exchanged between server and client, promoting interoperability.
• Server-Side Rendering (SSR) / Server Routes: Handling AI logic on the server for security and performance.
• Client-Side Tools: Functions executed directly in the browser, triggered by the LLM.
• Stop Conditions: Mechanisms to prevent runaway agent loops by limiting the number of iterations.

I. Introduction to TanStack AI & Architecture

TanStack AI aims to simplify the development of AI-powered applications. The core value proposition is enabling developers to integrate AI features directly into their applications, such as AI-enabled chat functionalities. The library is structured around a client-server architecture.

• TanStack AI Client: Runs in the browser (available in JavaScript, React, Solid, Vue, Svelte, and potentially Preact) and handles the presentation and interaction with the AI.
• TanStack AI Library (Server-Side): Connects to various AI backends, including local options like Ollama and cloud providers like OpenAI, Anthropic, Gemini, and Grok. It normalizes the interface, allowing code reuse across different providers by simply changing the provider configuration. The library also supports backends written in PHP or Python (using FastAPI), offering flexibility in the tech stack.

The communication between the client and server utilizes a common chunk language, facilitating interoperability. The project is moving towards adopting the AG-UI token format, a standardized format for tokens, further enhancing compatibility with different clients and servers.

II. Setting Up a TanStack AI Application

The demonstration utilizes Create Start App (latest version) with the --tanstack-ai flag to quickly scaffold a project. While TanStack AI isn’t tied to TanStack Start, it’s a convenient starting point. The example application showcases a chat interface.

• API Key Configuration: The application requires an API key (e.g., Anthropic) to function. The example supports multiple providers including OpenAI, Gemini, and Ollama, allowing developers to easily switch between them.
• Project Structure: The project is organized with server routes located in src/routes/demo/api/ai-chat.

III. Server-Side Implementation (API Route)

The core logic resides within a server route (src/routes/demo/api/ai-chat). This route handles POST requests representing user messages.

• Provider Selection: The server dynamically selects the AI provider based on the presence of an API key in the environment variables.
• Adapter Lookup: An adapterConfig lookup table maps the selected provider to its corresponding adapter (e.g., anthropicText for Anthropic, openaiText for OpenAI).
• Adapters Explained: Adapters provide a consistent interface for interacting with different AI providers, abstracting away their specific API details. Each provider exports a text adapter for text completion tasks, essential for chat functionality.
• Chat Function: The chat function from TanStack AI takes the adapter, a system prompt (defining the AI’s role), a list of messages (the conversation history), and an AbortController (for request cancellation) as input.
• Streaming Response: The chat function returns a stream of tokens. This stream is then converted into a Server-Sent Events (SSE) response using toServerSentEventsResponse, enabling real-time updates to the client. The library is transitioning to using the AG-UI token format for these chunks.

IV. Client-Side Implementation (UI)

The client-side component (AI Chat) utilizes the @tanstack/ai-react library, a thin wrapper around @tanstack/ai-client.

• useChat Hook: The useChat hook simplifies the integration with the server. It takes chatClientOptions including a connection (established via fetchServerSentEvents) and a list of tools.
• fetchServerSentEvents: This function handles the SSE stream from the server, passing the data to the chatClient.
• Tool Integration: The useChat hook accepts a list of both server-side and client-side tools. Providing both ensures proper typing of tool calls within the chat messages.
• Message Formatting: The client formats the messages received from the chatClient, extracting text and tool calls for display.

V. Tool Calls and Agents

The example demonstrates the use of tool calls to interact with a guitar database.

• Server-Side Tool (getGuitars): This tool retrieves a list of guitars from a database. It’s defined with a name, description, input schema (none in this case), and output schema. The .server function indicates that it executes on the server.
• Client-Side Tool (recommendGuitar): This tool is executed in the browser. It takes an ID as input and returns it. The example adds an alert to demonstrate when the tool is called.
• Agentic Behavior: The combination of an LLM and tool calls creates an agent capable of taking actions. The LLM can analyze user requests, determine which tools to use, and execute them to fulfill the request (e.g., recommending a guitar based on user preferences).

VI. Agent Control & Safety

To prevent uncontrolled agent behavior, TanStack AI implements stop conditions.

• Maximum Iterations: The agentLoopStrategy limits the number of round trips between the LLM and the tool calls to a maximum of five iterations. This prevents infinite loops and ensures the agent remains responsive.

VII. Conclusion

TanStack AI provides a robust and flexible framework for building AI-powered applications. Its key strengths lie in its client-server architecture, provider abstraction through adapters, and the ability to integrate both server-side and client-side tools. The move towards the AG-UI token format promises increased interoperability and standardization within the AI ecosystem. By leveraging tool calls, developers can create intelligent agents capable of performing complex tasks and enhancing the user experience. The library’s focus on safety, through mechanisms like stop conditions, ensures responsible AI application development.