From Arc to Dia: Lessons learned building AI Browsers – Samir Mody, The Browser Company of New York

Optimizing for AI: From Arc to DIA – A Deep Dive into The Browser Company’s Transition

Key Concepts:

• LLMs (Large Language Models): Powerful AI models like GPT capable of understanding and generating human-like text.
• Prompt Engineering: The art and science of crafting effective instructions (prompts) for LLMs to achieve desired outputs.
• Jeba: A sample-efficient technique for refining LLM systems without relying on Reinforcement Learning (RL) or fine-tuning.
• Model Behavior: The intentional design and shaping of an LLM’s responses, personality, and overall functionality.
• Prompt Injection: A security vulnerability where malicious prompts override an LLM’s intended instructions.
• Hill Climbing: An iterative optimization technique used to improve prompts and model performance.
• Dogfooding: Using your own product internally for testing and feedback.

I. The Evolution from Arc to DIA: A Vision for the AI Browser

The Browser Company began in 2019 with a mission to fundamentally rethink the internet browsing experience. Recognizing that the browser hadn’t kept pace with evolving user needs, they launched Arc in 2022 – a browser focused on personalization, organization, and a more refined user experience. However, the team felt Arc was merely an incremental improvement, not the transformative vision they held.

The turning point came with access to LLMs (like GPT models) in 2022. Initial experimentation with AI features within Arc led to a broader thesis: AI would fundamentally change how people interact with the internet, and therefore, the browser itself. This realization spurred the development of DIA, an “AI-native” browser launched earlier in the year, designed to function with a persistent AI assistant that learns user behavior, manages tabs, and streamlines workflows. While still evolving, DIA represents a significant step towards realizing this vision. As Samir stated, “It is not easy to build a product. Let alone two, the latter of which an AI native one.”

II. Optimizing Tools & Processes for Accelerated Iteration

A core principle at The Browser Company is rapid iteration – building, shipping, and learning faster than competitors. This principle has been amplified with the introduction of AI. The company has strategically invested in tooling to accelerate this process, focusing on four key areas:

• Prototyping for AI Features: Quickly testing new AI-powered ideas.
• Building & Running Evaluations (Evals): Systematically assessing the performance of AI features.
• Data Collection: Gathering data for both training and evaluating LLMs.
• Automation for Hill Climbing: Automating the process of refining prompts and model behavior.

Initially, prototyping involved a rudimentary prompt editor accessible only to engineers, resulting in slow iteration and a lack of contextual awareness. The solution was to integrate these tools directly into the company’s internal product workflow. Now, everyone – from the CEO to new hires – can ideate, iterate, and refine DIA features with full access to prompts, models, parameters, and relevant context. This has reportedly resulted in a “10x” increase in iteration speed and broadened participation in the product development process. Tools are also used to optimize the “memory knowledge graph” and “computer use mechanism” with tens of different strategies tested before implementation.

III. Jeba: Refining AI Prompts Through Reflective Mutation

Recognizing that ideation is only half the battle, The Browser Company adopted “Jeba,” a technique inspired by a recent research paper, to optimize and refine AI prompts. Jeba offers a sample-efficient way to improve LLM systems without requiring resource-intensive methods like Reinforcement Learning (RL) or full model fine-tuning.

The process involves:

1. Seeding: Providing the system with an initial set of prompts.
2. Execution & Scoring: Running the prompts across a set of tasks and assigning scores based on performance.
3. Prompt Selection: Utilizing a “PA selection” mechanism to identify the most effective prompts.
4. Reflective Mutation: Employing an LLM to analyze the successful prompts, identify strengths and weaknesses, and generate new, improved prompts.
5. Iteration: Repeating the process to continuously refine prompt performance.

Jeba’s key innovations lie in its reflective prompt mutation, its exploration of a wider prompt space, and its ability to tune text rather than model weights. A simple example provided showed Jeba optimizing a basic prompt based on defined metrics.

IV. Model Behavior as a Craft & Discipline

The Browser Company views “model behavior” as the core function responsible for defining, evaluating, and shipping the desired behavior of LLMs. This encompasses translating principles into product requirements, crafting effective prompts, designing robust evaluations, and ultimately shaping the personality and functionality of the AI assistant within DIA.

This process is broken down into three areas:

• Behavior Design: Defining the desired product experience – style, tone, and response format.
• Data Collection & Measurement: Gathering data for training and evaluation through rigorous testing.
• Model Steering: The technical implementation – prompting, model selection, context window management, and parameter tuning.

The team draws an analogy to the evolution of web design, noting that early websites were purely functional, but as technology advanced, design and user experience became increasingly sophisticated. Similarly, model behavior is evolving from simple prompt-response systems to more complex agent behaviors involving goal-directed reasoning, self-correction, and personality shaping. The team believes this field will become increasingly specialized and crucial for product companies. A key anecdote highlighted the contribution of a strategy and ops team member who rewrote all the prompts on a weekend, unlocking a significant improvement in product quality and leading to the formation of a dedicated “model behavior” team.

V. AI Security: Prompt Injection as an Emergent Property of Product Design

The Browser Company recognizes that AI security is not an afterthought but an inherent aspect of product development. They specifically addressed the threat of “prompt injection” attacks – where malicious prompts override an LLM’s instructions, potentially leading to data exfiltration or malicious actions.

Browsers are particularly vulnerable due to their access to private data, exposure to untrusted content, and ability to interact with external systems. Traditional technical defenses, like wrapping untrusted content in tags or separating instructions from data, are often easily bypassed.

The company’s approach is to design products with security as a foundational principle. The example of DIA’s autofill tool illustrates this: before filling a form, the user is presented with the data in plain text for confirmation, providing control and awareness. This approach is consistently applied across features like scheduling events and writing emails.

VI. A Fundamental Company Shift

The transition from Arc to DIA wasn’t simply a product evolution; it represented a fundamental shift in the company’s approach. This shift impacts not only how they build products but also how they train employees, hire new talent, and foster collaboration. Samir emphasized the importance of embracing technological shifts “with conviction,” highlighting that the company’s success hinges on its ability to adapt and innovate in the rapidly evolving landscape of AI.

Notable Quote:

“It is not easy to build a product. Let alone two, the latter of which an AI native one.” – Samir, Head of AI Engineering at The Browser Company.