Excalidraw: AI and Human Whiteboarding Partnership - Christopher Chedeau

Key Concepts

• AI and Human Partnership in Product Development
• Physical vs. Virtual Whiteboarding
• LLMs (Large Language Models) and their applications
• Domain-Specific Languages (DSLs)
• AI Integration in Excalidraw
• Active Procrastination
• Mermaid JS
• AI-native design

1. Introduction and the Story of Excalidraw

• The speaker, the creator of Excalidraw, discusses the integration of AI and human collaboration in whiteboarding.
• The origin story of Excalidraw began with "active procrastination" at Facebook in January 2020. Instead of doing performance reviews, the speaker tried to use Ziblr for blog post illustrations, but when it was down, he decided to reimplement it, leading to the creation of Excalidraw.
• The initial demo on Twitter received unexpected positive feedback, marking the beginning of Excalidraw.
• The speaker notes that today, he would use coding agents to start such a project, referencing Anthropic's keynote demo where an AI agent reimplemented a table layout in Excalidraw.

2. The Shift to Online Whiteboarding During COVID-19

• The COVID-19 pandemic forced people to work from home, highlighting the need for online whiteboarding solutions.
• The speaker was surprised by the adoption of Excalidraw as a whiteboarding tool, as it was initially designed for blog post illustrations.
• Existing whiteboarding tools at the time mirrored the limitations of physical whiteboards, such as fixed sizes and difficult text input.
• Excalidraw offered advantages like infinite canvas, text boxes, select-and-delete functionality, shape recognition, and copy-pasting, which were not available in traditional online whiteboards.
• This highlighted the difference between physical and virtual whiteboarding, a parallel to the current state of AI integrations.

3. AI Integrations: What Works and What Doesn't

• The speaker emphasizes that many current AI integrations simply sprinkle AI on existing processes, leading to suboptimal results.
• Unlike online whiteboarding, where users had prior experience, AI is new, making experimentation and sharing lessons crucial.
• The speaker shares experiences integrating AI into Excalidraw, including successes and failures.

3.1. Realistic Image Generation

• Gore Cam's AI model that generates real-time beach images from Excalidraw diagrams was impressive.
• However, it wasn't practical because users don't typically draw realistic images in Excalidraw.
• Key Lesson: Just because an AI model exists doesn't mean it should be integrated into a product. It's important to consider if the integration makes sense for the users and the product's purpose.

3.2. Diagram Generation with LLMs

• Initial attempts to generate diagrams directly with LLMs failed, producing nonsensical results.
• However, LLMs were successful in generating Mermaid JS syntax, a domain-specific language for describing diagrams.
• Mermaid JS: A text-based diagramming tool that uses a specific syntax to define diagrams like flowcharts, sequence diagrams, etc.
• Iancu Chișcă implemented an Excalidraw backend for Mermaid JS, enabling the conversion of Mermaid JS code into editable Excalidraw components.
• This allows users to input a prompt, generate a diagram in Mermaid JS, and then convert it into an Excalidraw drawing that can be further modified.
• This integration exemplifies the "human and AI partnership," where AI generates a starting point, and humans refine it to their needs.
• The speaker envisions a future where users can iteratively modify the AI-generated content.

3.3. Other AI Integrations in Excalidraw

• Document Naming: LLMs are used to generate relevant names for untitled documents based on their content.
• Illustration Generation: An upcoming feature (currently in closed alpha) will allow users to generate illustrations within Excalidraw.
• Logo Background Removal: The speaker challenges the audience to create a browser-based AI model that can reliably remove backgrounds from logos for integration into Excalidraw.

4. Conclusions and Key Takeaways

• The design product industry is currently in a transition phase, similar to the shift from physical to virtual whiteboards, and needs to discover AI-native interaction methods.
• Resist the urge to integrate every available AI model into your app. Focus on AI integrations that genuinely enhance the product and help users achieve their goals.
• LLMs work best with structured data. Representing problem domains in domain-specific languages (DSLs) can significantly improve the effectiveness of AI integrations.
• Domain-Specific Language (DSL): A specialized programming language designed for a particular domain or task, allowing for more concise and expressive code within that domain.

5. Notable Quotes

• "Just because there's a model that exists doesn't mean that it should be within your product."
• "LLMs love structure."