Build next-generation UIs with the HTML-in-Canvas API

Key Concepts

• HTML in Canvas API: A browser API that allows rendering DOM elements directly into a <canvas> (2D), WebGL, or WebGPU texture while maintaining interactivity and accessibility.
• DOM (Document Object Model): The live, interactive model of a web page that supports browser features like accessibility, translation, and text selection.
• Canvas/WebGL/WebGPU: Low-level graphics APIs used for high-performance rendering, which traditionally lack built-in UI/text layout capabilities.
• Layout Subtree: A property set on the canvas to inform the browser that the canvas contains interactive DOM content.
• Origin Trial: A testing phase allowing developers to experiment with new web features in production environments before they become standard.

---

1. Main Topics and Technical Details

The HTML in Canvas API bridges the gap between the high-level, feature-rich DOM and the low-level, high-performance rendering of Canvas/WebGL/WebGPU.

• The Problem: Traditionally, rendering UI into a canvas "breaks" browser features like accessibility, text selection, find-in-page, and autofill because the browser sees the canvas as a flat grid of pixels rather than structured content.
• The Solution: By using this API, developers can keep the UI as standard HTML/DOM elements while "painting" them into a texture. The browser remains aware of the underlying DOM structure, preserving all native browser integrations.

2. Step-by-Step Implementation

The API follows a three-phase process:

1. Setup: Set the layout subtree property on the <canvas> tag. This notifies the browser to perform hit-testing and expose the content to accessibility trees.
2. Rendering:
   • 2D Context: Use drawElementImage(element, x, y) within an onpaint event to ensure the UI updates during browser repaints.
   • WebGL: Use texElementImage2D(target, level, internalformat, format, type, element).
   • WebGPU: Use copyElementImageToTexture(element, texture, ...) to map the DOM element to a GPU texture.
3. Transform Updates: The API returns a transform that must be applied to the DOM element’s CSS transform property. This synchronizes the visual position of the element with its logical position in the browser.

3. Framework Integration

The speaker highlighted that developers do not need to handle complex coordinate mapping manually, as major frameworks are adding support:

• Three.js: Uses THREE.HTMLTexture to map DOM elements directly to material textures.
• PlayCanvas: Uses event listeners to upload the texture whenever the browser repaints the element, allowing for seamless integration into 3D scenes.

4. Key Arguments and Benefits

• Accessibility & SEO: Because the content remains in the DOM, screen readers can parse it, and search engines/AI agents can index it as text rather than an opaque image.
• Browser Features: Users retain the ability to use "Find in Page," browser translation, dark mode, and text selection, even when the UI is rendered inside a 3D WebGL object.
• Performance: It offloads complex text layout and responsive design to the browser’s native engine, reducing the need for heavy custom JavaScript layout libraries.

5. Notable Quotes

• "The HTML in Canvas API is the bridge that gives you the best of both worlds by letting you paint DOM elements into a Canvas or WebGPU texture." — Thomas Nattestad
• "It's vital that you update this transform whenever you repaint the element. If you stop painting the element, it's also critical to remove it from the canvas element."

6. Real-World Applications

• 3D Billboards/UI: Rendering interactive forms or sliders onto 3D objects (e.g., a 3D book where text can be selected and translated).
• Refractive Overlays: UI elements that appear to refract through 3D models while remaining fully interactable.
• Animated SVGs: Rendering animated SVGs directly into WebGL textures without extra code.
• AI/Indexing: Allowing AI agents to extract text from complex 3D visualizations because the content is still exposed in the HTML tree.

7. Synthesis and Conclusion

The HTML in Canvas API represents a significant shift in web graphics, effectively ending the trade-off between high-performance 3D rendering and standard web usability. By maintaining the DOM's integrity within a canvas, developers can build sophisticated, visually stunning applications (like those being prototyped by Figma, Adobe, and Miro) without sacrificing accessibility, localization, or user-friendly browser features. As of May 2026, the API is in an Origin Trial, and developers are encouraged to implement fallbacks for browsers where the API is not yet supported.