Stanford Seminar - Build less, design more (interactive systems)

Key Concepts

• Composable Software: Building software from reusable components, like Lego bricks, to create a combinatorial design space.
• Interoperable Software: Software that works seamlessly with other software, expanding the design space exponentially.
• APIs (Application Programming Interfaces): Defined interfaces that allow different software systems to interact, promoting composability and interoperability.
• Data Navigator: A framework for creating accessible data representations by decoupling structure, input, and rendering.
• Mosaic: A framework for interoperable visualizations and views with databases, enabling flexible, scalable, and interoperable data exploration.
• Any Widget: A specification for building JavaScript widgets primarily for Python notebooks, facilitating communication between front-end and back-end.
• Embedding Atlas: A library of reusable components (tables, embedding views, charts) built on Mosaic for creating generic analytic tools.
• Texture: A tool for text data exploration that leverages common components (embedding view, charts, instance view) and supports filtering over joint dimensions.
• Vega-Lite: A visualization grammar that allows users to create a wide range of charts by combining marks and encodings.
• M4 Optimization: An optimization technique used in Mosaic to reduce the amount of data needed to render pixel-perfect charts, improving performance.

Accelerating HCI Research

The speaker, Dominic, argues that HCI research can be accelerated by shifting focus from building one-off tools to designing composable and interoperable APIs and frameworks. He posits that by investing in design and how systems work together, researchers can "build more" in the long run.

HCI Research as a Science

• HCI research is framed as a scientific process involving hypothesis creation, experiment design, and evaluation.
• The goal is to accelerate the cycle of hypothesis -> experiment -> evaluation to generate knowledge faster.
• HCI combines traditional scientific experimentation with design, where new artifacts are developed to test hypotheses.

The Problem with One-Off Tools

• Tool-driven HCI research often results in one-off software that is rarely used by others.
• Foundations and frameworks, on the other hand, enable the creation of more tools and faster experimentation.
• Framework research itself is a scientific endeavor that tests hypotheses about how tools can be built.

Building Composable Software

• Composable software involves combining individual components to create a large design space.
• A quote from Tuki emphasizes the importance of reorganizing existing techniques and making them easy to assemble in new ways.
• Vega-Lite is presented as an example of a composable system that allows users to create a wide range of visualizations through atomic modifications.
• The development of Vega-Lite involved extensive discussions and attention to detail, resulting in widespread adoption and influence.
• Vega and Vega-Lite are considered potential candidates for a "lingua franca" of data visualization.

Building Interoperable Software

• Interoperability goes beyond composition by allowing software to work with other people's software, further expanding the design space.
• Careful design and small, well-defined interfaces are crucial for managing the complexity of interoperable systems.
• Herb Simon's quote from "The Sciences of the Artificial" highlights the importance of the interface between the inner workings of an artifact and its outer environment.
• Focusing on the outer behavior and defining it well is key to building interoperable systems.

Benefits of Interoperable Software

• Reduced glue code: Reusable interfaces minimize the amount of code needed to make systems compatible.
• Examples of standards that promote interoperability include HTTP, Apache Arrow, Language Server Protocol, and SQL.
• APIs are essential for composing software, reducing code, and accelerating science.
• Designing interoperable software is intellectually challenging and a valuable activity for researchers.

Examples of Interoperable Visualization Systems

Any Widget

• Any Widget is a specification for building JavaScript widgets for Python notebooks.
• It addresses the problem of tight coupling between Jupyter notebooks and widget implementations.
• Any Widget uses standard ES modules and provides an adapter to make widgets compatible with different front ends.
• It simplifies widget development and has seen widespread adoption in the Jupyter notebook community.

Mosaic

• Mosaic is a framework for interoperable visualizations and views with databases.
• It aims to provide more flexibility, interoperability, and scalability than existing visualization toolkits.
• Mosaic sits between visualization clients and a backing database, allowing users to build interactive dashboards.
• It uses Apache Arrow for efficient data transport and supports interactions through a coordinator that sends SQL queries to the database.
• Mosaic includes vgplot, a grammar of interactive graphics, and supports various mark types.
• It applies optimizations like M4 to reduce the amount of data needed for rendering, improving performance.
• Mosaic can be embedded into other environments like Jupyter notebooks using Any Widget.

Embedding Atlas

• Embedding Atlas is a library of reusable components built on Mosaic for creating generic analytic tools.
• It includes virtualized tables, embedding views, and charts that can be combined in different configurations.
• Embedding Atlas can be embedded into Jupyter notebooks and Streamlit apps.
• It allows users to explore data sets, compute densities, and filter data in real-time.

Texture

• Texture is a tool for text data exploration that leverages common components from Mosaic.
• It includes an embedding view, charts, and an instance view that are cross-linked.
• Texture supports filtering over joint dimensions, allowing users to explore relationships between text and other data.
• It enables users to perform various text analysis tasks, such as paper corpus exploration, topic analysis, and prompt comparisons.

Data Navigator

• Data Navigator is a framework for creating accessible data representations.
• It decouples structure, input, and rendering to support a wide range of navigation patterns.
• Data Navigator uses a graph structure to represent the relationships between data elements.
• It supports various input methods, such as keyboard, gesture, and speech.
• Data Navigator can be used with screen readers and is designed to be efficient for large data sets.

Conclusion

The speaker advocates for a shift in HCI research towards designing composable and interoperable APIs and frameworks. By focusing on design and how systems work together, researchers can accelerate the scientific process and create more impactful tools. The examples of Any Widget, Mosaic, Embedding Atlas, Texture, and Data Navigator demonstrate the potential of this approach to enable flexible, scalable, and accessible data exploration. The key takeaway is that investing in design and interoperability can lead to a more efficient and impactful HCI research ecosystem.