Key Concepts
Real-time co-scientist, AI co-scientist, data overload in science, hypothesis generation, experiment automation, open-source hardware, crystal growth, fermentation, multimodal data, context assembly, web USB API, simulation, Jubilee motion platform.
Real-Time Co-Scientist: A Live Demo and Vision
Stefania introduces the concept of a real-time co-scientist, drawing an analogy to pair programming or coding copilots, but for scientific experiments. The core idea is to use AI to analyze data from experiments in real-time, providing feedback and insights to the scientist.
Live Demo: Microbit and Sensors
A live demo showcases a Microbit board with a Jack DAC connected, measuring temperature. A heat pad is used to increase the temperature, and the data is sent to a "science assistant" for real-time analysis. The system identifies stable, dark, and quiet ambient conditions and reports the temperature. The user can define a protocol to provide context for the AI's feedback, specifying the type of experiment and constraints. Custom pages can be created for repeated experiments to monitor and plot data in real-time.
REC Camera: Autonomous Object Tracking
The REC camera, an open-source hardware component, is demonstrated tracking the speaker. The camera runs a model on-device and can be trained to track specific objects like crystal growth. It operates over Wi-Fi, allowing for remote monitoring and control of experiment conditions.
The Need for AI Co-Scientists
Stefania argues that AI can address data overload and complexity in science. AI can assist with:
- Data Analysis: Analyzing data quickly and at scale.
- Hypothesis Generation: Generating new hypotheses and identifying blind spots.
- Experiment Acceleration: Testing multiple hypotheses simultaneously.
Inspiration: DeepMind's AI Co-Scientist
The talk is inspired by a DeepMind paper on AI co-scientists, which demonstrated the potential of AI agents to perform various scientific roles, such as analyzing papers, summarizing information, ranking options, and developing research plans.
Case Study: Gene Transfer Mechanisms
The DeepMind AI co-scientist replicated a discovery about gene transfer mechanisms that took scientists 12 years in just two days, without prior knowledge of the data.
Case Study: Liver Fibrosis Treatment
The AI co-scientist identified potential drug targets for liver fibrosis treatment, which were then validated in wet lab experiments by experts.
Real-Time Hypothesis Formulation
Stefania proposes moving beyond asynchronous data analysis to real-time hypothesis formulation based on empirical data observed in the lab. This concept is also inspired by the "era of experience" described by Silver and Sutton, where AI learns from continuous interaction with the environment.
System Overview and Architecture
The system is built as a React app with various input sources:
- Jack DAC sensors via USB
- Webcams
- Text input
- Voice input
These inputs are converted into webhooks and sent to a backend that communicates with the Gemini API.
Information Flow
Physical sensors (e.g., Jack DAC) use the web USB API to send data to the frontend hooks. The context assembly dynamically builds a context based on the available modalities (text, voice, image, chat history) and the defined experiment protocol. This context is then sent to the API.
Unified Context Assembly
The context assembly is a crucial component that dynamically injects context based on the available sensors and the type of experiment.
Hardware Ingredients and Experiment Design
Stefania emphasizes the importance of listing available hardware components and designing experiments that can be measured in real-time, are safe to conduct at home, and are portable.
Experiment: Crystal Growth
The crystal growth experiment involves oversaturating a solution with salt in hot water and then gradually cooling it to induce nucleation and crystal growth. Key parameters to measure include:
- Salt dissolution rate
- Nucleation sites
- Crystal growth rate (affected by temperature and concentration)
The experiment uses a microscope to record crystal growth and sensors to measure temperature and humidity. The data is collected in a CSV file and analyzed to determine the crystal growth rate.
Insight: Crystal formation occurs in bursts once critical saturation is reached.
Experiment: Fermentation
The fermentation experiment involves controlling the amount of salt and sugar in dough and measuring the growth rate and CO2 production at different temperatures.
REC Camera for Mobile Object Tracking
The REC camera is used for mobile object tracking and can be trained to track specific objects.
Open-Source Ecosystem and Future Directions
Stefania highlights the open-source ecosystem supporting lab equipment recreation and automation, including:
- Jubilee motion platform (University of Washington)
- Open bioreactor
- Workshops on automating scientific experiments
Future: Simulation Integration
The future direction involves integrating simulations based on real-world experiment data. This would allow for exploring a wider range of conditions and identifying optimal parameters for experiments. For example, simulating bacteria colony growth to determine the best conditions for real-world experiments.
Conclusion
Stefania concludes by emphasizing the potential of real-time AI co-scientists to accelerate scientific discovery and the importance of open-source tools and collaboration in this field. She also mentions an upcoming AI education summit.
AI summaries can miss context or contain errors. Check important details against the original video.