This AI moves a REAL Kayak on a map

Key Concepts

• Cloud Developer Relations
• Google Cloud Next
• Destination Planner Demo
• Gemini
• Event-Driven Architecture (Pub/Sub)
• Three-Tier Web App (Frontend, Backend, Database)
• Cloud Run
• Raspberry Pi Agent
• Arduino Controller
• Hardware-Software Translation
• AI-Powered Response Critique

Destination Planner Demo Overview

The video showcases a physical demo built for Google Cloud Next, a destination planner for Lake Washington in Seattle. The demo integrates Google Cloud services, AI (Gemini), and hardware components to simulate a kayak journey between nine pre-defined locations. The goal is to demonstrate how developers can leverage these technologies to create engaging and interactive applications.

Architecture and Implementation

1. Three-Tier Web Application

• Frontend: Built with Vite and Node.js, responsible for user interface and interaction.
• Backend: Developed in Python, handles user requests and orchestrates the overall process.
• Hosting: Both frontend and backend are hosted on Google Cloud Run, a serverless compute platform.

2. Event-Driven Architecture with Pub/Sub

• When a user selects a destination, the backend publishes a message to a Pub/Sub topic.
• Pub/Sub enables asynchronous communication between the backend and the Raspberry Pi agent.

3. Raspberry Pi Agent

• A Python-based agent running on a Raspberry Pi, subscribed to the Pub/Sub topic.
• Functionality:
  • Receives destination information from Pub/Sub.
  • Maps the destination to specific routes and traces using a pre-defined dictionary.
  • Sends commands to an Arduino controller to move the kayak.

4. Arduino Controller and Hardware

• The Arduino controller receives commands from the Raspberry Pi agent.
• Hardware Components:
  • Rails and motors: Physically move the kayak along the mapped route.
  • Magnet: Secures the kayak to the rails.
  • Fidgets controller: Controls light assemblies to indicate arrival at the destination.

Demo Workflow

1. User interacts with the frontend to select a destination (e.g., "American football").
2. The backend uses a prompt template and Gemini to determine the most relevant location (e.g., University of Washington Husky Stadium).
3. The backend publishes a message to Pub/Sub with the destination information.
4. The Raspberry Pi agent receives the message and determines the corresponding route.
5. The agent sends commands to the Arduino controller to move the kayak along the route.
6. As the kayak moves, the frontend displays real-time updates.
7. Upon arrival, the Fidgets controller activates a light to indicate the destination has been reached.

Areas for Improvement

The presenter identifies several areas for improvement in future iterations of the demo:

1. Synchronous Process for Kayak Movement: Implement a mechanism to ensure the frontend accurately reflects the kayak's progress and only indicates completion when the kayak has fully reached its destination. Currently, the frontend updates are not perfectly synchronized with the physical movement.
2. Hardware-Software Translation: Improve the control and precision of the kayak movement by using a motor with a brake. Consider using CAN bus for more robust hardware communication.
3. AI-Powered Response Critique: Integrate Gemini to evaluate the accuracy of the initial location suggestions. If the suggestion is incorrect, use Gemini to provide a corrected response based on the available destinations. This would involve a multi-turn interaction with Gemini, acting as a "rater" to refine the prompt and improve the results.

AI-Powered Response Critique Example

The presenter suggests using Gemini to critique the initial location suggestion. For example, if the initial response for "American football" is incorrect, Gemini could be used to:

1. Identify the error.
2. Provide a list of the nine available destinations.
3. Suggest the correct destination (University of Washington Husky Stadium).
4. Feed this corrected information back into the prompt to refine the results.

Notable Quotes

• "What that ultimately means is that I get to spend a lot of time playing with tech, and then helping developers actually build things with Google Cloud..."
• "...translating software and hardware's actually very difficult."

Technical Terms

• Vite: A frontend build tool.
• Node.js: A JavaScript runtime environment.
• Cloud Run: A serverless compute platform on Google Cloud.
• Pub/Sub: A messaging service for event-driven architectures.
• Raspberry Pi: A small, low-cost computer.
• Arduino: An open-source electronics platform.
• Fidgets: A platform for connecting physical interfaces to software.
• CAN bus: A robust communication protocol commonly used in automotive and industrial applications.
• Gemini: Google's family of generative AI models.

Synthesis/Conclusion

The destination planner demo effectively showcases the integration of Google Cloud services, AI, and hardware to create an interactive and engaging experience. The demo highlights the potential of event-driven architecture, serverless computing, and AI-powered decision-making in real-world applications. While the presenter acknowledges areas for improvement, the demo serves as a valuable example for developers looking to leverage these technologies in their own projects. The key takeaways are the importance of careful hardware-software integration, the benefits of event-driven architectures, and the potential of AI to enhance user experiences and improve application accuracy.