Tuning custom Gemma models for high speed computer vision

Key Concepts

• Food Waste Mitigation: Addressing the global issue where 30–40% of food grown is wasted, accounting for approximately 10% of global greenhouse gas emissions.
• Edge AI: Running sophisticated machine learning models (specifically Gemma) directly on hardware devices rather than in the cloud to reduce latency, cost, and privacy concerns.
• Computer Vision: Utilizing high-frame-rate cameras (120–240 fps) to identify, categorize, and quantify food waste in real-time.
• Agentic AI: Systems that move beyond simple data collection to provide actionable insights, such as automated procurement suggestions or menu adjustments for commercial kitchens.
• Hardware-Software Integration: Combining physical dehydration technology with AI to transform food waste into a manageable, stable, and recyclable resource.

---

1. The Problem: Food Waste at Scale

Matt Rogers, founder of Mill and Nest, identifies food waste as a "multi-hundred-billion-dollar problem." Beyond the economic impact—estimated at $400 billion annually in the U.S.—it is a significant environmental issue. Rogers emphasizes that food is approximately 80% water; by dehydrating it, the waste becomes a fine, stable powder that is easy to transport and recycle, preventing the rot and methane production associated with traditional organic waste.

2. Methodology: From Dehydration to Intelligence

The core of Mill’s technology is a high-capacity dehydrator. However, the innovation lies in the transition from simple waste management to waste prevention.

• Data Collection: Mill utilizes a massive dataset—over 5 terabytes of labeled food waste imagery—to train their models. Much of this foundational data was sourced from historical projects at Google X.
• Edge Processing: To avoid the prohibitive costs and latency of streaming video to the cloud, Mill integrates NVIDIA Jetson hardware directly into their units. This allows them to run custom-tuned versions of Google’s Gemma model locally on the device.
• Benchmarking: The team uses a specialized tool to compare model performance against "golden samples" (perfectly labeled data) and "messy" real-world data (e.g., food scraps in a commercial kitchen) to ensure accuracy in identifying items like eggs, produce, and complex mixtures.

3. Real-World Applications

• Commercial Kitchens: By identifying what is being thrown away (e.g., whole eggs vs. shells), the system provides data to chefs. This allows them to adjust procurement orders or modify menus to utilize ingredients before they expire, effectively reducing waste at the source.
• Operational Efficiency: The system aims to reduce the administrative burden on chefs by automating the tracking of waste, allowing them to focus on cooking rather than paperwork.
• Consumer Insights: While the consumer-facing product avoids cameras for privacy reasons, the company uses aggregate mass data to identify trends, such as spikes in food waste during holidays like Thanksgiving or the Super Bowl.

4. Key Arguments and Perspectives

• Problem-Centric Innovation: Rogers argues that successful product design starts with identifying a "big, meaty problem" that affects humanity, rather than starting with the technology itself.
• The "Edge" Advantage: Rogers notes that for high-volume, real-time tasks like monitoring trash, cloud-based AI is unsustainable. He advocates for "on-device" AI as the only viable path for scaling this technology.
• AI for Good: Rogers frames this application of AI as "pure good"—it is economically beneficial for businesses, environmentally necessary for the planet, and lacks the "scary" connotations often associated with generative AI.

5. Notable Quotes

• "I became an engineer to solve really big problems for humanity." — Matt Rogers
• "The more we measure it, the more we can manage it." — Matt Rogers
• "If you can take away some of that paperwork and say, 'Hey, you're going to have extra eggs today, here's what you could do with those eggs,' [you can] change procurement." — Matt Rogers

6. Synthesis and Conclusion

The conversation highlights a shift in the AI landscape from general-purpose cloud models to specialized, edge-deployed agents. By combining hardware (dehydrators) with sophisticated computer vision and on-device LLMs (Gemma), Mill is transforming food waste from a "gross" liability into a data-driven opportunity. The ultimate takeaway is that the most effective AI applications are those that integrate seamlessly into existing workflows to solve massive, systemic inefficiencies while providing clear, measurable economic and environmental returns.