New AI Model Lets Robot Neo Learn Tasks From Scratch

Key Concepts

• World Models: The core of the update, enabling robots to reason about and interact with the world in a more human-like and adaptable way.
• Embodiment: The physical design of the robot (Neo) being closely aligned with human form and capabilities is crucial for transferring knowledge from human-generated data (like videos).
• Scaling Laws: The observation that robotic intelligence is now scaling with the number of deployed robots rather than the amount of human-collected data.
• Passive Intrinsic Safety: Designing robots with inherent safety features like soft compliance, low energy output, and lightweight materials.
• AI Alignment: Ensuring robots choose the safest and least risky path when performing tasks, leveraging their understanding of the physical world.

The Updated Neo Robot: Adaptability and Scaling Through World Models

This discussion centers on a significant update to the Onyx Neo robot, focusing on its newfound ability to perform tasks it couldn’t previously handle without extensive, task-specific training data. The key advancement lies in the implementation of “world models,” allowing Neo to approach unfamiliar tasks with “sensible” reasoning.

The Post-it Note Example & The Power of Generalization

A prime example illustrating this capability is Neo’s ability to pick up a Post-it note from a board and read it. The speaker emphasizes that Neo had never been trained on this specific task – there was no training data involving robots manipulating and reading Post-it notes. Despite this, Neo can perform the task effectively, demonstrating a crucial aspect of learning: the ability to generalize to novel situations. As stated, “it’s not yet perfect, there are examples where it fails, but what it does is the ability to have this very sensible approach to anything which is a cornerstone of learning.”

Neo’s Architecture & Collaboration with Nvidia

Neo’s “brain” utilizes an Nvidia inference chip. While acknowledging a strong collaboration with Nvidia and the use of their technology, the speaker clarifies that Onyx chose not to simply adopt Nvidia’s models directly. This decision stems from a focus on “embodiment” – the physical design of the robot. Ben Bernanke explains that if a robot isn’t physically similar enough to a human, knowledge gained from human-centric data sources (like YouTube videos) won’t translate effectively to physical actions like picking up a cup or opening a door.

Embodiment and the Transfer of Knowledge

The concept of embodiment is central to Onyx’s approach. Neo has been deliberately designed over the last decade to closely mimic human form and capabilities. This design choice allows the robot to leverage the vast amount of knowledge humans have accumulated and documented in visual formats. The speaker argues that this is “quite unique to Onyx,” differentiating them from competitors.

The Importance of Safe Experimentation

Beyond physical similarity, the ability for Neo to safely experiment is critical. The robot must be able to attempt tasks and learn from failures without causing damage to its environment. As the speaker notes, “you don’t want your door to be scratched because there was a robot trying to open it.” This emphasis on safety is a key differentiator.

Safety Considerations & Guardrails

The discussion addresses the inherent safety concerns associated with robots learning autonomously. Onyx is actively working on safety measures, including adherence to industry standards, independent third-party audits of their safety protocols, and a layered approach to safety.

This layered approach includes:

• Passive Intrinsic Safety: Designing robots to be inherently safe through features like soft compliance, low energy output, and lightweight construction. The speaker highlights that humans are naturally “safe in the sense that we have to actively work to hurt each other,” and this principle is being incorporated into Neo’s design.
• AI Alignment: Ensuring the robot actively reasons about safety when performing tasks, identifying potential risks and choosing the safest possible path. World models contribute to this by enabling the robot to understand the physical world and anticipate potential problems.

Scaling Laws and the Future of Robotic Intelligence

A significant shift in robotic intelligence scaling is discussed. Traditionally, improving robot intelligence required more human-collected data. However, with the new world models and Neo’s embodiment, intelligence now scales with the number of deployed robots.

As Ben Bernanke explains, “once you can do that and your robot can now approach almost any task as long as you can ask for it… your intelligence doesn’t scale with the amount of data you can collect with humans anymore. It actually scales with the number of robots you’ve deployed.”

This represents a move towards a self-improving system where robots learn from their own experiences in the real world, accelerating progress towards “general intelligence.” The need for extensive human data gathering is diminishing, leading to faster deployment and broader application of Neo. This shift aligns with the “scaling laws” observed in large video models like those developed by competitors (specifically mentioned as “saw our video”).

Addressing Data Gathering Challenges

The discussion acknowledges the challenges of gathering real-world data for robotic training. Current efforts are focused on leveraging simulation and synthetic data to augment real-world learning.

Concluding Remarks

The update to Neo, driven by the implementation of world models and a focus on embodiment, represents a significant step forward in robotics. The ability to generalize to novel tasks, coupled with a commitment to safety and a new scaling paradigm, positions Onyx for accelerated deployment and continued innovation in the field of artificial intelligence. Ben Bernanke, CEO and CTO of Onyx, concludes the discussion, highlighting the transformative potential of this technology.