Stanford Robotics Seminar ENGR319 | Winter 2026 | Robot Motion Learning w/Physics-Based PDE Priors

Robot Motion Learning with Physics-Based PD Prior: A Detailed Summary

Key Concepts:

• Motion Planning: Coordinating an agent’s behavior from a start to a goal while satisfying constraints.
• Physics-Based PD Prior: Utilizing principles from physics, specifically the Pontryagin’s Minimum Principle (PMP) and its simplified form, the Eikonal PD, as a prior for robot motion learning.
• Eikonal PD: A simplified form of the Hamiltonian-Jacobi-Bellman (HJB) equation, representing travel time as a function of robot configuration and environment constraints.
• Stochasticity in Neural Networks: Incorporating randomness (e.g., dropout, variational autoencoders, diffusion models) to help neural networks escape local minima during training.
• Value Function: A function representing the optimal cost (e.g., travel time) to reach a goal from a given state.
• Temporal Difference (TD) Learning: A reinforcement learning technique used to learn value functions by bootstrapping from estimates of future values.
• Metric Learning: Enforcing the learned solution to respect properties of a geodesic distance, ensuring consistency with geometric constraints.
• Judecic Distance: The shortest path between two points on a manifold, respecting the geometry of the space.

1. Introduction to Motion Planning & Applications

The presentation focuses on the problem of motion planning – enabling an agent (typically a robot) to navigate from a starting point to a goal while adhering to specified constraints. The speaker’s lab concentrates on applications including: full-body motion planning, mobile robot navigation with moving obstacles, reactive manipulation (adapting to disturbances during manipulation tasks), and reactive multi-agent planning (collision avoidance among multiple robots). The overarching goal is to develop real-time coordination tools for robots operating in unstructured, constrained environments with minimal pre-training or trial-and-error.

2. Evolution of Approaches: From Sampling-Based to Data-Driven

The speaker’s research journey began with sampling-based motion planning methods, focusing on adaptive sampling techniques to improve efficiency. However, these methods become computationally expensive as the robot’s dimensionality (degrees of freedom) increases, hindering real-time applications. This led to exploring data-driven approaches using neural networks, starting around 2018/2019. Early attempts to use neural networks for motion planning were unsuccessful without incorporating stochasticity. The key finding was that stochasticity (implemented through dropout, variational autoencoders, and now diffusion models) is crucial for neural networks to escape local minima and find effective solutions.

3. Limitations of Data-Driven Approaches & the Need for Physics Priors

While data-driven methods offer fast inference, the speaker identified a significant drawback: the high cost of training. These methods typically require extensive data collection using classical planning techniques before training the neural network, negating the potential benefits of faster inference. This prompted a search for methods to train neural networks without relying on expert demonstrations. The core idea is to leverage physics-based priors to achieve inference efficiency, training efficiency, and adaptability to complex scenarios.

4. The Physics-Based PD Prior: Utilizing the Eikonal PD

The proposed solution centers around incorporating physics-based priors, specifically the Pontryagin’s Minimum Principle (PMP) and its simplified form, the Eikonal PD. The Eikonal PD governs the motion of dynamical systems and can be expressed as a partial differential equation (PDE). Solving this PDE yields a travel time function (T) representing the time required to reach any point in the environment from a given starting point. The speaker emphasizes that this is not a physics simulation, but rather a PD equation used as a prior. The Eikonal PD has two components: T (the unknown value function) and S (a known function representing constraints, such as distance to obstacles or manipulation manifolds).

5. Solving the Eikonal PD with Neural Networks

Numerically solving the Eikonal PD becomes challenging in high dimensions. The speaker proposes using a neural network to approximate the solution. The neural network takes the robot’s start configuration (QS), goal configuration (QG), and environment perception as input and outputs the travel time function (T). Training is achieved by minimizing a gradient matching loss: the gradient of the predicted travel time is compared to the inverse of the constraint function (S) as defined by the Eikonal PD. This approach requires only randomly sampled robot configurations and their corresponding constraint values as training data.

6. Addressing Limitations: Multimodality & Uncontrolled Gradients

Initial attempts to scale this approach beyond four dimensions encountered two key limitations:

• Multimodality: The Eikonal PD has multiple solutions, and the neural network struggled to capture them all.
• Uncontrolled Gradients: Training on randomly sampled configurations led to uncontrolled gradients between consecutive configurations, resulting in inaccurate trajectories.

7. Improvements: Viscosity & Geometric Properties

To address multimodality, the speaker explored using a viscous Eikonal PD, which has a unique solution. While this improved performance, it increased training cost due to the computational expense of calculating the viscosity term. Further research revealed that the core issue lies in respecting the geometric properties of the Eikonal PD, specifically its nature as a geodesic distance.

8. Incorporating Metric Learning & Temporal Difference Learning

The solution involves two key enhancements:

• Metric Learning: Enforcing the neural network’s predictions to adhere to the properties of a geodesic distance (symmetry, triangular inequality) through a specialized neural network architecture. This architecture uses max pooling to approximate multiple solutions and learn a latent space representing the geodesic distance.
• Temporal Difference (TD) Learning: Applying TD learning, inspired by reinforcement learning, to regulate the gradients between consecutive configurations. This is achieved by incorporating a loss term based on the Bellman principle of optimality, ensuring consistency in the value function.

9. Results & Performance

The proposed method demonstrates significant advantages:

• Inference Efficiency: Faster planning times compared to optimization-based methods (e.g., Empire, RRT) and sampling-based methods (e.g., PRM).
• Training Efficiency: Significantly reduced training time and data requirements compared to data-driven approaches like imitation learning. Training can be achieved with minimal expert data.
• Adaptability: Scalability to high-dimensional systems (up to 15 degrees of freedom) and complex environments.
• Transferability: The model can be easily transferred to new environments with minimal retraining.

The method has been successfully applied to various tasks, including maze navigation, manipulation (door opening, object transport), and multi-agent planning.

10. Future Directions

Future research directions include:

• Real-Time Training: Further reducing training time to enable on-the-fly learning.
• Reactive Planning: Developing methods that can quickly adapt to dynamic environments and disturbances.
• Multimodal Manipulation: Extending the approach to handle complex manipulation tasks involving multiple constraints.
• Assistive Manipulation: Applying the method to human-robot collaboration scenarios, such as assisted dressing.
• Dynamic Environments: Solving the Eikonal PD for dynamic systems, incorporating kinematic constraints.

This summary provides a detailed overview of the presented research, preserving the technical precision and language of the original transcript. It highlights the key concepts, methodologies, and results, offering actionable insights into the potential of physics-based priors for robot motion learning.