Stanford Robotics Seminar ENGR319 | Spring 2026 | Unlocking Autonomous Medical Robotics

Key Concepts

• Autonomous Surgical Robotics: The integration of AI and robotics to perform surgical tasks with minimal human intervention.
• Teleoperation: A system where a human operator controls a robot remotely (e.g., the da Vinci Surgical System).
• Digital Twinning: Creating a virtual, physics-based model of a physical environment (tissue, fluids) to simulate and predict outcomes.
• Position-Based Dynamics (PBD): A simulation method using particles and constraints to model deformable objects faster than real-time.
• Differentiable Rendering: A technique to align simulation models with real-world camera observations by back-propagating errors.
• Embodied Intelligence: The integration of perception, modeling, planning, and control within a physical robot.
• Knowledge-Grounded Reinforcement Learning: A framework that sequences modular, learned behaviors to complete complex, multi-step tasks.
• Haptics: The technology of providing tactile and force feedback to a user, crucial for surgical precision.

---

1. The Need for Surgical Autonomy

The speaker highlights a critical shortage of skilled healthcare labor (surgeons and nurses). While current surgical robots like the da Vinci system improve precision and reduce surgeon fatigue, they are purely teleoperated and require significant human personnel. The goal is to transition from teleoperation to autonomous surgical robots to scale expertise, ensure uniformity of care, and address labor shortages.

2. Foundational Pillars of Autonomous Robotics

The lab at UC San Diego focuses on four pillars to achieve embodied intelligence:

• Perception: Identifying, localizing, and tracking targets in challenging environments (e.g., narrow fields of view, specular reflections from blood, smoke, and deformable anatomy).
• Modeling and Simulation: Using Digital Twinning to understand the physics of tissue.
• Planning: Making decisions based on the evolution of the environment.
• Control: Executing actions with sub-millimeter precision.

3. Methodologies and Frameworks

• Physics-Based Simulation: The lab utilizes Position-Based Dynamics (PBD) because it runs faster than real-time and satisfies constraints exactly.
• Iterative Correction: To bridge the "reality gap," they use differentiable rendering. By comparing streaming video to the simulation, they back-propagate the loss to adjust the simulator’s mechanical properties (e.g., tissue stiffness, viscosity) in real-time, reducing prediction error from 5mm to under 2mm.
• Safety-Aware Control: The team employs Bayesian inferencing to map tissue connections. By tugging on tissue and observing discontinuities, the robot identifies connected regions and calculates an "energy threshold" to avoid accidental tearing.

4. Scaling and Lifelong Learning

To move beyond hand-engineering every task, the lab uses Knowledge-Grounded Reinforcement Learning. This involves:

• Knowledge Modules: Pre-trained behaviors (e.g., grasping, cutting, scanning).
• Neural Sequencing: A sparse neural network architecture that learns to combine these modules to perform complex, multi-step procedures like the Fundamentals of Laparoscopic Surgery (FLS) tasks.

5. Humanoid Platforms in Healthcare

The speaker argues that humanoid robots (e.g., the "Surgi" platform) offer a better value proposition than specialized, multi-million dollar surgical systems.

• Versatility: Humanoids can perform diverse tasks, from intubation to acting as surgical assistants (retracting tissue, holding cameras, or handing off instruments).
• Hardware Challenges: Robot hands often lack the degrees of freedom and tactile sensing of human hands. The lab developed a custom haptic glove with multi-directional force feedback to allow operators to "feel" buttons, knobs, and tissue resistance, significantly improving teleoperation performance.

6. Real-World Applications

• Hemorrhage Control: Using fluid dynamics modeling to suction blood efficiently.
• Chronic Wound Care: Automating the removal of bandages and the application of tape. This addresses a massive burden on caregivers and patients, using vision and haptic models to minimize skin stretching and pain.

7. Notable Quotes

• "Robot autonomy can only be as good as its perception and its ability to recognize and understand the world."
• "I don’t think any amount of software can make a crummy robot achieve what it needs to achieve." (Emphasizing the necessity of high-quality, back-drivable hardware).

8. Synthesis and Conclusion

The path to autonomous surgery lies in a hybrid approach: combining the robustness of mathematical physics models with the flexibility of foundation models. While foundation models are excellent for perception, they currently lack the safety guarantees required for surgery. The speaker concludes that by building "context-aware" systems that understand the physics of the human body, researchers can create reliable, scalable, and explainable robotic assistants that significantly improve patient outcomes and alleviate the global healthcare labor crisis.