Stanford Seminar - Hardware / controls co-design to overcome challenges for aerial robots

Key Concepts:

• Aerial Robotics: Design and control of flying robots.
• Autonomous Systems: Systems that can operate without human intervention.
• Cluttered Environments: Complex spaces with many obstacles.
• Collision Avoidance: Preventing robots from colliding with their surroundings.
• Motion Planning: Generating paths for robots to follow.
• State Estimation: Determining the robot's position and orientation.
• Visual Inertial Odometry (VIO): Estimating motion using cameras and inertial sensors.
• Shape Shifting/Morphing: Changing the robot's physical form.
• Tensegrity: Structures that use tension and compression for stability.
• Actuator Disc Model: Simplified model of propeller performance.
• Extremum Seeking Control: Optimizing system performance without a precise model.
• Closed-Loop Co-design: Simultaneously designing mechanical and control aspects.
• H2 Control: Control design method minimizing the effect of disturbances.
• Learning-Based Control: Using machine learning to train controllers.
• Large Language Models (LLMs): AI models for generating text and code.
• Genetic Algorithms: Optimization algorithms inspired by natural selection.

1. Moving in Complex Cluttered Environments

• Challenge: Navigating aerial robots in environments with many obstacles.
• Three Approaches:
  • Being Smarter (Reactive Planning):
    • Traditional Approach: Modular stack with state estimation, mapping, and global planning. Fragile due to narrow connections between modules.
    • Reactive Planning: Integrate fast, reactive planning into the lowest-level control loop.
    • Methodology:
      • Use onboard depth cameras (Intel RealSense) to perceive free space.
      • Decompose depth images into pyramids for fast collision checking.
      • Generate motion primitives within the free space.
      • Implement on a 1.4 kg drone with an RB5 computer, ROS 1, PX4, and OpenVINS.
    • Example: Drone flying through a eucalyptus forest using onboard planning.
    • Limitation: Perception becomes the limiting factor before dynamics.
  • Being Smarter (Perception-Aware Planning):
    • Incorporate perception information (visual features, motion blur) into the planning loop.
    • Methodology:
      • Predict feature motion blur based on camera motion and shutter speed.
      • Minimize uncertainty along the path by selecting trajectories with more visual features and less motion blur.
      • Use a receding horizon planner running at 10 Hz.
      • Combine speed and perception objectives using a convex combination.
    • Example: Drone veering towards areas with more visual contrast (e.g., the side of a road with trees) instead of flying straight.
    • Data: EKF variance is reduced, and the number of tracked features is higher with the perception-aware planner.
  • Being Smaller (Shape Shifting):
    • Inspired by birds that change their shape to fly through narrow spaces.
    • Design: Quadcopter with four arms, each with a thruster that can fold 90 degrees up or down.
    • Mechanism: Relies on the existing four motors for actuation.
    • Functionality: Can fly like a regular quadcopter or fold the arms down to fit through narrow spaces.
    • Example: Drone flying through a narrow tunnel in the shape-shifted configuration.
    • Additional Capabilities: Limited manipulation (e.g., picking up an empty box), perching on wires.
  • Cheating (Embracing Collisions):
    • Inspired by the DARPA Subterranean Challenge.
    • Design: Quadcopter enclosed in a six-bar tensegrity structure.
    • Rationale: Tensegrity distributes external loads as tension or compression, making the structure robust to collisions.
    • Functionality: Can crash through obstacles without external sensors.
    • Example: Robot hopping forward through a forest, colliding with trees.
    • Key Technique: Zero velocity updates during hops to improve dead reckoning accuracy.

2. Improving Energy Efficiency

• Challenge: Reducing energy consumption in aerial robots.
• Three Approaches:
  • Being Smarter (Adaptive Trajectories):
    • Data-Driven Approach: Optimize trajectories based on real-time energy consumption data.
    • Methodology:
      • Use extremum seeking control to adapt speed and sideslip angle (rotation around the thrust direction).
      • Overlay sine waves on the input signals and use convolution to estimate the gradient of the cost function.
    • Example: Drone flying in a circle, adapting its speed and sideslip angle to minimize energy consumption.
    • Benefit: Adapts to different payloads without needing a precise model.
  • Morphing (Tilting Propellers):
    • Design: Quadcopter with propellers that can tilt forward without tilting the body.
    • Mechanism: Sprung mechanism where all four propellers tilt together based on thrust.
    • Functionality: Can switch between an untilted configuration (for maneuverability) and a tilted configuration (for high speed).
    • Example: Drone switching between tilted and untilted configurations in flight.
    • Data: Achieves 12% faster flight and 10-15% decrease in energy consumption at the same speed.
  • Cheating (Flying Batteries):
    • Concept: Use a smaller drone to deliver additional battery power to a main drone in flight.
    • Rationale: Avoids the trade-off between battery size and flight time.
    • Example: Main drone flying for 80 minutes with battery swaps from a smaller drone.
    • Potential Application: Air taxis with extended range without carrying large batteries.

3. Automating the Design Process

• Challenge: Automating the design of aerial robotic systems.
• Goal: Create tools that provide designers with rough, feasible designs for further refinement.
• Three Ideas:
  • Closed-Loop Co-design:
    • Application: Material handling with multiple lifting units (drones).
    • Problem: Determine the optimal attachment points for the lifting units on a payload.
    • Objective: Minimize the probability of actuator saturation.
    • Methodology:
      • Model the system and linearize it.
      • Assume Gaussian disturbances.
      • Use an H2 approach to calculate the statistics of the inputs.
      • Calculate the Mahalanobis distance to approximate the probability mass within the feasible range.
    • Example: Attaching three agents to an "M"-shaped payload. The tool finds a locally optimal, non-intuitive configuration that is more robust to disturbances.
  • Learning from Simulation:
    • Goal: Design a controller that can fly any quadcopter without knowing its parameters.
    • Methodology:
      • Train a low-level controller in simulation using a neural network.
      • Use an extrinsics encoder to estimate parameters from sensor data.
      • Deploy the controller in experiment.
    • Architecture: RMA (Recurrent Motor Adaptation) architecture.
    • Example: Controller adapting to a large payload being added mid-flight.
    • Example: Controller recovering from a 20% reduction in propeller effectiveness.
  • Synthesizing Interpretable Policies (LLMs):
    • Concept: Use large language models to generate Python code for controllers.
    • Methodology:
      • Use a genetic algorithm where the genome is the Python code.
      • Use an LLM for crossover (generating children from two parent programs).
    • Example: Controlling an inverted pendulum. The LLM generates legible code that can be modified by an engineer.
    • Example: Controlling a ball and cup. The LLM generates code with an 80% success rate. The engineer can improve the success rate to 99% by adding a few lines of code.

4. Key Arguments and Perspectives

• Control Theory as a Lens: Using control theory to guide the design of aerial robotic systems.
• Simultaneous Mechanical and Control Design: Designing the mechanical and control aspects of a robot in a single loop.
• Cheating with Physics: Exploiting the dynamics of a system to simplify a problem.
• Importance of Mechanical Design: Recognizing that mechanical design is as important as control algorithms.

5. Notable Quotes

• "System design is control design using control as a lens to think about these problems."
• "How can I cheat with physics? How can I take the dynamics of my system to make something that is hard less hard by sort of being creative in in how you interpret what your degrees of freedom are?"

6. Synthesis/Conclusion

The presentation explores various approaches to designing and controlling aerial robotic systems, emphasizing the importance of considering both mechanical design and control theory. It highlights the potential of reactive planning, shape shifting, and embracing collisions for navigating complex environments. It also discusses methods for improving energy efficiency through adaptive trajectories, morphing, and innovative solutions like flying batteries. Finally, it explores the use of closed-loop co-design, learning from simulation, and large language models to automate the design process, ultimately aiming to provide designers with tools that can generate feasible and adaptable robotic systems. The key takeaway is that by integrating mechanical design and control theory, and by creatively exploiting the physics of the system, engineers can develop more robust, efficient, and adaptable aerial robots.