Learn OpenMV in 1 hour

Key Concepts

OpenMV, MicroPython, Computer Vision, Embedded Systems, Image Processing, Object Tracking, Machine Learning, OpenMV IDE, Edge Detection, QR Code Scanning, Person Detection, Color Detection, TensorFlow Lite, Haar Cascade, Pinhole Camera Model, Region of Interest (ROI), Frames Per Second (FPS), Camera Initialization, Thresholding.

OpenMV Overview

OpenMV is a compact, Python-programmable computer vision camera designed for embedded systems. It runs MicroPython, enabling image processing, object tracking, and machine learning using Python code. It's targeted towards students and developers aiming to integrate visual intelligence into their projects with minimal hardware and coding expertise.

Hardware and Software Requirements

• Hardware: OpenMV camera (specifically OpenMV Cam H7 Plus), Type A to microB USB cable, optional SD card.
• Software: OpenMV IDE (available for Windows, macOS, and Linux).

Setting up OpenMV

1. Cleaning the Sensor: Clean the camera sensor with isopropyl alcohol and a lint-free cloth.
2. Connecting the Camera: Connect the OpenMV camera to the PC using a microB USB cable. A blue LED should blink, and a USB drive named "OpenMV" should appear.
3. File System: The OpenMV camera uses the FAT file system, which is prone to corruption. Avoid writing to the internal drive from Python scripts.
4. OpenMV Disk File: A file named "openmv.disk" should always appear inside the OpenMV drive. If it doesn't, reboot the camera.

OpenMV IDE

The OpenMV IDE is the software used for coding, testing, and interacting with the OpenMV camera.

Installation

The installation process is straightforward and well-supported on Windows, macOS, and Linux. Download the appropriate version from the OpenMV website and follow the installation instructions.

Interface Walkthrough

• Connection Icon: Indicates whether the OpenMV camera is plugged in and connected.
• Menu Tab:
  • File: Create new files, open existing files, access examples.
  • Examples: Provides built-in examples for various functionalities.
  • Camera: Take snapshots, record videos, access optical flow data.
  • Image Processing: Drawing functions (arrows, circles, text).
  • Machine Learning: TensorFlow Lite and Haar Cascade examples.
  • Barcode: QR code and barcode detection examples.
  • Feature Detection: Edge detection, point circles, template matching, April tags.
  • Tools:
    • Load Custom Firmware: Manually upload a firmware file to the camera.
    • Erase Internal FAT File System: Clears the camera's internal storage.
    • ROM File System: Create, manage, or edit the read-only space on the camera.
    • Auto Reconnect to OpenMV Camera: Automatically connects to the camera when the IDE is opened.
    • Stop Script on Connect/Disconnect: Stops script execution when the camera is disconnected.
    • Open Drive Folder: Opens the folder where files saved on the OpenMV camera appear.
    • Reset OpenMV Camera: Restarts the camera hardware.
    • Machine Vision:
      • Open Model Zoo: Built-in collection of ready-to-use TensorFlow Lite models.
      • Convert Model for NPU: Converts models to run on hardware with neural processing units.
      • Threshold Editor: Fine-tuning color detection.
      • Keypoint Editor: Training template matching or keypoint models.
      • April Tag Generator, QR Code Generator, Barcode Generator.
      • Data Set Editor: Create new data set, open existing data set or export any data set from edge impulse.

General Purpose Code Structure

Most OpenMV scripts follow a similar structure:

1. Imports:
   • import sensor: Controls the OpenMV camera (resolution, color format, image capture).
   • import time: Measures frame duration and calculates frames per second (FPS).
2. Camera Initialization:
   • sensor.reset(): Resets the camera.
   • sensor.set_pixformat(sensor.RGB565): Sets the color format (RGB565 for color, sensor.GRAYSCALE for grayscale).
   • sensor.set_framesize(sensor.QVGA): Sets the frame size (QVGA = 320x240 pixels).
   • sensor.skip_frames(time = 2000): Waits for 2 seconds for exposure and white balance adjustment.
3. Clock Initialization:
   • clock = time.clock(): Creates a clock object to track frame duration.
4. Main Loop:
   • clock.tick(): Starts the timer for the current frame.
   • img = sensor.snapshot(): Takes a picture and stores it in the img variable.
   • print(clock.fps()): Prints the frames per second (FPS) in the console.

Built-in Examples

Edge Detection (Canny)

• Purpose: Detects edges in the camera feed, outlining objects.
• Key Steps:
  1. Import image module.
  2. Set gainceiling to 8 for better visibility in grayscale mode.
  3. Use img.find_edges(image.EDGE_CANNY, threshold=(50, 80)) to apply the Canny edge detection algorithm. The threshold values (50, 80) determine the low and high thresholds for edge detection.

QR Code Detection

• Purpose: Detects QR codes in the camera feed, draws a rectangle around them, and prints the data.
• Key Steps:
  1. sensor.set_auto_gain(False): Disables auto gain control for accurate QR code detection.
  2. img.find_qrcodes(): Returns a list of QR code objects found in the frame.
  3. Iterate through the list of QR codes and draw a red rectangle around each detected QR code using img.draw_rectangle(code.rect(), color=(255, 0, 0)).
  4. Print the QR code data to the terminal.
  5. img.lens_correction(1.8): Corrects lens distortion, especially near the edges of the frame.

Person Detection (TensorFlow Lite)

• Purpose: Uses a pre-trained TensorFlow Lite model to detect whether a person is present in the frame.
• Key Steps:
  1. net = tf.load('person_detect.tflite'): Loads the TensorFlow Lite model.
  2. net.classify(img): Runs the image through the model and returns confidence scores.
  3. Combine labels ("person", "no person") and confidence values into a list of tuples.
  4. Sort the list by confidence values in descending order.
  5. Print the label with the highest confidence score.

Custom Projects

Color Detection

• Purpose: Detects objects of a specific color (e.g., red) in the camera feed.
• Key Steps:
  1. Define a color threshold (e.g., red_threshold = (0, 100, 15, 127, 15, 127)). The six values control lightness, the A channel (green to red), and the B channel (blue to yellow).
  2. Use img.find_blobs([red_threshold], pixels_threshold=200) to find blobs that match the color threshold.
  3. Iterate through the list of blobs and draw a red rectangle around each detected blob using img.draw_rectangle(b.rect(), color=(255, 0, 0)).
  4. Draw a white cross at the center of the blob using img.draw_cross(b.cx(), b.cy(), color=(255, 255, 255)).
  5. Print the X and Y position of the detected colored object.

Face Tracking

• Purpose: Detects and tracks a face in the camera feed.
• Key Steps:
  1. Load a Haar Cascade model for face detection: face_cascade = image.HaarCascade("frontalface", image.HAAR, 1.25).
  2. Use img.find_features(face_cascade, threshold=0.5, scale_factor=1.25) to detect faces in the image.
  3. Draw a rectangle around each detected face.
  4. Define a Region of Interest (ROI) around the detected face to improve tracking accuracy and performance.
  5. In subsequent frames, search for faces only within the ROI.
  6. If no face is detected in the ROI, perform full-frame detection.

Face Tracking with Distance Measurement

• Purpose: Extends the face tracking project to estimate the distance of the detected face from the camera.
• Key Concepts:
  • Pinhole Camera Model: Uses the formula D = (W * F) / w to calculate the distance (D), where W is the average width of an adult human face (14 cm), F is the focal length of the camera (160 pixels), and w is the width of the detected face in the image.
• Key Steps:
  1. Define the variables: W = 14.0, F = 160.0, w = 0, D = 0.
  2. Extract the width (w) of the detected face from the drag_face tuple.
  3. Calculate the distance (D) using the pinhole camera model formula.
  4. Draw a string on the image to display the calculated distance using img.draw_string(20, 30, "Distance: %d cm" % D, color=(0, 255, 0), scale=2).
  5. If no face is detected, display "No face detected" and the last calculated distance.

Conclusion

The OpenMV Cam is a powerful tool for bringing computer vision to embedded systems. With Python and a few lines of code, you can build projects that would take far more complex setups. The course provides a beginner-friendly introduction to OpenMV, covering the basics of connecting the camera, using the IDE, writing general-purpose code, and exploring built-in examples. It also guides you through creating custom projects, including face tracking and distance measurement.