Stanford Webinar - From Pixels to Human Impact: Unlocking Real-World Value with Computer Vision

Key Concepts

• Computer Vision: Bridging the gap between meaningless pixels and meaningful concepts, ultimately leading to human impact.
• Deep Learning: A subset of machine learning that powers computer vision forward.
• Ambient Intelligence: Using smart sensors, computer vision, and machine learning to understand human activities and react to perceived needs in healthcare spaces.
• Activities of Daily Living (ADLs): Routine activities people do every day without assistance.
• Clinical Behavioral Atlas (CBA): A high-throughput comprehension of clinical care delivery activities in ICUs.
• Multi-Object Multi-Actor (MOMA) Action Recognition: A new technical problem and benchmark for understanding complex scenes with multiple actors and objects.
• Neuropsychiatric Symptoms (NPS): Behavioral symptoms that are early indicators of dementia and Alzheimer's disease.
• Edge Computing: Running AI algorithms locally on devices (sensors) to minimize data transfer and enhance privacy.
• Linguistic Intelligence: AI focused on understanding and reasoning in the language space.
• Spatial Intelligence: AI focused on understanding physical spaces and interactions.
• Embodied AI: Robots that can see and act.

Introduction

Professor Ehsan Adeli discusses how computer vision has evolved beyond simple image recognition to reshape healthcare, science, and society. He emphasizes directing computational power towards health, safety, and creativity. The presentation covers the history of computer vision, its applications in healthcare, and considerations for privacy and trustworthiness.

The Evolution of Computer Vision

• Early Roots: Computer vision's roots trace back to neuroscience, with experiments by Hubel and Wiesel demonstrating edge detection in the brain.
• Early Models: Larry Roberts used geometry to model 3D vision in the 1960s, and David Marr described vision as a series of computational stages in the 1970s.
• AI Winter: Progress slowed down in the 1980s, but edge detection and Gestalt principles continued to guide machine organization of scenes.
• Feature Extraction: In the 2000s, matching techniques like SIFT and datasets like Caltech 101 drove recognition, with face detection becoming a major success.
• Deep Learning Emergence: Deep learning advanced from simple perceptrons to more advanced neural networks with back-propagation.
• ImageNet Breakthrough: The ImageNet dataset and AlexNet's success in the 2012 ImageNet challenge marked a turning point, leading to a deep learning explosion. AlexNet cut error rates almost in half overnight.
• Modern Applications: Deep nets now power classification, retrieval, generation, object detection, segmentation, and video understanding.
• Generative Models: AI can now generate images, with text-to-image models creating images from words.
• Key Enablers: Data, algorithms, and compute power are the engines driving progress in computer vision.
• Tasks and Models: Tasks (e.g., object recognition, scene understanding) and models (computational frameworks mimicking the visual system) feed each other.
• Architectural Advancements: Convolutional Neural Networks (CNNs) became the workhorse for vision, followed by recurrent neural networks for sequential data, and now transformers are redefining architectures.
• Generative Model Types: Diffusion models generate images from noise.
• Vision Language Models: Merge pixels with words.
• 3D Vision: Brings depth, geometry, and structure into play.

Computer Vision in Healthcare: Ambient Intelligence

• Goal: To transform care delivery using AI technologies to reduce workloads, lower costs, and improve outcomes.
• Ambient Intelligence Vision: Endowing healthcare spaces with smart sensors, computer vision, and machine learning algorithms to understand human activities and react to perceived needs.
• Applications:
  • Daily Living Spaces: Quantify activities of daily living (ADLs), monitor health, and intervene at the right time.
  • Hospital Spaces: Monitor clinical activities to improve the quality of care and patient health.
• Implementation Steps:
  1. Transform healthcare spaces with sensing capabilities (contactless audio-visual data, RGB, depth, thermal images, wearable technologies).
  2. Recognize activities (ADLs in home environments, clinical care bundles and checklists in hospitals).
  3. Integrate into the full clinical data ecosystem.
  4. Provide timely and critical interventions when necessary.

Technologies for Hospital Care

• Implementation at Stanford Hospital: Sensors with depth, RGB, and thermal sensing technologies, along with edge computers with GPU cores, were installed in eight ICU rooms and hallways.
• Data Infrastructure: Data is streamed to the Stanford School of Medicine's research side through STARR (Stanford Research Repository) and used in the Stanford neuro platform (HIPAA compliant).
• Clinical Behavioral Atlas (CBA): A high-throughput comprehension of clinical care delivery activities in ICUs.
• ICU Clinical Behavioral Atlas: A taxonomy of behaviors and actions that can happen in the ICU, defined using a hierarchical structure.
  • Hierarchical Structure: Bundles -> Clinical Procedures -> Actions -> Atomic Actions.
  • Example: ABCD bundle with sub-procedures and sub-actions.
  • Elements Defined: Objects, actors, locations, relationships between objects and actors.
• MOMA (Multi-Object Multi-Actor) Action Recognition: A new technical problem and benchmark for understanding complex scenes.
  • Compositional Approach: Hierarchically decompose and understand human activities.
  • Data Set: MOMA data set with multi-object multi-actor relationships.
  • Hyper-edges: Define relationships between multiple actors and objects.
• Algorithm Deployment: Machine learning algorithms are deployed on cloud servers and edge devices to recognize clinically relevant visual concepts.
• Quality Improvement: Identification of clinical activities is used for quality improvement, workflow optimization, and documentation.
• Descriptive Analytics: Automatic and passive analysis of patient status, functional status, bedside activities, and device usages.
• Foundation Models: Newer foundation models are used to describe what's happening in the scene and gain clinical insights (e.g., patient is alert and calm, RASS score of 0).
• Intermountain Healthcare: Depth sensors were used to understand activities in patient rooms and ICU rooms (e.g., getting out of bed, sitting).

Home-Based Senior Care

• Urgent Need: To develop technologies to support independent living of seniors at home.
• Limitations of Wearable Technologies: Offer a small set of functionalities with limited granularity and can be annoying to use.
• Computer Vision Solution: Recognize a broad range of activities of daily living (mobility, infection, sleep, diet).
• Benefits: Continuous monitoring, proactive health risk detections, and timely interventions to delay assisted living needs.
• Clinical Insight: Gain clinical insight into the individual's health.
• Examples:
  • Detect severe fever and monitor respiratory rates using depth and thermal sensors.
  • Understand the mobility status of seniors and offer detailed gait analysis.
  • Track seniors' sleeping patterns and detect deviations from normality.
  • Provide information about dietary activities and assess levels of independence.
• Neuropsychiatric Symptoms (NPS) Detection: Use computer vision to detect NPS, which are early indicators of dementia and Alzheimer's disease.
  • Study Design: Clinical measurements (telephone interview of cognitive status), questionnaires (neuropsychiatric inventory, mild behavioral impairment checklist, apathy evaluation scale), and weekly self-assessment manikin.
  • Behavioral Changes: Continuous assessment of behavioral changes and summarization in clinical interfaces.
  • Depression Detection: Identify individuals with high depression based on time spent in bed or agitation during the day.
• Vital Signs Monitor of Behavior: Computer vision provides a new window into daily lives and serves as a vital signs monitor of behavior and mental health.

Privacy and Trustworthiness

• Concerns: Computer vision deals with vast amounts of personal data, raising concerns about privacy and security.
• Privacy Protection Methods: Facial blurring, dimensionality reduction, encryption, body masking, federated learning, and differential privacy.
• Edge Computing Solution: Run machine learning algorithms locally on devices (sensors) to minimize data transfer and enhance privacy.
• Lens-Based Privacy Technology: Create distortions on the lens itself so that the generated image is not recognizable by humans but can still be processed by the algorithm.
  • Point Spread Function (PSF): Used to define the lens surface profile and emulate waveform propagation.
  • Distorted Images: The camera registers distorted images that are not easily interpretable by humans.
• Ethical Considerations: Ethical aspects and issues of AI algorithms running at home or in healthcare environments.
  • Six Stages and Mitigation Plans: Identified for potential ethical issues.

Conclusion

Professor Adeli concludes by emphasizing that technology should enhance human care, not replace it. Computer vision has the potential to greatly improve healthcare and quality of life. He highlights the importance of spatial intelligence and understanding human intentions as the next big thing in AI. He also addresses the importance of privacy and ethical considerations when implementing these technologies.