Stanford CS231N Deep Learning for Computer Vision | Spring 2025 | Lecture 1: Introduction

CS231n Lecture 1 Summary

Key Concepts:

• Computer Vision (CV)
• Artificial Intelligence (AI)
• Machine Learning (ML)
• Deep Learning (DL)
• Neural Networks (NNs)
• Cambrian Explosion
• Receptive Fields
• Backpropagation
• ImageNet
• Convolutional Neural Networks (CNNs)
• Object Recognition
• Image Classification
• Semantic Segmentation
• Object Detection
• Instance Segmentation
• Video Classification
• Multimodal Video Understanding
• Self-Supervised Learning
• Generative Models
• Diffusion Models
• Vision Language Models
• 3D Vision
• Embodied Agents
• Human-Centered AI

1. Introduction and Scope of the Course

• Professor Fei-Fei Li introduces CS231n as a course focused on the intersection of computer vision and deep learning.
• AI is a broad field, with computer vision as a cornerstone. Vision is considered a key aspect of intelligence.
• Machine learning, particularly deep learning, is a crucial mathematical tool for solving AI problems.
• Deep learning utilizes neural networks, a family of algorithms.
• The course will cover the core intersection of computer vision and deep learning, acknowledging the interdisciplinary nature of AI with fields like NLP, robotics, mathematics, neuroscience, and various application areas.

2. A Brief History of Computer Vision and Deep Learning

• The Cambrian Explosion: Vision's history dates back 540 million years to the Cambrian explosion, a period of rapid animal speciation. The development of photosensitive cells (eyes) in trilobites marked a shift from passive metabolism to active interaction with the environment, driving the evolution of intelligence.
• Human Innovation: Humans have long been interested in building machines that see, as evidenced by Leonardo da Vinci's studies of camera obscura and earlier thinkers in ancient Greece and China.
• 1950s: Neuroscience and Visual Pathways: Hubel and Wiesel's experiments on cat visual cortex revealed two key principles:
  • Neurons have individual receptive fields, responding to specific patterns (oriented edges) in a confined space.
  • The visual pathway is hierarchical, with neurons feeding into each other, creating increasingly complex receptive fields in higher layers (e.g., corner or object receptors).
  • "Neurons that are responsible for seeing in the primary visual cortex have their own individual receptive fields."
• 1963: Early Computer Vision: Larry Roberts' PhD thesis on shape recognition marked the beginning of computer vision as a field.
• 1966: MIT Summer Vision Project: An ambitious project aimed to solve computer vision in one summer, highlighting early overoptimism in AI.
• 1970s: David Marr's Vision Framework: Marr proposed a systematic approach to vision, inspired by neuroscience and cognitive science, involving:
  • Primal sketch (edges)
  • 2.5D sketch (depth separation)
  • Full 3D representation (the ultimate goal)
  • Recovering 3D information from 2D images is an ill-posed problem, solved by nature through multiple eyes and triangulation.
• Philosophical Difference Between Vision and Language: Language is generated and 1D, while vision is based on a physical world respecting physics and materials.
• 1970s-1980s: Early Object Recognition: Pioneering work on object recognition, such as generalized cylinders by Rodney Brooks and Tom Binford at Stanford.
• AI Winter: A period of reduced enthusiasm and funding for AI research due to unmet expectations.
• Cognitive and Neuroscience Influence: Cognitive and neuroscience research highlighted the importance of studying object recognition in natural settings.
  • Irv Biederman's study showed that the context of an image impacts object detection.
  • Experiments demonstrated the speed of human visual processing (e.g., detecting humans in 100ms frames).
  • Simon Thorpe's EEG studies showed brain categorization signals after 150ms of seeing a photo.
  • Specialized brain areas for face, place, and body part recognition were discovered.
• Object Recognition in Natural Settings: The field shifted towards studying object recognition in natural settings, separating foreground from background, and using features like SIFT.
• Early 21st Century: The Rise of Data: The internet and digital cameras led to the proliferation of data, enabling the use of datasets like Pascal VOC and Caltech 101.

3. The Deep Learning Revolution

• Early Neural Network Research: Early studies of neural networks, including perceptrons and work by Rumelhart and Hinton.
• Marvin Minsky's Critique: Minsky's argument that perceptrons cannot learn XOR logic functions caused a setback in neural network research.
• Neocognitron: Fukushima's neocognitron, a hand-designed neural network inspired by the visual pathway, demonstrated digit and letter recognition.
• Backpropagation: The introduction of backpropagation by Rumelhart and Hinton in 1986 was a watershed moment, enabling error correction and parameter optimization in neural networks.
• LeNet: Yann LeCun's convolutional neural network (LeNet) was an early application of backpropagation, used for digit and letter recognition in postal offices and banks.
• The Data Bottleneck: Despite improvements, neural networks struggled with complex natural images due to a lack of data.
• ImageNet: Fei-Fei Li and her students recognized the importance of data and created ImageNet, a large dataset with 15 million images across 22,000 categories.
• ImageNet Challenge: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) was launched, using a subset of ImageNet with 1 million images and 1,000 object classes.
• AlexNet: In 2012, Jeff Hinton and his students achieved a breakthrough with AlexNet, a convolutional neural network that significantly reduced the error rate in the ImageNet challenge.
• The Birth of Modern AI: The year 2012 and the AlexNet algorithm are considered the historical moment of the birth or rebirth of modern AI and the deep learning revolution.
• Key Factors for Success: Backpropagation and the availability of large datasets were crucial for the success of deep learning.
• "The recognition of data and the understanding of data driving these high capacity models...was critical for setting off the deep learning for this to work."

4. The Era of Deep Learning Explosion

• Explosion of Research: The number of papers in computer vision conferences (e.g., CVPR) and on arXiv has exploded.
• Progress in Computer Vision Tasks: Significant progress has been made in various computer vision tasks, including:
  • Object recognition
  • Image retrieval
  • Multiple object detection
  • Image segmentation
  • Video classification
  • Human activity recognition
  • Medical imaging
  • Scientific discovery
  • Sustainability and environment applications
  • Image captioning
  • Relationship understanding
  • Style transfer
  • Face generation
• The Convergence of Forces: Computation, algorithms, and data have driven the field to a new level.
• Hardware Progress: The progress of hardware, particularly NVIDIA GPUs, has played a significant role in the deep learning revolution.
• AI Global Warming: The field is experiencing an "AI global warming period" with accelerated growth in compute and AI applications.

5. Challenges and Ethical Considerations

• Unsolved Problems: Computer vision is still not totally solved, and there is much more to be done.
• Human Bias: AI algorithms are driven by data, which can carry human biases, leading to biased AI systems.
• Ethical Implications: AI can be used for both good and harm, raising ethical questions about its impact on human lives (e.g., job decisions, financial loans).
• Human Factors and Societal Issues: Addressing AI issues requires considering human factors and societal implications, not just engineering aspects.
• AI in Medicine and Healthcare: AI has the potential to deliver care to people, particularly for aging populations and patients.
• The Nuance of Human Vision: Human vision is still far more nuanced, subtle, rich, complex, and emotional than computer vision.

6. Course Overview and Learning Objectives (Professor Adeli)

• The course will cover a wide variety of topics around computer vision and the use of deep learning.
• Four main topics:
  • Deep Learning Basics
  • Perceiving and Understanding the Visual World
  • Generative and Interactive Visual Intelligence
  • Human-Centered Applications and Implications
• Deep Learning Basics:
  • Image classification as a fundamental task.
  • Linear classification and its limitations.
  • Overfitting and underfitting.
  • Regularization and optimization.
  • Neural networks for modeling non-linear functions.
• Perceiving and Understanding the Visual World:
  • Defining tasks (object detection, scene understanding, motion detection).
  • Using models (neural networks) to solve these tasks.
  • Tasks beyond classification: semantic segmentation, object detection, instance segmentation.
  • Temporal dimensions: video classification, multimodal video understanding.
  • Visualization and understanding of model behavior (attention maps).
  • Models: CNNs, recurrent neural networks, transformers, attention-based frameworks.
• Large-Scale Distributed Training:
  • Training large language models and large vision models.
  • Data parallelization and model parallelization.
  • Challenges: synchronization between models and workers.
• Generative and Interactive Visual Intelligence:
  • Self-supervised learning.
  • Generative models: style transfer, image generation (Dall-E, diffusion models).
  • Vision language models.
  • 3D vision.
  • Embodied agents.
• Human-Centered Applications and Implications:
  • Impact of computer vision and AI.
  • Turing Award and Nobel Prize recognition.
• Learning Objectives:
  • Formalizing computer vision applications into tasks.
  • Developing and training vision models.
  • Understanding the current state and future directions of the field.

7. Conclusion

The lecture provides a comprehensive overview of the history, current state, and future directions of computer vision and deep learning. It highlights the key milestones, breakthroughs, and challenges in the field, emphasizing the importance of data, algorithms, and hardware. The lecture also underscores the ethical considerations and societal implications of AI, urging students to consider the human factors and potential biases in AI systems. The course aims to equip students with the knowledge and skills to develop and train vision models, understand the current state of the field, and contribute to its future advancements.