Building the world’s largest 3D object library | Microsoft Azure and NVIDIA | Catalyst E3

Key Concepts

• Photorealistic 3D Content: Creating digital representations that are indistinguishable from real-world objects.
• Digital Twins: Virtual replicas of physical objects or environments.
• AI Training Data: Using 3D object data to train AI models to understand spatial relationships and the real world.
• Authenticity: Maintaining the accuracy and realism of digital objects to ensure they are true representations of their real-world counterparts.
• Procedural Tech Stack: A workflow that uses automated processes to capture, process, and transform data into digital assets.
• Accelerated Computing: Utilizing GPUs (NVIDIA RTX A6000s and H100s) and cloud infrastructure (Azure) to handle the intensive processing of large scan data.
• Cultural Preservation: Digitizing artifacts and environments to preserve them for future generations.
• Spatialness: The concept of understanding and representing objects and environments in three dimensions.
• Lidar: Light Detection and Ranging, a remote sensing technology that uses laser light to create high-resolution surface maps.
• Photogrammetry: The science of obtaining reliable measurements from photographic images.

The Hunger of AI and the Need for High-Quality 3D Data

The video emphasizes the growing demand of AI systems for more and more detailed information. The core argument is that AI needs high-quality, photorealistic 3D data to truly understand the world. The creation of Global Objects stemmed from a recognized gap in the availability of such content. The company aims to "digitize the world" by creating accurate digital twins of real-world objects.

• AI's Data Appetite: AI systems require exponentially growing amounts of information, emphasizing the importance of detail and granularity.
• The 3D Frontier: The real world of 3D objects is presented as the next frontier for AI training, moving beyond textual, audio, and 2D data.
• Spatial Understanding: The concept of "spatialness" is crucial for AI to understand environments and relationships between objects.
• Data Set Limitations: There is a current lack of good three-dimensional data sets to train AI models effectively.

Global Objects: Digitizing the World

Global Objects focuses on creating authentic object models for various applications, including media, entertainment, and other industry verticals. Their workflow, powered by Microsoft and NVIDIA technologies, aims to achieve the highest possible fidelity, making digital twins indistinguishable from real objects.

• Truth Sets: Every piece of data created by Global Objects is based on a "truth set," meaning it is derived from a real-world object.
• Segmentation: Data sets are segmented to identify and categorize different components (e.g., "this is a bush, this is a tree").
• Prop House Collaboration: Global Objects partners with prop houses to access a vast library of physical objects for digitization.
• Multiple Use Cases: The digitized assets are intended for various applications, onboarding assets into the digital world.

Technology and Infrastructure

The video highlights the importance of a robust tech stack for capturing, processing, and delivering high-fidelity 3D data. NVIDIA GPUs and Microsoft Azure cloud services play a crucial role in handling the intensive computational demands.

• Procedural Tech Stack: Global Objects utilizes a procedurally driven tech stack for capturing data at super high fidelity.
• Capture Process: Each object is captured using thousands of pictures and high-precision lasers.
• Hardware: NVIDIA RTX A6000 GPUs are used at the desktop level, while NVIDIA H100 GPUs are utilized on the Azure Cloud.
• Accelerated Computing: NVIDIA GPUs handle the heavy lifting of processing and transforming large scan data into AI-ready digital assets.
• Scalability: The NVIDIA GPU architecture and Azure VMs provide on-demand scalability, allowing Global Objects to compete with larger companies.
• Data Processing: Processing speed and file size management are critical, requiring the best systems, servers, operators, and GPUs.

Applications and Use Cases

The digitized 3D objects have a wide range of applications across various industries, including gaming, media, entertainment, robotics, medicine, real estate, retail, and CPG.

• Vast Prop Library Digitization: Digitizing a vast prop library opens up opportunities for gaming applications, rendering in advertising, and social content creation.
• Electric Vehicle Repair Training: Global Objects is working with a group to train people on repairing electric vehicles using realistic 3D models.
• PPE Training: Creating realistic environments for training workers on how to inspect PPE equipment.
• Virtual Reality Training: Using VR to create interactive environments for training, allowing users to inspect parts and understand procedures.

Scanning and Data Replication

The video details the process of scanning objects and environments using various technologies, including lidar, photogrammetry, and drones. It also highlights the importance of data replication for efficient collaboration.

• Scanning Process: A combination of blue laser, photogrammetry, photometric, and lidar techniques is used to capture objects.
• Lidar Technology: Lidar is used for high-resolution surface mapping, sending out laser pulses and measuring the return.
• Drone Coverage: Drones are used to capture color and texture data.
• Global Data Replication: Data is replicated across the globe using Azure Blob Storage, enabling teams in different time zones to collaborate efficiently.

Cultural Preservation and Authenticity

A key motivation for Global Objects is cultural preservation. The company is working with museums to digitize artwork and environments, making them accessible to a global audience and preserving them for future generations.

• Museum of Latin American Art (MOLAA): Global Objects digitized artwork and environments at MOLAA, allowing people to experience the art in a digital space.
• Conservation Reference: High-definition digitization provides a realistic visual reference for conservators to use when restoring artwork.
• Proactive Stewardship: Taking measures to ensure the artwork is preserved and its lifespan is extended.
• Authenticity and AI: Maintaining authenticity is crucial for AI-generated objects to connect with human understanding and the real world.

Conclusion

Global Objects is building the foundation for the future of AI and digital content creation by providing high-quality, photorealistic 3D data. Their work spans across various industries, from entertainment and gaming to training and cultural preservation. By leveraging advanced technologies and a commitment to authenticity, Global Objects aims to make 3D accessible to everyone and unlock new possibilities for AI and human interaction. The company's focus on creating a "foundation of the real world" positions them as a key player in the convergence of AI and 3D technology.