Digital Twins Meet Game Engines: A New Era for AEC Projects

Key Concepts

• GIS (Geographic Information System): Computer mapping system to store, manage, distribute, analyze, and simulate real-world geospatial information in 2D and 3D.
• SDK (Software Development Kit): A package provided to developers including APIs, documentation, samples, and toolkits for building client applications.
• 3D Digital Twin: A digital representation of a physical asset or system that mirrors its real-world counterpart in 3D, often updated with real-time data.
• Game Engines (Unity, Unreal Engine): Software frameworks designed for creating video games, offering high-powered visualization, physics engines, and particle systems.
• Arctis Maps SDKs: EZRI's software development kits for building mapping applications across web, native, and game engine platforms.
• XR (Extended Reality): An umbrella term encompassing augmented reality (AR), virtual reality (VR), and mixed reality (MR) technologies.
• AEC: Architecture, Engineering, and Construction industry.
• BIM: Building Information Modeling.
• VPS (Visual Positioning System): A system that uses visual data to determine the precise location and orientation of a device, often indoors or in areas with poor GPS coverage.

Rex Hansen's Background and Role at EZRI

Rex Hansen is the Principal Product Manager for ArcJist Maps SDKs for native apps and game engines at EZRI. His interest in maps, history, and science led him to GIS. He holds a master's degree in GIS and geostatistics from the University of Arizona. He joined EZRI in 1997, working with desktop products like Arcview and developing skills in Avenue scripting language. He then worked on server-based products like ArcIMS using Java. His experience evolved through .NET and C# and then to native mobile development with iOS and Android using Objective-C and Java. He transitioned to focusing on 3D capabilities and became a product manager to drive developer tools for both EZRI and its partners to build 2D and 3D applications. He focused on native and game engine capabilities for Arches Maps SDKs.

GIS and SDK Defined

GIS (Geographic Information System) is essentially computer mapping, an informative system that allows you to store 2D and 3D spatial data and manage, distribute, share, collaborate, analyze, and simulate around real-world geospatial information.

SDK (Software Development Kit) is what's provided to developers to build client applications. It includes an API for use within an IDE, samples, documentation, and toolkits. It enhances mapping experience with additional components. Rex uses the standing on the shoulders of giants analogy because developers can use the SDK to customize their work for a specific use case.

Integration of GIS with Real-Time Game Engines and Immersive Digital Twins

The combination of GIS and real-time game engines arose from the maturing of EZRI's 3D GIS platform and the introduction of mixed reality experiences like Microsoft's Hollow Lens (2016) and location-oriented games like Pokémon Go (2016). Customers wanted to use their digital geospatial information to enhance their business, communicate, collect data, and collaborate more effectively.

Game engines provided realistic but artificial worlds with high-powered visualization capabilities, particle systems, and physics engines. AEC and other industries wanted to combine these capabilities with real-world data, real-time information, animation, and simulation workflows.

EZRI responded by working with Unity and Epic Games (Unreal Engine) and introduced a beta program in 2020, providing plugins for the developer environments. The beta program was very popular with 7,000 participants. AEC firms led the way in using these tools, seeking differentiation, attention, and engagement for real estate, city planning, and construction projects. They needed real-world digital twins to look compelling and act like the real world, with spatial accuracy provided by EZRI and immersive fidelity provided by the game engine.

Real-World Examples of Arctis Maps SDKs Use in AEC

Rex categorizes the ways that AEC professionals use Arctis maps SDKs into inspirational, developmental, and operational needs.

Inspirational:

• House Lavine and St. Louis: Housed Lavine worked with St. Louis and the St. Louis Development Commission to rebuild the Monarch and MLK campus as a symbol of transformation and rebirth for the neighborhood. The city of St. Louis and EZRI provided data, and local architects provided models. Housed Lavine used CityEngine to ingest and refine structures and published them as services within the arch system. The project needed something more compelling. Housed Lavine used the Arches Maps SDK and Unreal Engine to deliver an application with compelling visuals and an interactive experience for the community. They also generated videos for political leaders to promote the capabilities.

Operational:

• Port Corpus Christi: Port Corpus Christi modernized its site management using living digital twins. They migrated from decentralized systems to a modern GS system of record based on RJS. They worked with the acceleration agency to build a visual and compelling experience for both back-office and field use. They used the Arches Maps SDK for Unity to build the Optics application, which provides a common operating picture for personnel involved in monitoring and securing the port.

Offline Access to Rich Geospatial Data

The RJS system supports the ability to generate local cache files of 3D data, including objects, meshes with attributes, terrain, vectors, imagery. These files can be taken offline and accessed within Arches Maps SDKs for native apps and game engines.

The geospatial context in a game engine scene or level can be established and utilized by multiple layers from different sources, both online and offline.

Integration of Real-World Sensor and IoT Data

Tools like Unreal and Unity are evolving to integrate real-world sensor and IoT data. The idea is to reduce the abstraction required by users of GIS and digital twins, making the 3D environment look and act more like the real world. EZRI works with Unity and Unreal Engine to help introduce high precision real-world data and access to multiple sources of RS data through services or packages. This allows for weather simulations, pedestrian traffic simulations, and other scenarios.

GIS and Game Engine Integration Supporting City-Scale Digital Transformation

GIS provides spatial intelligence, and game engines deliver immersive experiences. Combined, they can generate living digital ecosystems that can be planned, monitored, and improved collaboratively.

Arches Urban and Urban XR:

Arches Urban is designed to streamline urban planning workflows. EZRI is developing Urban XR, an application built with Unreal Engine, to provide a more immersive experience for urban planners and decision-makers. Users can author a scene in a browser, bring in building data, terrain, and imagery, and then visualize it in an XR headset as a virtual tabletop experience or at full scale at ground level. This allows for more effective presentations and collaboration with stakeholders.

Emerging Trends and Capabilities in GIS and Immersive Technology

Rex identified five emerging trends:

1. Increasing Integration Between GIS and BIM: This will drive immersive digital twins across industries. Partnerships like EZRI and Autodesk will help streamline workflows around building construction, engineering redevelopment, and material movement.
2. Visual Positioning Systems (VPS) and Better Connectivity: 5G and other technologies will provide industrial-grade precision for mobile devices in the field, enabling more intuitive inspection and collection workflows.
3. High-Fidelity 3D Data Capture Technology: Will provide better assessments of what's in the field and enable better positioning systems.
4. AI and Geospatial Combination: AI will enable faster data interpretation, classification, and assessment of changes, as well as predictive analytics for planning and forecasting.
5. Advancements in Extended Reality (XR): High-fidelity pass-through in headsets like Apple Vision Pro and Vario's XR4 are emerging, wearables that can overlay geospatially accurate digital data will emerge in 5-10 years.

Advice for AEC Professionals

• Invest in a flexible, industrial-strength GIS: This GIS should be designed to capture, store, deliver, and analyze a wide variety of heterogeneous geospatial data. Ensure data is 3D GIS-ready.
• Grow knowledge in game engine technology: Become familiar with the immersive systems delivered by Unity and Unreal Engine and their continued advancements.
• Engage with partners with expertise: If internal growth is not possible, reach out to partners with expertise to combine GS capabilities with the power of game engines.
• Adopt the crawl, walk, run approach: Introduce new technology and experiences gradually, demonstrating an initial return on investment and getting buy-in before expanding.

Contact Information

• LinkedIn: Reach out to Rex Hansen on LinkedIn.