Reality at a Distance | Haotian Zhang | TEDxHKU

Key Concepts:

• Generative AI: Using AI to create new content (images, videos, 3D models).
• Text-to-Image/Video/3D: AI models that generate content from text prompts.
• 3D Scanning: Capturing the geometry and appearance of real-world objects.
• Photogrammetry: A 3D scanning technique using photographs.
• Neural Radiance Fields (NeRFs) & Gaussian Splatting: Advanced 3D scanning/rendering methods.
• PBR (Physically Based Rendering): A rendering technique that simulates realistic material properties.
• AI Interpolation: AI creating frames between two given frames.
• Image-to-3D: AI models that generate 3D models from 2D images.
• Recursive Translation: Translating data between different formats multiple times (e.g., 3D model to text to 3D model).
• Ray Tracing: A rendering technique that simulates the path of light rays.
• Shading: Calculating the brightness and color of a surface in a 3D scene.
• Tool vs. Product: Differentiating between technologies that empower users and those with predefined functions.

1. AI Buzzwords and the Designer's Perspective

• The speaker begins by referencing a survey conducted using GPT to identify common buzzwords associated with AI in newspaper articles. These include phrases like "the future of technology," "a new frontier," "a new paradigm," and "new era."
• The speaker emphasizes their perspective as a designer, not an AI scientist or investor, preferring to view AI as a commonplace tool, similar to a pen or violin.
• The speaker introduces themselves as an architect experimenting with digital mediums, including generative AI, 3D scanning, and 3D visualization.

2. The Rapid Evolution of AI: Examples and Comparisons

• The speaker highlights the astonishing speed at which AI technologies are evolving, referencing the "Will Smith eating spaghetti" meme (though not shown due to copyright).
• Video Generation Example:
  • A video generated in 2021 using an advanced AI generator with the prompt "an endless corridor in the film The Shining" is shown. The result captures iconic elements but suffers from flickering, ghostly figures, and a poor understanding of space and perspective.
  • A later experiment uses a single frame from the previous video with a newer AI video generator. While the first frame is still unrealistic, the AI quickly adjusts and generates a convincing walk down the corridor.
• Text-to-3D Example:
  • In 2022, OpenAI's Point-E, one of the first text-to-3D models, produced rough point cloud models with limited colors.
  • By 2024, tools can generate high-resolution mesh models with PBR materials in minutes, producing results closer to reality (e.g., a realistic pumpkin).
• 3D Scanning Example:
  • Conventional photogrammetry struggles with capturing non-matte or non-solid objects like trees or water.
  • Recent methods like Neural Radiance Fields (NeRFs) and Gaussian Splatting overcome this limitation, realistically rendering refraction, reflection, and fuzziness.

3. The Challenge of Keeping Up and the Value of Early Experimentation

• The speaker acknowledges the challenge of keeping up with the rapid pace of AI development, especially for designers who might spend a month learning a tool only to find a more powerful one released.
• Instead of waiting for mature tools, the speaker advocates for experimenting with early versions to understand their limits and discover their potential.
• The speaker argues that early versions of AI tools aren't necessarily bad; they're just different and sometimes more original.

4. AI-Generated Film: Blending Reality and the Unreal

• The speaker presents an AI-generated film of a tour down Hill Road in Hong Kong, created by interpolating between real street views using AI.
• AI demonstrates an incredible knowledge of space, creating coherent transitions between dissimilar beginning and end frames.
• However, the speaker highlights intentional "weird moments" (e.g., a bridge turning at an awkward angle, a building appearing cut in half) to create a distance between the real and the unreal, prompting viewers to question what they see.
• The project aims to forge two spaces that don't belong together into a coherent journey.

5. Image-to-3D: Capturing the Essence of Hong Kong

• The speaker describes a project using image-to-3D AI to translate 2D cutouts of typical Hong Kong street objects into 3D models.
• The AI captures funny imperfections (e.g., a red bag recognized as a pendant, a missing hand).
• Despite individual models being ambiguous and funky (blobby shapes, low-resolution textures), they collectively capture the chaotic but vibrant nature of Hong Kong streets.
• The work was initially done with students in a collaborative workshop.

6. Recursive Translation: Aestheticizing Infidelity

• The speaker and partners pushed the project further by translating the 3D models into three keywords using a different AI tool, then using those keywords to generate new 3D models.
• While each translation is fairly accurate on its own, the final result appears more generic, losing the unique qualities that made the original models relevant to Hong Kong.
• The speaker argues that in this recursive translation, the infidelity of AI translation accumulates and becomes a valuable aesthetic quality.
• The speaker is analyzing the tool and aestheticizing it, especially when such unique qualities are most visible in the early stages.

7. Questioning the Definition of a "Better" Tool

• The speaker notes that within months of completing the first demo, both AI tools used had been upgraded and became more powerful.
• The speaker questions whether AI tools creating more convincing results with almost no effort necessarily makes them "better."
• The speaker questions the very direction of advancement, drawing an analogy to painting before photography. The pursuit of realistic representation in painting became less relevant after photography could do it faster and better, leading artists to explore other qualities of the medium (e.g., Impressionism).
• The speaker suggests examining AI tools retrospectively, as if standing 50 years from now, and imagining a "post-AI" condition.

8. Tool vs. Product: The Guitar and the iPad

• The speaker draws an analogy between an iPad and a guitar to illustrate the difference between a product and a tool.
• iPads are upgraded almost every year, and users buy new versions believing they offer more powerful functions and a smoother experience.
• The speaker is skeptical of the idea of inventing a "new guitar" because, unlike the pursuit of higher performance in a product like an iPad, it doesn't make sense to make a louder guitar.
• An iPad is a product with predefined functions, while a guitar is a tool that lends itself fully to the user's creativity.
• The speaker aims to bring digital mediums closer to being tools rather than products.

9. Physical Rendering: Reclaiming Agency

• The speaker presents a project dealing with rendering, an archaic digital technique, done in an even more archaic way – completely physically.
• The speaker explains the basics of ray tracing, where a ray is shot from the camera through a pixel until it hits something, and the brightness and color of that spot are calculated.
• The speaker manually calculated the shading by measuring the angle between a string representing a light source and the surface of the classroom.
• The speaker illuminated the space spot by spot, taking over 3,000 individual pictures that were composited into the final rendering.
• The speaker prolonged a process that is typically instantaneous in digital rendering (milliseconds) to three days.
• The speaker is trying to open the black box and examine what is going on in these instantaneous digital processes (rendering or AI) to reclaim our agencies as designers and as humans.

10. Conclusion: Examining and Understanding AI

• The speaker emphasizes that rapidly developing techniques like AI call for careful examination to exploit their creative potential and understand them better.
• Understanding these techniques allows for discussion and incorporation into the discourse of art and architecture before their unique qualities are lost to history.
• The speaker points out the accidental capture of their hands in the final image as an imperfection of the process, representing the trace of the material and physical process itself.
• This separates the final work from a seamless digital image or reality, exposing the distance between them.

Main Takeaways/Synthesis:

The speaker advocates for a critical and experimental approach to AI, emphasizing the importance of understanding its limitations and unique qualities, especially in its early stages. By treating AI as a tool rather than a product, designers can reclaim their agency and explore its creative potential in meaningful ways. The speaker's projects demonstrate how embracing imperfections and prolonging instantaneous digital processes can lead to new aesthetic possibilities and a deeper understanding of the technology's impact on art and architecture. The core message is to not blindly accept AI's advancements but to actively engage with it, question its direction, and shape its role in creative practices.