AGI, Sci-Fi, and Storytelling | Neal Stephenson, Ken Liu, & more

Key Concepts

• AGI (Artificial General Intelligence): A hypothetical level of AI that can perform any intellectual task that a human being can.
• LLMs (Large Language Models): AI models trained on vast amounts of text data, capable of generating human-like text, translating languages, and answering questions.
• Cognitive Architectures: Frameworks that attempt to model the structure and processes of the human mind in AI systems.
• Mythology of Technology: The underlying narratives and cultural meanings associated with technological advancements, often more enduring than specific technical details.
• World Building: The process of constructing a coherent and consistent fictional world, with its own rules, history, and culture.
• Cyber Animism: The belief that spirits are self-organizing pieces of software that emerge in nature and shape it.
• Emotional Labor: The effort required to manage emotions as part of a job or social role.

Nuclear Age vs. AI Age

• Neil Stephenson: The rapid onset and diffusion of AI into general awareness is similar to the early days of nuclear fission. Some are questioning if other fields are now irrelevant compared to AI.
• Ken Liu: The atomic age was a disappointment. Sci-fi envisioned it as transformative, but it became a tool of stasis. Nuclear power plants are also disappointing. AI could replace semi-creative jobs, leading to an "industrial revolution 2.001," or lead to innovations like protein folding.
• Protein Folding: Ken Liu considers AI's most impactful application to be in protein folding, solving a previously deemed unsolvable problem.

Progress and the Future of AI

• Yosha Bach: The future is changing faster than existing systems can adapt. The answer to AGI wasn't a "smart master algorithm," but more compute and data.
• LLMs and Diamond Age: In Diamond Age, LLMs are smart but not agentic, and don't change the world's identity. The future may involve humans hybridizing with AI, with personal AIs making decisions.
• Ken Howery: Investors should look 6-8 years ahead, focusing on the "picks and shovels" of the AI revolution (power, compute). AI has the potential to be a mental capacity multiplier, serving as a general interface between domains of expert knowledge.
• Mental Capacity Multiplier: AI can enable a small number of individuals to understand and utilize diverse domains like nanotechnology and bioengineering.
• Human Intelligence: The question of how many people can become agents in a world where AI seamlessly extends their minds.

Narratives and Expectations

• Narrative Violation: The way AI developed went against expectations shaped by science fiction. AI became creative before it could perform simple physical tasks.
• Ken Liu: Science fiction authors get the specifics wrong but the mythology right. AI research has been focused on answering Turing's question: "Can machines think?" The imitation game has revealed that "thinking" is many different things.
• Creativity and AI: Ken Liu was surprised that AI could carry on conversations without consciousness or self-awareness.
• Mythmaking: Good sci-fi authors are mythmakers, delving into the collective unconscious to talk about technology. Mary Shelley's Frankenstein got the mythology right, even if the technical details were wrong.
• Ken Howery: AI will unlock the ability for non-writers to dream about the future. It will be easier to create utopias in stories.
• Salis Lamb: Used writing to inspire a generation to build thinking computers.
• Imagination Limitations: Authors often lack the imagination to foresee the obvious applications of technology (e.g., search on the internet).

Sci-Fi and Reality

• Blurring Boundaries: The boundary between sci-fi and reality is blurring, with real-world projects resembling concepts from science fiction.
• Ken Liu: When something in the news "rhymes" with a prediction, it means the zeitgeist is right.
• Neil Stephenson: If a prediction comes true, it means he did his homework on the engineering and business logic.
• World Building: Creating a suspension of disbelief requires internal rules that are coherent and consistent.
• Ken Howery: World building is an important part of investing.
• Yosha Bach: Building a company requires creating meaning.
• Elon Musk: Was a storyteller, connecting people to the future with stories of going to Mars.
• Ken Liu: Technology is the human mind made tangible. Building things is a way to express values.
• Neil Stephenson: Every book is different. The world building should serve the story.

Author vs. Founder

• Neil Stephenson: Writing requires a balance between focused work and unrelated activities to allow for background processing.
• Ken Liu: Enjoys his job because it's playful, involving scientific conferences, video game discussions, and random conversations.
• Ken Howery: Her job is pure play, immersing herself in worlds and talking to people.

AI and Consciousness

• Yosha Bach: No one really knows if LLMs have phenomenology. When prompted, LLMs create a causal model of what they're doing.
• Simulation vs. Reality: Is the simulation created by an LLM more stimulated than what our brain produces?
• Universality Hypothesis: Models tend to converge on the same feature hierarchy, even with different architectures.
• Self-Organizing Systems: Can a self-organizing system exposed to complex tasks develop phenomenal experience?
• Data Processing Capacity: An H100 should be able to emulate what happens in our inner cinema.
• Neil Stephenson: Has a hard time getting a grip on what is meant by AGI.
• Snow Crash: Written in 1989-1990, before the internet as we know it.
• Ken Liu: The concept of emotional labor and its ties to AI. The displacement of emotional labor from the social realm to a professional realm.
• Machine Therapists: Patients often show great willingness to talk to machines for therapy, even preferring them to humans.
• Death and Creativity: The importance of death and perishability in shaping consciousness and behavior.
• Ken Howery: Understanding our own consciousness is more important than understanding whether AI will become conscious.
• Second Coming of Christ: AI consciousness is a downright miracle, like an immaculate conception.
• Post-Rapture Stories: It's difficult to tell stories about humans after they merge and become subset agnostic.
• Yosha Bach: LLMs are doing what we expect them to do, emulating human textual output.
• New Forms of Art: LLMs can create new forms of art that couldn't be written by humans.
• Universe Construction: LLMs can create universes as rich as the one in Diamond Age just for the purpose of making a couple sentences.

Conclusion

The panel explored the complex interplay between science fiction, technological advancement, and the human experience. They discussed the potential and pitfalls of AI, the importance of mythology in shaping our understanding of technology, and the challenges of predicting the future in a rapidly changing world. The conversation highlighted the need for both visionary thinking and a grounded understanding of the ethical and societal implications of AI.