On Curiosity — Sharif Shameem, Lexica

Key Concepts

Curiosity, Demos, Generative Models, Context Window, AI Engineering, Man-Machine Symbiosis, Exploration, Possibility, Moral Obligation, GPT-3, Gemini 2.5 Pro, Agency, Reasoning Engines, Subconscious Patterns, AI Pioneers.

Demos and Curiosity in AI Development

The Power of Curiosity

• The speaker, Sharief, argues that curiosity is the main force for pulling ideas from the future into the present. He references mathematician Poare's experience of suddenly understanding a complex problem while boarding a bus as an example of how subconscious patterns recognized by the brain surface as curiosity.
• He emphasizes that demos are a way of exploring what's possible with AI models and that the best way to interact with these models is by creating interesting demos driven by curiosity.

Demo Development Pattern

• Sharief outlines a common pattern in his demo development:
  1. Initial excitement about a fantastic idea.
  2. Realization that the idea is not initially feasible.
  3. Finding a way to make it work through effort and determination, even with limitations like small context windows.
  4. Experiencing a sense of pride and joy upon successful implementation.

Early GPT-3 Demos (2020)

• Context: GPT-3 had a 2,000 token context length and cost $75 per million output tokens. Sharing information required permission from OpenAI, and chat apps were prohibited.
• JSX Compiler in the Browser: Inspired by Brett Victor's principle of immediate feedback for creators, Sharief created a JSX compiler in the browser. This allowed for real-time code editing and execution, which felt "magical" compared to the traditional copy-paste-compile-run workflow.
• Google Homepage Generator: This demo generated a Google homepage by using three parallel prompts to overcome the small context window limitations. It demonstrated how to synthesize new ideas from existing experiences.

Agentive Model Demo (2021)

• Objective: To give the model the objective of buying AirPods in Chrome.
• Challenge: Web pages like walmart.com exceeded the 4,000 token context window.
• Solution: A custom HTML parser was created to extract the core essence of a webpage, fitting it within the context window.
• Outcome: The model got distracted by the terms of service, but the demo revealed that models pre-trained on web text had an internal sense of agency.

Gemini 2.5 Pro Basketball Shot Tracker

• A friend, Farsza, created a demo using Gemini 2.5 Pro to provide basketball feedback as if Michael Jordan were watching.
• This demo inspired the idea of video-first experiences with Gemini 2.5 Pro, expanding beyond screen feedback for coding.

The Potential of Existing Models

• Sharief believes that even without further model development (freezing the weights), there is enough potential in existing models to build amazing products for the next 10 years.
• He quotes Richard Hamming: "In science, if you know what you're doing, you should not be doing it. In engineering, if you know what you're doing, you should not be doing it." He argues that AI engineering is more like excavation, where demos are used to discover hidden capabilities within models.

The Role of Uncertainty

• Even researchers at OpenAI and Anthropic don't fully understand the capabilities of their models.
• Charles Darwin's eight years of studying barnacles before publishing his theory of evolution is used as an analogy. Sometimes, seemingly useless exploration leads to significant discoveries.

Exploring the Search Space

• Demos are a way to explore what's possible and expand the search space.
• The process is likened to crossing a foggy pond, where you take one step at a time and adjust your path based on the results.

The Intergalactic Spaceship Analogy

• A tweet comparing Claude to an intergalactic spaceship being sold as a toaster highlights the vast untapped potential of these models.
• Good demos reveal interesting capabilities through exploration and play.

The Uniqueness of Individual Perspective

• Each person's life and experiences create a unique "context window."
• Ideas that arise from this unique context may be exclusive to that individual, making it essential to try and realize them.

Man-Machine Symbiosis and Moral Obligation

• Referencing Lick Lighter's "Man-Machine Symbiosis," Sharief emphasizes the vast difference between the computing power of the 1960s and today.
• He argues that there is a moral obligation to honor the pioneers of computing by following curiosity and sharing explorations with the world.
• Sharing demos helps reveal what's possible with these models and moves the field forward.

Q&A Demos

• Vibe Coding: The 2020 demo was the start of vibe coding and allowed people to take these models as reasoning engines.
• Multivac: An old GPD3 demo where the idea was how to get these models to solve very large and ambitious problems by breaking down the problems into more digestible sub problems.

Conclusion

The key takeaway is that curiosity-driven exploration through demos is crucial for unlocking the full potential of AI models. Sharief advocates for a shift in perspective, viewing AI engineering as an excavation process where uncertainty is embraced and individual perspectives are valued. He urges developers to share their explorations, contributing to a collective understanding of what's possible and honoring the vision of AI pioneers.