Competence in the Age of LLMs | Peter Danenberg | TEDxCSTU

Key Concepts:

• Brain scans and neurological activation during problem-solving
• Pencil and paper, Google search, and LLM usage compared
• Aristotle's rhetoric: Poetic (generation) vs. Parasitic (dialectic/testing)
• Socratic method as a midwife of ideas
• Enkindling of the soul through dialectic
• Over-reliance on LLMs and outsourcing critical thinking
• Responsibility of AI tool builders to challenge users
• Achieving mastery and competence in the age of LLMs

1. Brain Scans and Problem-Solving Approaches

• An MIT study examined brain scans of individuals engaged in problem-solving and creative exercises.
• Participants were divided into three groups: pencil and paper, Google search, and LLM users.
• Pencil and paper users showed high neurological activation in the middle of the brain, potentially reflecting the struggle with generating ideas from scratch. This represents the traditional approach of investing "10,000 quality hours" to achieve competence.
• Google search users exhibited a more general activation pattern, suggesting a focus on synthesizing information from multiple sources (the "10 blue links") before generating something new.
• LLM users displayed minimal brain activation, indicating a state of neurological inertia, raising concerns about the long-term effects on competence and mastery.

2. Aristotle's Rhetoric: Poetic vs. Parasitic

• Aristotle's rhetoric distinguishes between the "poetic" and the "parasitic."
• The "poetic" refers to the art of rhetoric and generation, particularly the generation of words. LLMs excel at this, producing fluent text, compilable code, and convincing hallucinations.
• The "parasitic" represents dialectic, the art of testing and challenging ideas. Socrates' method of questioning and guiding individuals to birth their own ideas exemplifies this.
• Socrates acted as a "midwife of ideas," helping others develop their own thoughts through labor and effort, rather than simply providing answers.
• The absence of this process can lead to a twofold tragedy: ideas may never come into existence, and individuals may not develop the capacity to generate their own ideas.

3. Enkindling of the Soul through Dialectic

• Plato's seventh letter describes the "enkindling of the soul," a process that occurs through lifelong dialectic with a trusted companion.
• This process makes ideas neurologically "sticky" and salient, ensuring they remain with the individual for life.
• The question is raised whether LLMs can serve as parasitic companions, challenging users to develop their own ideas. Currently, LLMs are primarily poetic, focusing on generation rather than dialectic.

4. Over-Reliance on LLMs and Critical Thinking

• A Microsoft study found that engineers who over-rely on LLM-enabled tools tend to outsource their critical thinking to the LLM.
• Instead of being critical thinkers, they become "verifiers of information," simply approving or rejecting LLM-generated content.
• The study suggests that AI tool builders have a responsibility to challenge users and promote growth, rather than simply providing convenient solutions.

5. Achieving Mastery in the Age of LLMs

• Achieving mastery in the age of LLMs requires a twofold approach:
  • Users: Need to actively challenge LLMs, demanding that they force users to demonstrate mastery before providing solutions.
  • AI Tool Builders: Need to align models to challenge users, even if it means sacrificing some engagement metrics.
• A parasitic LLM may be uncomfortable at times, but it has the potential to foster growth and "enkindle the soul."

6. Notable Quotes

• "Socrates mentioned when Socrates talks with people he is a he's a midwife of ideas and what that means is when he gets together with somebody he instead of instead of giving them an answer instead of um sparing them from the labor of of thinking about things his job is to actually help people birth their own ideas"
• "Plato in the seventh letter describes this really interesting process where um having engaged in a lifelong of dialectic or parastic um uh activity with somebody that you love, somebody that you trust, a sort of sparring partner and dialectic, this process happens which he calls the enkindling of the soul."
• "As creators of tools, we have a responsibility in some sense to make sure that we're actually challenging the users uh that we're actually, you know, engaged in something with the user like a parastic process where we're actually challenging the user to grow, challenging the user maybe to demonstrate mastery."

7. Technical Terms and Concepts

• LLM: Large Language Model, a type of AI model capable of generating human-like text.
• Neurological Activation: Activity in the brain, often measured through brain scans.
• Poetic: In the context of Aristotle's rhetoric, the art of generation, particularly of words.
• Parasitic: In the context of Aristotle's rhetoric, the art of dialectic and testing ideas.
• Dialectic: A method of philosophical argument involving the exchange of opposing ideas.
• Enkindling of the Soul: A process of deep learning and understanding that occurs through lifelong dialectic.

8. Synthesis/Conclusion

The advent of LLMs presents both opportunities and challenges for learning and mastery. While LLMs excel at generating content (the "poetic"), it's crucial to cultivate critical thinking and dialectical skills (the "parasitic"). Users must actively challenge LLMs, and AI tool builders must design models that promote growth and "enkindle the soul," even if it means sacrificing some immediate gratification. The key is to avoid over-reliance on LLMs and to actively engage in the process of generating and testing ideas.