A Masterclass on AI: How do LLMs Work and how does is Impact Investing?

Key Concepts

• Machine Learning (ML): The process of teaching computers to learn from data, moving from rule-based programming to statistical models.
• Deep Learning: A subset of ML using multi-layered neural networks to learn complex patterns.
• Transformer Architecture: The foundational technology behind modern LLMs, utilizing "attention" mechanisms to process data.
• Reasoning Models: AI models that perform multi-step thinking, self-verification, and planning before outputting an answer.
• AI Agents: Systems that use an LLM as a "brain" to orchestrate tools (APIs, browsers, file systems) to complete complex tasks.
• Context Window: The amount of information (tokens) an LLM can process in a single interaction.
• Embeddings: Mathematical representations (vectors) of words or data that allow models to understand relationships and context.
• Inference: The process of running a pre-trained model to generate outputs, which is computationally cheaper than training.
• RAG (Retrieval-Augmented Generation): A technique to improve LLM accuracy by feeding relevant, structured data into the context window.

---

1. Evolution of the Technology Landscape

The speaker traces the shift from Classical Machine Learning (supervised, unsupervised, and reinforcement learning) to Generative AI.

• Classical ML: Relied on "feature engineering," where humans manually identified important variables (e.g., square footage for house prices). It was limited by scale (single-machine processing) and struggled with unstructured data.
• Deep Learning: Introduced layered architectures where each layer learns a facet of the data (e.g., edges, then shapes, then objects). This enabled Transfer Learning, allowing models trained on one dataset to be fine-tuned for different domains.
• The Transformer Moment: The "Attention is All You Need" paper introduced the ability to process data in parallel, allowing for massive scaling.

2. The Mechanics of LLMs and Reasoning

• Tokens and Embeddings: Words are converted into numerical vectors (embeddings). The model uses these to calculate probabilities for the next word in a sequence.
• Temperature: A parameter that controls randomness. Lower temperature makes the model deterministic (most likely word); higher temperature increases creativity.
• Reasoning Models (e.g., DeepSeek, Claude Sonnet): Unlike "one-shot" models that provide immediate answers, reasoning models break problems into sub-tasks, verify their own work, and iterate before delivering a final response.

3. AI Agents and Tool Orchestration

The speaker argues that the "agentic" shift is the most significant innovation for software engineering.

• Agent Harness: An agent is an LLM combined with an "agent harness"—a set of tools (APIs, file access, web browsers) that the model can orchestrate.
• Automation of Programming: Because code follows fixed syntax rules, LLMs are highly effective at writing and executing code to perform tasks (e.g., parsing PDFs, querying databases). This effectively turns legacy software (Salesforce, Workday) into a collection of tools for the agent to manage.
• OpenCloud: A platform that democratizes agent building by allowing users to define "skills" (instructions in plain English/Markdown) that agents can execute.

4. Hardware and Economic Moats

• Nvidia’s Moat: Nvidia’s dominance is not just in hardware, but in the CUDA software layer, which translates high-level code (PyTorch) into system instructions. Replicating this ecosystem is a massive barrier to entry for competitors.
• Training vs. Inference: Training requires massive GPU clusters and hundreds of millions of dollars. Inference (running the model) is significantly cheaper and can often run on standard hardware, provided there is enough memory to hold the model weights.
• Data as the Future Moat: The speaker suggests that as LLMs become commodities, the true competitive advantage will shift from the model itself to the proprietary data companies hold about their users.

5. Key Quotes and Perspectives

• "The beauty of code is that it follows a set of rules... there’s no new words coming... code automatically becomes a first-class contract to be automated by the LLMs."
• On the future of software: "You will never have to log into Salesforce, you never have to log into Workday... the idea of you having a fancy UI kind of goes away."
• On the pace of change: "I’ve never seen this much amount of pace where my skills so rapidly degraded... I feel like an idiot. But that said, I’ve become more valuable to my customers simply because I have so much wealth of knowledge."

Synthesis/Conclusion

The current AI landscape is moving away from simple "chatbots" toward autonomous agents that can reason, plan, and execute tasks across enterprise systems. While the underlying LLM models are becoming commodities, the ability to build "agent harnesses" and integrate them with proprietary data is where the real value lies. The speaker concludes that while technical skills are evolving rapidly, the ability to understand and leverage these tools to solve real-world problems remains the most valuable asset for professionals.