TWiST 500 interviews with Cortical Labs, Turing, AND Mercor | E2159

Key Concepts

LLMs (Large Language Models), Benchmarking, Evaluation, Data Generation, Real-world Applications, Synthetic Data, AGI (Artificial General Intelligence), Cortical Labs, Biologic Computing, Digital Computing, Neurons, Multi-electrode Array, Free Energy Principle, CL1, Energy Efficiency, Data Scarcity, Reinforcement Learning, Turing, Private Evals, Public Benchmarks, Data Contamination, SWEBench, Frontier Data, Supervised Fine-tuning, Reinforcement Learning Gym, Mker, Labor Market Inefficiency, Matching Problem, Remote Work, AI Interviewer, Two-Sided Retention, Power Laws.

GPT-5 Speculation and AI Hype

The discussion begins with speculation around GPT-5, the anticipated next iteration of OpenAI's language model. Sam Altman's comments about GPT-5 have sparked both excitement and skepticism.

• Altman's PR Strategy: It's noted that Sam Altman is known for generating hype around OpenAI's products, sometimes leading to unrealistic expectations.
• Ethical Concerns: Concerns about the misuse of AI, such as voice cloning for scams, are acknowledged, echoing worries from previous model releases like GPT-3 and GPT-2.
• Iterative Improvement: The speakers suggest that GPT-5 might represent a gradual improvement rather than a revolutionary leap, similar to the progression from GPT-2 to GPT-4.
• Sci-Fi Influence: The conversation touches on the influence of science fiction narratives on perceptions of AI, particularly the fear of AI becoming self-aware and potentially dangerous.
• Release Timeline: The expected release of OpenAI's open weights model and GPT-5 (anticipated in August) is discussed, with predictions and wagering already occurring in online markets.
• Model Flurry: The discussion mentions a "flurry of models" from various companies like XAI (Grok), Moonshot (Kimi K2), and Alibaba (Quinn), indicating intense competition in the AI space.

Cortical Labs: Hybrid Biologic and Digital Computing

This section focuses on an interview with Han Wing Chong, CEO of Cortical Labs, a company developing hybrid biologic and digital computers.

• Core Technology: Cortical Labs uses neurons grown on multi-electrode arrays to create computing systems. Electricity serves as the shared language between neurons and chips.
• Inverted Neuralink: Cortical Labs is described as an "inverted Neuralink," taking the brain out and putting it into a computer, as opposed to putting a computer into the brain.
• Pong Experiment: The company gained initial recognition for teaching neurons in a petri dish to play Pong. The neurons were given electrical signals, with structured signals for correct actions and unstructured signals for incorrect actions.
• Free Energy Principle: The approach is based on the "free energy principle," which posits that the brain is a predictive machine generating hypotheses about the world and using sensory input to test those hypotheses.
• CL1 Commercial Device: The CL1 is Cortical Labs' first commercial device, designed to support research by providing a reliable and accessible platform for working with biologic computing.
• Life Support Plumbing: A significant challenge in developing the CL1 was creating a reliable life support system for the neurons, including maintaining the correct chemical balance and preventing contamination. The CL1 can keep neurons alive for six months.
• Energy Efficiency: Biologic computing is extremely energy-efficient, potentially using orders of magnitude less energy than traditional silicon-based computers. The energy source is glucose in the cell culture media.
• Data Scarcity: Cortical Labs' technology is particularly suited for problems where large datasets are not available, as biologic systems can learn with significantly less data than traditional AI models.
• Commercial Applications: Potential applications include real-time problem-solving and domains where data is scarce.
• Future Development: Future plans include making the CL1 smaller, cheaper, easier to program, and more modular.
• Neuron Source: Cortical Labs uses both mouse and human neurons. Human neurons are grown from stem cells derived from adult cells, avoiding the need to kill animals.
• Demand: Demand for the CL1 and Cortical Cloud has exceeded expectations, creating challenges in logistics and supply chain management.
• Funding: Cortical Labs is seeking further funding, particularly from investors who are willing to take risks on unconventional technologies.

Turing: LLM Evaluation and Data Generation

This section summarizes an interview with Jonathan Sedarth, CEO of Turing, a company specializing in LLM evaluation and data generation.

• LLM Evaluation: Turing helps companies benchmark LLMs to identify strengths and weaknesses, enabling them to generate data to improve model performance.
• Real-World Relevance: A key focus is on evaluating LLMs in real-world applications rather than relying solely on academic or synthetic benchmarks.
• Evaluation Dimensions: The three dimensions to look at when evaluating models are complexity, real world use, and diversity.
• Public vs. Private Evals: While public benchmarks are useful for bragging rights and recruiting, private evaluations are more important for assessing performance on specific use cases and query distributions.
• Benchmark Saturation: Existing benchmarks, such as SWEBench for coding, are becoming saturated as models achieve high scores, necessitating the creation of more challenging benchmarks.
• Data Generation: Turing not only evaluates models but also generates data to help them improve. This involves expert humans creating question-answer pairs that expose the model's weaknesses.
• Frontier Data: As models become more advanced, the data required to improve them becomes more complex, requiring specialized expertise from fields like physics, chemistry, and mathematics.
• Reinforcement Learning Gyms: Turing creates "reinforcement learning gyms," virtual environments that simulate real-world scenarios, allowing agents to learn through trial and error.
• AI-Assisted Data Generation: Turing uses AI tools to assist humans in generating high-quality data, including AI-driven quality control.
• Spillover Effects: Improvements in specific domains, such as coding and mathematics, can have spillover effects on other areas of model performance.

Mker: AI-Powered Talent Matching

This section summarizes an interview with Brendan Foody, CEO of Mker, a company aiming to revolutionize the labor market using AI-powered talent matching.

• Labor Market Inefficiency: Mker's vision is based on the idea that the labor market is the largest and most inefficient market in the world due to the difficulty of matching candidates with jobs.
• Unified Labor Market: Mker aims to create a global, unified labor market where candidates can apply to all available jobs and companies can access a much broader talent pool.
• Remote Work: Mker believes that remote work will continue to be prevalent, enabling access to a global talent pool.
• Centralized Hub: Mker envisions becoming the central hub for job searching and hiring, replacing fragmented job boards and traditional recruiting agencies.
• AI-Powered Matching: Mker uses AI to automate the matching process, including resume review, interviews, and candidate evaluation.
• AI Interviewer: Mker has developed an AI interviewer that conducts interviews and evaluates candidates.
• Multilingual Capabilities: Mker is expanding its AI interviewer to support multiple languages.
• Target Market: Mker initially focused on helping AI companies find talent but is expanding to other industries.
• Two-Sided Retention: Mker tracks two-sided retention, measuring both candidate and company satisfaction.
• Performance Reviews: Mker collects performance reviews and other data to improve its matching algorithms.
• Power Laws: Mker focuses on identifying and placing top-performing candidates, recognizing that a small percentage of individuals often drive the majority of value.
• Growth: Mker has experienced rapid growth, with revenue increasing significantly since its last funding round.
• Expansion: Mker is expanding its services beyond the AI industry and is also working on longer-term job placements.

Synthesis/Conclusion

The interviews highlight innovative approaches to AI development and application. Cortical Labs is pioneering biologic computing, offering potential advantages in energy efficiency and data-scarce environments. Turing is addressing the critical need for robust LLM evaluation and high-quality data generation. Mker is leveraging AI to transform the labor market, aiming to create a more efficient and accessible system for both job seekers and employers. These companies represent diverse facets of the evolving AI landscape, each tackling unique challenges and opportunities.