SO MANY THINGS need to go right just so you can watch a TikTok! | E2215

Key Concepts

• Agentic AI SR (Site Reliability Engineering): Applying modern AI to ensure cloud setups function correctly amidst increasing complexity.
• IT Operations (IT Ops): The management and maintenance of an organization's IT infrastructure.
• Site Reliability Engineering (SRE): A discipline that incorporates aspects of software engineering and applies them to infrastructure and operations problems.
• Generative AI (Gen AI): A type of artificial intelligence that can create new content, such as text, images, audio, and video.
• Context Engineering: The process of providing relevant and focused information to large language models (LLMs) to ensure accurate and efficient problem-solving, preventing hallucinations and high inference costs.
• Adaptive Learning: An educational method that uses AI to personalize the learning experience for each student.
• Flip Classroom: An educational model where students learn content at home and apply it in the classroom with teacher guidance.
• Teacher of Record: An accredited instructor who provides educational services to students, often when a district lacks a credentialed teacher for a specific subject.
• Machine Learning (ML): A subset of AI that enables systems to learn from data without explicit programming.
• Data Annotation: The process of labeling data to make it understandable for machine learning models.
• General AI (AGI): Artificial intelligence that possesses the ability to understand, learn, and apply knowledge across a wide range of tasks at a human-like level.

Newbird AI: Agentic AI for IT Operations

The Problem: Increasing Complexity in IT Operations

The digital landscape is becoming increasingly complex, with multi-cloud setups and a constant demand for faster, better software and features. This complexity creates numerous layers of interacting technology, making IT operations a significant challenge. The example of delivering a TikTok video illustrates this: it involves media storage, quality control, data integrity checks, distribution networks, and edge caching, all requiring sophisticated software and infrastructure. Outages, like the recent AWS incident, highlight the critical importance of reliable IT operations.

The Solution: Agentic AI for IT Ops

Newbird AI is addressing this challenge by applying modern AI, specifically Generative AI, to IT operations. They are building what they call an "Agentic AI SR" (Site Reliability Engineering) solution.

• What is an Agentic System? An agentic system leverages Gen AI for specific task domains. In IT ops, it means using AI to analyze complex data sources like logs, metrics, traces, and alerts.
• Hawkeye: The AI IT Ops Engineer: Newbird's product, Hawkeye, acts as an AI IT Ops engineer. It uses large language models (LLMs) to extract reasoning capabilities and interface with IT operational data to solve issues much faster than humans.
• Distinction between IT Ops and SRE: Site Reliability Engineering (SRE) originated at Google for large enterprises, focusing on maintaining site operations and responding to alerts (e.g., PagerDuty). In mid-market companies, IT Ops and DevOps roles often blend. Hawkeye aims to serve both use cases, reducing the burden on engineers who often perform double duty.
• Engineer vs. Agent: Newbird calls Hawkeye an "engineer" because it autonomously identifies issues, diagnoses them, and creates remediation solutions. It can also be interacted with for "what-if" scenarios. This is distinct from "co-pilots" (assistants) or single-shot "agents."
• Core Functionality: Hawkeye autonomously and asynchronously identifies IT operational issues and proposes remediation. It aims to streamline IT operations by handling a significant percentage of alerts (e.g., 80%) and automatically resolving them.
• Use Case Example: Cloud Spend Monitoring: A simple use case involves Hawkeye monitoring daily cloud spend. If it projects to be more than 5% greater than the previous day's baseline, it identifies the cause (e.g., increased instance usage, storage consumption, or egress traffic). This task, while seemingly simple, requires significant manual effort from human engineers.
• Context Engineering is Key: A core problem Newbird solves is the cost and accuracy of using LLMs. They specialize in "context engineering," which involves providing the LLM with the right data (alerts, logs, metrics, traces) to diagnose problems accurately and efficiently, avoiding high inference costs and "garbage out" answers (hallucinations). This is analogous to a patient preparing for a doctor's visit by isolating their primary issue.
• Plug-and-Play Integration: Hawkeye is designed to be plug-and-play, integrating with various systems like Elasticsearch, Prometheus, PagerDuty, MongoDB, Redis, and Snowflake. It doesn't require company-specific training, as foundational LLMs have encountered numerous IT scenarios.
• Advanced Reasoning: Newbird employs methodologies where models "argue" with each other, cross-check findings, and consult knowledge bases to ensure the reasoning is sound and based on past occurrences.
• Performance Metrics: Since its general availability a year ago, Hawkeye has achieved over 90% reduction in the meantime to incident response and resolution, significantly improving operational efficiency.
• Remediation Capabilities: Hawkeye can both recommend remediation steps (e.g., code changes, corrective actions) and, depending on customer comfort levels, autonomously make changes (e.g., enabling/disabling feature flags). This increasing autonomy signifies growing base model intelligence and sophisticated applied AI.
• Proactive Scanning: Hawkeye supports three phases: retrospective analysis (diagnosing past issues), real-time analysis (monitoring current operations), and trend analysis (predicting future issues based on current progress). It can also assess if an environment is ready for planned software rollouts.

Subject AI: Personalized Education with AI

The Evolution of Subject AI

Subject AI began as a direct-to-consumer platform offering premium, cinematic-quality educational videos, inspired by the Netflix model. However, with the advent of AI, they have pivoted to providing more personalized tools for both students and teachers.

• Mission: To amplify teachers and coaches by giving them more time for one-on-one student interaction, while AI handles mundane tasks like grading and lesson planning.
• Target Audience: Primarily K-12 education, with a focus on empowering teachers and providing equitable access to quality education, especially in underserved communities.
• Adaptive Learning and Flip Classroom: Subject AI implements adaptive learning and the flip classroom model. Students engage with personalized AI-driven content asynchronously, freeing up teachers for in-person small group or one-on-one instruction.
• Engagement Metrics: Subject AI boasts high student engagement, with 96% of students completing at least one course. Students spend an average of three hours daily on the platform.
• AI-Powered Tools for Teachers:
  • Grading and Feedback: AI handles grading of open-form essay responses and provides instantaneous feedback, saving teachers significant time.
  • Lesson Planning: AI assists with lesson planning, generating state-specific suggestions, scoring, and worksheets.
  • Progress Monitoring: A teacher console provides a quick overview of student progress, highlighting those not passing and enabling timely intervention.
• Content Format: Videos are short-form (around 5 minutes or less), catering to the attention spans of younger generations. They also incorporate AI-powered video games to enhance engagement.
• Underlying Models: Subject AI leverages various LLMs, including Claude, ChatGPT, and Gemini, with the ability to swap them out as needed.
• Teacher of Record Service: Subject AI is accredited by WASC and Cognia, allowing them to act as "Teacher of Record." This means they can provide credentialed instruction for districts that may lack qualified teachers, particularly for specialized courses like AP classes or STEM subjects in rural areas.
• Academic Integrity: The platform emphasizes academic integrity, aiming to ensure students are genuinely learning and prepared for college or the workforce, rather than simply passing courses. They are developing a career technical education product for 2026.
• Sales Strategy: While initially direct-to-consumer, Subject AI pivoted to a B2B model selling into school districts. This offers a stickier business model and broader impact, reaching students in diverse socioeconomic backgrounds. Sales cycles have been reduced to under six months after an initial learning period.
• Company Culture: Subject AI prioritizes in-person collaboration, requiring employees to work in their Los Angeles or Austin offices five days a week, fostering a "championship sports team mentality." They are actively hiring top-tier engineers and data professionals.

Flashback: Alexander Wang on Scale AI (November 2019)

This segment revisits a 2019 interview with Alexander Wang, then CEO of Scale AI, discussing the AI market before the widespread adoption of tools like ChatGPT.

Early Career and Scale AI's Mission

• Prodigy Founder: Alexander Wang, at 22 years old, had already raised over $100 million for Scale AI. He started coding at 19 and attended MIT briefly before founding Scale. His parents are physicists who worked at Los Alamos National Lab.
• Data as the Bottleneck: Wang identified data as the primary bottleneck for machine learning. Scale AI acts as a "data refinery," processing raw data from customers to train AI models.
• Autonomous Vehicles and Data Annotation: A significant focus was on autonomous vehicles. Scale AI annotates data (e.g., identifying cars, people, lane markers in images and videos) to teach machine learning models. This process involves both machine processing and human correction to ensure accuracy and minimize bias.
• Prediction on Self-Driving Cars: Wang predicted that self-driving cars would be common in major cities by 2030 or shortly after. He believed regulation would not be a significant hurdle, drawing parallels to the adoption of autopilot in airplanes.

The Future of AI and Human Labor

• Elevating Human Work: Wang envisioned AI as a tool to help humans focus on higher-value work. He used the example of truck driving, suggesting AI would automate long-haul, monotonous routes, allowing human drivers to focus on more complex, local deliveries.
• Economic Efficiency: AI's introduction would improve economic efficiency by addressing existing shortages (like truck drivers) and making current jobs more impactful.

Dangers and Regulation of AI

• Skepticism of Sci-Fi Narratives: Wang expressed skepticism towards the common narrative of AI taking over the world. He argued that AI errors would likely stem from programming flaws or overlooked edge cases, rather than malicious intent.
• Oversight is Crucial: He emphasized the importance of oversight for AI systems, drawing a parallel to the regulation of airplane autopilots.
• Lack of Current AI Regulation: At the time of the interview, Wang noted a significant lack of AI regulation in the US, contrasting it with China's approach. He suggested that governing bodies would eventually need to understand the technology and code to regulate it effectively.
• EU's ADAS Regulation: The discussion touched upon the EU's regulations for Advanced Driver-Assistance Systems (ADAS), requiring manufacturers to validate system performance with large datasets and pass trials.
• Skepticism of AGI Timelines: Wang was skeptical of rapid timelines for Artificial General Intelligence (AGI). He argued that Moore's Law was slowing down and that simply having more compute power wouldn't guarantee AGI. He proposed simulating evolution as a more plausible, albeit distant, path to AGI.
• Compute vs. Intelligence: He believed that while compute power aids narrow AI, it's not the sole determinant of general intelligence. The focus for the next few years in "applied AI" would be on building the context and support systems around existing LLMs.
• Proactive vs. Reactive AI: Newbird's Hawkeye demonstrates a shift towards proactive AI, not just reacting to issues but also predicting potential problems and assessing readiness for changes.

Conclusion and Synthesis

The YouTube transcript highlights two key areas of AI application: IT Operations and Education. Newbird AI is leveraging agentic AI to tackle the growing complexity of IT infrastructure, aiming to automate troubleshooting and remediation, thereby reducing operational burdens and costs. Subject AI is transforming education by using AI to personalize learning experiences, empower teachers, and improve student outcomes, particularly in underserved areas.

The flashback segment with Alexander Wang provides a valuable historical perspective on the evolution of AI. It underscores the foundational importance of data in AI development and the early focus on applications like autonomous vehicles. Wang's insights on the potential of AI to augment human capabilities, rather than solely replace jobs, and his cautious optimism regarding AGI and regulation, remain relevant today. The discussion also reveals a shift in the AI discourse from existential risks to more practical economic and implementation concerns. The overarching theme is the continuous innovation and application of AI across diverse sectors, driven by the need for efficiency, personalization, and problem-solving.