The New Stack Agents - Matt White, exec director, PyTorch Foundation & GM of AI, Linux Foundation

Key Concepts

PyTorch Foundation, vendor neutrality, open source, Linux Foundation, umbrella foundation, ecosystem, large language models (LLMs), open models, adoption, governance, guardrails, AI 101, open science, model openness framework, open weights models, open source licenses, open data licenses, openMDW license, agent-to-agent (A2A), Model Context Protocol (MCP), open standards, security, privacy, controllability, observability, sovereign AI, EU AI Act.

PyTorch Foundation and its Role

• PyTorch: Developed by Meta around 2017 as a Pythonic version of Torch, it grew within Meta before becoming a Linux Foundation project around 2022.
• PyTorch Foundation: Formed to provide a vendor-neutral home for PyTorch, ensuring its longevity and fostering contributions. It became an umbrella foundation under the Linux Foundation, similar to CNCF.
• Vendor Neutrality: The foundation's vendor-neutral status provides comfort to the industry, ensuring the project's long-term viability and encouraging contributions.
• Growth: The foundation has expanded beyond the PyTorch framework to include tools like BLM (a high-performance inference engine) and DeepSpeed (an abstraction layer for training models).
• Ecosystem: The PyTorch ecosystem comprises around 70 adjacent tools that address industry-specific and general-purpose problems. Projects like BLM can graduate from the ecosystem to become foundation-hosted projects.

Linux Foundation Ecosystem and AI Stack

• Complementary Roles: The Linux Foundation ecosystem includes various projects that play complementary roles in the AI stack.
• LF AI & Data: Focuses on data and data stores (e.g., vector stores, graph databases) and the application side of the stack (e.g., Acumos).
• PyTorch Foundation: Concentrates on content ingestion, training, pre-training, post-training activities, inference optimizations, and running models on edge devices or in the cloud.
• Collaboration: The different parts of the Linux Foundation collaborate on various projects.

Challenges in Open Source Training

• Resource and Monetary Hurdles: Training large language models requires significant resources, including access to GPUs.
• Open Models: The availability of open models allows users to avoid the expense of pre-training models from scratch.
• Adoption and Governance: Organizations face challenges in adopting and governing AI, including implementing domain-specific guardrails and inspecting prompts.

Convergence of AI Research and Open Source

• AI Research: Traditionally lab-centric, AI research has rapidly become productionalized.
• Open Source: Open source principles are converging with AI research, but challenges exist due to differing understandings of AI nuances and open source licensing.
• Standards: Standards are emerging to address these challenges, along with specifications for general use cases.

Data Set Debate and Model Openness

• Model vs. Data: A key point of contention is whether releasing a model's weights and documentation is sufficient, or whether the training data should also be released.
• Open Science: In academia, open science emphasizes releasing data sets, research reports, and training/inference code for reproducibility and innovation.
• Enterprise Adoption: Enterprises may not require petabytes of training data, being content with the model itself.
• Model Openness Framework: The LF AI & Data's model openness framework addresses the completeness of models and the licensing of components.

OpenMDW License

• Need for AI-Specific License: The industry has expressed the need for a model-specific license.
• Open Weights Models: Often use community licenses with restrictions (e.g., MIT with limitations).
• OSI-Approved Licenses: Permissive (e.g., MIT, Apache 2.0) and copyleft licenses are used, but may not be appropriate for AI models.
• OpenMDW: A permissive, succinct license designed for AI models, covering trademark, patent, and database rights. It allows for any use (study, modify, redistribute) without requiring the release of all artifacts.
• Compatibility: OpenMDW is compatible with other licenses (e.g., CC BY 4.0 for data sets).
• Model Outputs: OpenMDW does not restrict model outputs, addressing the licensing ambiguity of generated content.
• Adoption: The license has been socialized with labs and industry, with over 150 attorneys reviewing it.

Agent-to-Agent (A2A) Protocol and Open Standards

• Shift to System Thinking: The focus is shifting from individual models to systems of interconnected agents.
• A2A: A protocol for agents to communicate with each other, now hosted by the Linux Foundation.
• Open Standards: Open standards are crucial for interoperability and industry-wide adoption.
• Standards as Open Source: Standards are being developed in GitHub, with community contributions and reference implementations.
• Model Context Protocol (MCP): Focuses on pulling context into a model from various systems.
• Complementary Protocols: A2A and MCP are complementary, addressing different aspects of agent communication and context retrieval.
• Extensibility: Protocols should be extensible to accommodate new features and use cases.

Exciting Developments in AI

• Open Source Models: Rapid progress in open source models, particularly in reasoning capabilities.
• Agents: Hype around agents, but need for concrete adoption and real-world use cases.
• Role Models and Embodied AI: Simulations and replicating the physical world in virtual spaces.
• AI in Products: AI is being integrated into products transparently, enhancing existing tools.

Challenges and Opportunities with Agents

• Storming Phase: The agent ecosystem is in a "storming" phase, with convergence around core projects.
• Security and Privacy: Concerns around security, privacy, controllability, and observability of agents.
• Controllability: The need for governance frameworks and controllability assurances for agents.
• Security as an Afterthought: Security is often an afterthought, creating opportunities for companies to develop solutions.

Open Source Summit in Amsterdam

• Potential Topics: Talks on agents, industry use cases, ROI, sovereign AI, and the EU AI Act.
• Diversity: The need for diversity in models, data, and culture to globalize AI.

Conclusion

The discussion highlights the rapid evolution of AI, the importance of open source and vendor neutrality, and the challenges and opportunities in areas like model licensing, agent communication, and AI governance. The PyTorch Foundation plays a crucial role in fostering collaboration and innovation within the AI ecosystem, while the Linux Foundation provides a platform for developing open standards and addressing key challenges in the field.