Building LinkedIn's GenAI Platform — Xiaofeng Wang

LinkedIn's GenAI Platform: A Journey and Insights

Key Concepts:

• GenAI Platform
• Collaborative Articles
• Co-pilot/Coach
• Multi-Agent System (LinkedIn Hiring Assistant)
• Orchestration
• Prompt Engineering
• Skill Invocation
• Content and Memory Management
• Compound AI System
• Prompt Source of Truth
• Skill Registry
• Conversational Memory
• Experiential Memory

1. Introduction: Building LinkedIn's GenAI Platform

The talk focuses on LinkedIn's journey in building its GenAI platform, covering the "why," "how," and "what" of its development. It emphasizes the platform's critical role in the current agent-driven world and provides insights into building and hiring for such a team.

2. GenAI Product Experience: Evolution

• 2023: Collaborative Articles (GenAI Feature 1.0):
  • First formal GenAI feature, a straightforward prompt-in, string-out application.
  • Leveraged GPT-4 to create long-form articles, inviting member comments.
  • Initial components: gateway for model access, Python notebooks for prompt engineering.
  • Two separate tech stacks: Java for online serving, Python for backend.
  • Limitations: Lacked the ability to inject rich data.
• Mid-2023: Co-pilot/Coach (GenAI Feature 2.0):
  • Example: Personalized job recommendations based on profile and job description using Retrieval-Augmented Generation (RAG).
  • Platform capabilities introduced: Python SDK on top of the Langchain framework for orchestrating LLM calls.
  • Key value: Integration with large-scale infrastructure.
  • Unified tech stack (Python) to avoid costly Python-to-Java prompt translation.
  • Prompt management/source of truth module for versioning and structuring prompts.
  • Conversational memory infrastructure to track LLM interactions and inject content into the product.
• Current Year: LinkedIn Hiring Assistant (GenAI Feature 3.0):
  • First real multi-agent system, automating recruiter tasks (job posting, candidate evaluation, outreach).
  • Platform evolved into an agent platform.
  • Extended Python SDK support into a distributed agent orchestration layer (handling execution, retry logic, traffic shift).
  • Skill Registry: Centralized tool for publishing APIs as skills, enabling skill discovery and invocation.
  • Experiential Memory: Memory storage to extract, analyze, and infer knowledge from agent-user interactions, organized into working, long-term, and collective memory layers.
  • Operability: In-house solution built on OpenTelemetry (otel) for low-level telemetry data tracking, enabling agent call replay and analytics for optimization.

3. Platform Components: Four Layers

The GenAI platform is structured into four layers:

1. Orchestration: Manages the flow and execution of GenAI tasks.
2. Prompt Engineering: Tools and processes for designing and managing effective prompts.
3. Skills Invocation: Mechanisms for discovering and calling APIs as skills.
4. Content and Memory Management: Infrastructure for storing and retrieving relevant information.

The platform acts as a unified interface for the broader LinkedIn GenAI ecosystem, including modeling, responsible AI, and machine learning infrastructure teams. It allows developers to access the ecosystem without needing to understand individual components, switch between models with a single parameter change, and enforces best practices and governance.

4. Why Build a GenAI Platform?

GenAI systems differ from traditional AI systems due to the blurring of lines between model optimization and serving phases. This requires a new approach to tooling and best practices. GenAI/agent systems are considered "compound AI systems," requiring collaboration between AI and product engineers. The platform bridges the skill gap between these groups.

5. Hiring for a GenAI Platform Team

• Ideal Candidate: Strong software engineer with infrastructure integration skills, developer/PM skills, AI/data science background, and hands-on experience.
• Realistic Approach: Prioritize strong software engineering skills over AI expertise. Hire for potential rather than experience. Build a diversified team with full-stack engineers, data scientists, AI engineers, data engineers, fresh grads, and individuals with startup backgrounds. Emphasize critical thinking.

6. Key Takeaways and Lessons Learned

• Tech Stack: Python is strongly recommended due to its prevalence in research and open-source.
• Key Components:
  • Prompt Source of Truth: Robust system for version controlling prompts.
  • Memory: Critical for injecting rich data into agent experiences.
  • Skills: Uplift APIs into skills that can be easily called by agents.
• Scaling and Adoption:
  • Solve immediate needs first.
  • Focus on infrastructure and scalable solutions.
  • Prioritize developer experience and align with existing workflows.

7. Conclusion

Building a GenAI platform is a journey that requires a phased approach, a focus on developer experience, and a team with diverse skills. The platform is critical for bridging the gap between AI and product engineering, enabling the development of complex, agent-driven applications.