I tried getting LLMs to work together using ACP (Agent Communication Protocol)

Key Concepts

• ACP (Agent Communication Protocol): A vendor and agent-agnostic framework using REST to enable communication between different agents.
• Agents: Independent software entities designed to perform specific tasks.
• LLM (Large Language Model): A deep learning model used for natural language processing tasks.
• REST (Representational State Transfer): An architectural style for building networked applications.
• CrewAI: A framework for creating autonomous agents that work together as a team.
• Small Agents: A lightweight framework for building individual agents.
• Agent Collection: A collection of agents that can be discovered and called by the ACP calling agent.
• ACP Calling Agent: An agent that automatically discovers and calls other ACP agents based on the prompt.
• Supervisory LLM Model: An LLM used by the ACP calling agent to determine which agents to call.

Wrapping an Agent with ACP (Phase One)

• Objective: To make an existing agent ACP compliant.
• Example Agent: A RAG (Retrieval-Augmented Generation) agent built using CrewAI and Qwen 2.5. This agent answers questions about hospital benefits from a PDF, specifically regarding waiting periods.
• Steps:
  1. Import Dependencies: Import necessary components from the ACP SDK, including the server.
  2. Wrap the Agent: Create a new instance of the ACP server and wrap the existing agent within it, similar to creating an MCP tool.
  3. Metadata: The function name (e.g., policy_agent) and docstring are used to generate metadata for the agent.
  4. Adapt Task Input: Modify the task input passed to CrewAI to accept input from the ACP server endpoint.
  5. Asynchronous Conversion: Convert the CrewAI kickoff to an asynchronous function that returns a generator.
  6. Start the Server: Run the server on a specified port (e.g., 8001) using uvicorn.
• Client Interaction: ACP includes its own client for interacting with ACP-compliant agents.
  1. Import Client SDK: Import the client SDK.
  2. Create Asynchronous Function: Set up an asynchronous function (e.g., run_hospital_workflow).
  3. Connect to Client: Connect to the ACP client.
  4. Pass Prompt: Pass the desired prompt to the client.
  5. Run Client: Execute the client using uvicorn.

ACP 411

• Release Date: March 2025 by IBM Research.
• Purpose: Provides a vendor and agent-agnostic framework for agent communication using REST.
• Key Features:
  • Vendor Agnostic: Works with agents from different vendors.
  • Agent Agnostic: Supports various types of agents.
  • REST-Based: Uses REST for communication.
  • Production Grade: Compatible with Kubernetes, includes telemetry, and supports identity federation.
  • Managed by the Linux Foundation: Ensures open governance and community support.
• Benefits:
  • Facilitates agent connections between teams and organizations.
  • Enables cross-industry agent interactions (e.g., insurers and hospitals, banks and card processors).

Chaining Agents (Step Two)

• Objective: To enable multiple agents to work together in a sequence.
• Example: Chaining an insurance agent with a hospital agent.
• Agents:
  • Insurance ACP Server: The previously wrapped CrewAI agent (renamed from hospital RAG).
  • Hospital ACP Server: An agent built using the Small Agents framework.
• Hospital Agent Details:
  • Framework: Small Agents.
  • Components: Code agent, DuckDuckGo search tool, light LLM model, and visit web page tool.
  • LLM: Qwen 2.5 14B running at a standard endpoint with a fake API key and a context size of 8,192.
  • Function: Handles health-based questions for patients and prospective patients.
• Steps:
  1. Create Second Server: Create a new ACP server for the hospital agent.
  2. Define Agent Logic: Implement the agent's logic using Small Agents components.
  3. Unpack Prompt: Unpack the prompt received by the default ACP endpoint.
  4. Run Agent: Execute the Small Agents agent and return a generator.
  5. Start Servers: Run both the insurance and hospital ACP servers using uvicorn.
• Chaining Implementation:
  1. Add New Client: Add a new client pointing to the hospital ACP server (port 8000).
  2. Copy Client Run: Copy the existing client run and modify it to point to the correct clients.
  3. Pass Context: Pass the context from the initial prompt to the subsequent agent calls.
  4. Run Clients: Execute the clients to trigger the chained agent workflow.

Nick's Prototype ACP Calling Agent (Step Three)

• Objective: To create an agent that automatically discovers and calls other ACP agents.
• Components:
  • Agent Collection: A collection of ACP agents that can be discovered.
  • ACP Calling Agent: The agent responsible for discovering and calling other agents.
  • Supervisory LLM Model: An LLM that guides the ACP calling agent in selecting the appropriate agents.
• Steps:
  1. Create Light LLM Model: Create a light LLM model to serve as the supervisory agent.
  2. Create Agent Collection: Create an agent collection to discover agents from the insurer and hospital ACP servers.
  3. Store Agents: Store the discovered agents and their corresponding clients in a dictionary.
  4. Create ACP Calling Agent: Instantiate the ACP calling agent, passing in the agent collection and the supervisory LLM model.
  5. Pass Concatenated Prompt: Pass a concatenated prompt that requires input from multiple agents.
  6. Run Client: Execute the client to trigger the ACP calling agent.
• Workflow:
  1. The client calls the ACP calling agent.
  2. The ACP calling agent uses the supervisory LLM to determine which agents to call based on the prompt.
  3. The ACP calling agent calls the appropriate ACP servers.
  4. The ACP servers execute their respective agents.
  5. The ACP calling agent aggregates the responses and returns them to the client.

Synthesis/Conclusion

The video demonstrates how ACP can be used to connect and orchestrate different agents, regardless of their underlying framework or vendor. It covers the process of wrapping existing agents with ACP, chaining agents together for sequential workflows, and using an ACP calling agent to automatically discover and call agents based on the prompt. The key takeaway is that ACP provides a standardized and production-ready framework for building complex agent-based systems that can span across teams and organizations.