How Agentic AI Calls Tools | Why LLMs Don’t Act Alone

Key Concepts

• Agentic AI: LLMs making decisions and calling tools in addition to writing responses.
• Tools: APIs or Python code called by the agentic AI system to perform specific tasks.
• LLM Tool Calling: The process by which an LLM requests an external system to execute a tool and then incorporates the result into its response.
• Human-in-the-Loop: The involvement of a human to review and approve actions before a tool is executed.

Agentic AI and the Role of LLMs

The core of agentic AI lies in augmenting the traditional capabilities of Large Language Models (LLMs) beyond simply generating text. Historically, LLMs were primarily used for writing responses to prompts, such as answering questions or creating content. Agentic AI expands the LLM's role by adding two crucial responsibilities:

1. Decision-Making: LLMs now determine the next course of action, introducing non-deterministic workflows.
2. Tool Calling: LLMs can request the execution of external tools (APIs, code) to interact with the real world.

Tool calling enables LLMs to perform actions beyond generating text, such as retrieving data, scheduling appointments, or modifying external systems.

How LLMs Call Tools: A Detailed Explanation

The video addresses the common misconception of how LLMs interact with external tools. It clarifies that LLMs do not directly call tools themselves. Instead, they request that an external system (the agentic AI system) call the tool on their behalf.

Illustrative Example: Employee Information Retrieval

Consider a system designed to provide employees with HR-related information. A user might ask, "What is Alice Jones's email address?" The system architecture involves:

• User Interface: Where the user submits their query.
• Application (Tenant/Data Center): The system responsible for processing the request.
• LLM Service (e.g., OpenAI): The language model used for understanding the query and generating responses.
• Employee Lookup API (Tool): An API that retrieves employee information based on the employee's name.

The Actual Workflow

1. Prompt Construction: The application constructs a prompt containing the user's question and a description of available tools (in this case, the Employee Lookup API). The tool description includes the input parameter (employee name) and the expected output (employee information).
2. LLM Request: The prompt is sent to the LLM.
3. Tool Call Request: The LLM analyzes the prompt and determines that it needs to use the Employee Lookup API to answer the question. It responds with a request to call the tool, specifying the tool name and the required parameter (e.g., "Alice Jones").
4. Tool Execution: The application (agentic AI system) receives the LLM's request, parses it, and then directly calls the Employee Lookup API with the provided parameter.
5. Response Transmission: The application receives the response from the Employee Lookup API (e.g., Alice Jones's email address) and sends this response back to the LLM. The application typically does not modify or interpret the response.
6. Final Response Generation: The LLM receives the tool's response and uses it to formulate a final answer to the user's question (e.g., "Alice Jones's email is [email protected]").

Key Difference: The LLM never directly interacts with the Employee Lookup API. It only requests the application to do so.

Reasons for Indirect Tool Calling

The video highlights two primary reasons why LLMs do not directly call tools:

1. Security and Access Control: LLMs are often hosted on external services (e.g., OpenAI, Google Cloud), outside of an organization's firewall. Internal tools, such as proprietary databases and APIs, are typically protected by the firewall and are inaccessible to external services.
2. Human-in-the-Loop Requirement: For tools that perform more sensitive or impactful actions (e.g., sending emails, scheduling appointments, processing orders), it is often necessary to have a human review and approve the action before it is executed. This allows for error correction, validation, and adherence to organizational policies.

Conclusion

LLMs in agentic AI systems do not directly call tools. They request the agentic AI system to call the tool, providing the necessary parameters. This indirect approach is driven by security considerations and the need for human oversight in certain scenarios. The agentic AI system acts as an intermediary, executing the tool and relaying the results back to the LLM for final response generation. The next video will explore a more complex case to further illustrate this process.