Coding Local AI Agent in Python (Ollama + LangGraph)

Key Concepts

• AI Agent: An autonomous system that completes tasks involving multiple steps, dynamically using various tools.
• Tool Calling: The ability of an LLM to utilize external functions (tools) to gather information or perform actions.
• Langchain: A framework for building applications powered by language models.
• Langraph: A library for creating robust and modular conversational AI agents using graphs.
• Ollama: A tool for running language models locally on your system.
• State Graph: A representation of the agent's workflow, consisting of nodes (representing actions or decisions) and edges (representing transitions between nodes).
• Nodes and Edges: Fundamental components of a Langraph graph, where nodes represent processing steps and edges define the flow of execution.
• Router (Conditional Edge): A decision point in the graph that determines the next node to visit based on the current state.
• Typed Dictionary: A Python type that allows specifying the types of keys and values in a dictionary, used here for defining the agent's state.

Building a Local AI Agent in Python

Introduction

The video demonstrates how to build a local AI agent in Python using Langchain, Langraph, and Ollama. The agent can autonomously complete tasks by combining multiple tools dynamically. The key advantage is running everything locally without relying on external APIs like OpenAI or Google.

Prerequisites

• A GPU with sufficient VRAM (at least 8GB recommended) to run a language model. The presenter uses an Nvidia GeForce RTX 3060Ti with 8GB VRAM.
• Ollama installed and running with a chosen language model (e.g., Quen 3 with 8 billion parameters).
• A mail account with IMAP access enabled, along with the IMAP host, email address, and password.

Setup and Installation

1. Create a Project Directory: Create an empty directory for the project.

2. Install Dependencies: Use uv (a Rust-based package manager) or pip to install the necessary packages:

   • langchain
   • langchain-olama
   • langraph
   • python-dotenv
   • imap-tools

3. Configure Environment Variables: Create a .env file to store sensitive information like IMAP host, username, and password. Example:

   IMAP_HOST=mail.your-server.de
   IMAP_USER=tutorial@neuralnineonline.com
   IMAP_PASSWORD=your_password
   

4. Ollama Setup:

   • Install Ollama from olama.com.
   • Pull a suitable language model using olama pull <model_identifier> (e.g., olama pull quen3:8b).
   • Ensure Ollama is running using olama serve.

Code Implementation

1. Imports and Constants

import os
import json
from typing import TypedDict
from dotenv import load_dotenv
from imap_tools import MailBox, AND
from langchain.chat_models import ChatOllama
from langchain_core.tools import tool
from langgraph.prebuilt import ToolNode
from langgraph.graph import StateGraph, END

• Import necessary modules for environment variables, JSON handling, type hinting, IMAP access, Langchain, and Langraph.
• Load environment variables from the .env file using load_dotenv().
• Define constants for IMAP host, user, password, inbox folder, and the chosen language model (chat_model = "quen3").

2. Chat State Definition

class ChatState(TypedDict):
    messages: list

• Define a ChatState class as a typed dictionary to represent the agent's state, which primarily consists of the message history.

3. Mailbox Connection Function

def connect():
    mailbox = MailBox(IMAP_HOST)
    mailbox.login(IMAP_USER, IMAP_PASSWORD, initial_folder=IMAP_FOLDER)
    return mailbox

• Create a connect() function to establish a connection to the mail server using the imap-tools library. It takes no arguments and returns a MailBox object.

4. Tool Definitions

• list_unread_emails() Tool:

  @tool
  def list_unread_emails():
      """Return a bullet list of every unread message's subject, date, sender, and UID."""
      print("List unread emails tool called")
      with connect() as mb:
          unread = mb.fetch(AND(seen=False), headers_only=True, mark_seen=False)
          if not unread:
              return "You have no unread messages."
          response = json.dumps([
              {
                  "UID": mail.uid,
                  "subject": mail.subject,
                  "date": mail.date.astimezone().strftime("%Y-%m-%d %H:%M"),
                  "sender": mail.from_
              } for mail in unread
          ])
          return response
  

  • Uses the @tool decorator to expose the function as a tool to the LLM.
  • Fetches unread emails from the inbox, extracts relevant information (UID, subject, date, sender), and returns a JSON string containing a list of email dictionaries.
  • The docstring serves as the tool's description for the LLM.

• summarize_email() Tool:

  @tool
  def summarize_email(uid: str):
      """Summarize an email given its IMAP UID. Return a short summary of the email's content/body in plain text."""
      print(f"Summarize email tool called on {uid}")
      with connect() as mb:
          mail = next(mb.fetch(uid=uid, mark_seen=False), None)
          if not mail:
              return f"Could not summarize email with UID {uid}"
          prompt = (
              f"Summarize this email concisely:\n\n"
              f"Subject: {mail.subject}\n"
              f"Sender: {mail.from_}\n"
              f"Date: {mail.date}\n\n"
              f"Content: {mail.text or mail.html}"
          )
          return raw_llm.invoke(prompt).content
  

  • Takes an email UID as input.
  • Fetches the email with the given UID.
  • Constructs a prompt containing the email's subject, sender, date, and content.
  • Invokes the raw_llm (LLM without tools) to summarize the email content and returns the summary.

5. LLM Initialization

llm = ChatOllama(model=chat_model)
llm = llm.bind_tools([list_unread_emails, summarize_email])
raw_llm = ChatOllama(model=chat_model)

• Initializes two instances of ChatOllama:
  • llm: The main LLM instance with tool access.
  • raw_llm: An LLM instance without tool access, used for tasks like summarizing email content.
• Binds the list_unread_emails and summarize_email tools to the llm instance using bind_tools().

6. Langraph Graph Definition

def llm_node(state):
    response = llm.invoke(state["messages"])
    return {"messages": state["messages"] + [response]}

def route(state):
    last_message = state["messages"][-1]
    if hasattr(last_message, "tool_calls"):
        return "tools"
    else:
        return "end"

def tools_node(state):
    result = tool_node.invoke(state)
    return {"messages": state["messages"] + result["messages"]}

tool_node = ToolNode([list_unread_emails, summarize_email])

builder = StateGraph(ChatState)
builder.add_node("llm", llm_node)
builder.add_node("tools", tools_node)

builder.add_edge("start", "llm")
builder.add_edge("tools", "llm")

builder.add_conditional_edges(
    "llm",
    route,
    {
        "tools": "tools",
        "end": END
    }
)

graph = builder.compile()

• llm_node(state): Takes the current state, invokes the llm to generate a response based on the message history, and updates the state with the new response.
• route(state): A router function that examines the last message in the state. If the message contains tool calls, it returns "tools"; otherwise, it returns "end".
• tools_node(state): Invokes the tool_node to execute a tool call based on the current state and appends the result to the message history.
• tool_node: A ToolNode instance that manages the execution of available tools.
• Graph Construction:
  • Creates a StateGraph instance with the ChatState as the state type.
  • Adds the llm_node and tools_node to the graph.
  • Defines edges connecting the nodes:
    • "start" -> "llm": The initial state transitions to the LLM node.
    • "tools" -> "llm": After a tool call, the flow returns to the LLM node.
  • Adds a conditional edge from the "llm" node to the route function, which determines whether to go to the "tools" node or the "end" state.
  • Compiles the graph using builder.compile().

7. Main Loop

if __name__ == "__main__":
    state = {"messages": []}
    print("Type an instruction or 'quit':")
    while True:
        user_message = input("> ")
        if user_message.lower() == "quit":
            break
        state["messages"].append({"role": "user", "content": user_message})
        state = graph.invoke(state)
        print(state["messages"][-1].content)

• Initializes an empty state with an empty message list.
• Enters a loop that prompts the user for input.
• Appends the user's message to the state's message history.
• Invokes the graph with the current state to execute the agent's workflow.
• Prints the content of the last message in the state (the agent's response).

Execution

Run the script using uv run main.py or python main.py. The agent will prompt for instructions. You can interact with it by typing commands like:

• "list all my unread emails"
• "summarize the content of all emails from Florian"
• "quit"

Conclusion

The video provides a detailed guide to building a local AI agent using Langchain, Langraph, and Ollama. It covers the necessary setup, code implementation, and graph definition. The agent can autonomously perform tasks by combining multiple tools dynamically, demonstrating the power and flexibility of these frameworks. The presenter emphasizes the importance of experimentation and customization to create more sophisticated and capable agents.