LangChain Full Crash Course - AI Agents in Python

Key Concepts

• Langchain: A Python framework for building and working with AI agents, offering an abstract way to interact with various AI models and tools.
• Langchain Ecosystem: Includes Langchain (core framework), LangGraph (low-level, event-driven agents), and LangSmith (observability, evaluation, monitoring, deployment).
• Abstracted Models: Langchain allows swapping AI model providers (OpenAI, Anthropic, Google) without changing core code due to consistent classes and methods.
• AI Agents: Systems that can use tools, reason, and take actions to accomplish tasks.
• Tools: Functions annotated with the @tool decorator that agents can use to interact with the external world or perform specific operations.
• Message History/Memory: Mechanisms to retain conversation context across multiple turns.
• Streaming Responses: Displaying model output in real-time as it's generated.
• Structured Output: Forcing models to return information in a predefined format (e.g., JSON, data classes).
• Multimodal Input: Allowing models to process and understand different types of data, such as text and images.
• Retrieval Augmented Generation (RAG): A technique that combines retrieval from a knowledge base (often using vector stores and embeddings) with generative AI to provide more informed responses.
• Vector Stores: Databases that store data as numerical vectors, enabling efficient similarity searches.
• Embedding Models: Models that convert text or other data into numerical vector representations.
• Middleware: Code that sits between a request and a response, allowing for pre-processing, post-processing, or dynamic adjustments to the agent's behavior.
• Hooks: Specific points in the agent's execution lifecycle where middleware can be injected.

Langchain Crash Course: Building AI Agents

This video provides a comprehensive crash course on Langchain, a Python framework designed to simplify the development and integration of AI agents. It covers the core concepts, practical examples, and advanced features of Langchain, focusing on version 1.0.

1. Langchain and its Ecosystem

• Core Functionality: Langchain is a Python framework for building and working with AI agents. Its primary benefit is providing an abstracted, high-level interface for interacting with various AI models and tools, regardless of the provider (e.g., OpenAI, Anthropic, Google). This abstraction allows for easy swapping of underlying technologies without significant code changes.
• Ecosystem Components:
  • Langchain: The main framework for building agents and integrating components.
  • LangGraph: A more low-level framework for building complex, graph-based, event-driven agents. (Not covered in detail in this video).
  • LangSmith: A platform for observing, evaluating, monitoring, and deploying models. (Not covered in detail in this video).
• Langchain 1.0 Changes:
  • Consolidated Package Structure: Previously, components like langchain-core, langchain-community, and provider-specific packages (e.g., langchain-openai) were separate imports. In 1.0, most functionalities are accessed through the main langchain package (e.g., from langchain import ...).
  • Agent-Centric Focus: Langchain 1.0 is more heavily focused on building agents, with features like create_agent becoming central. Previously, Langchain was seen more as a general toolkit, with LangGraph handling agent-specific logic.

2. Environment Setup and Basic Agent Creation

• Installation: The video demonstrates using uv (a Rust-based Python package manager) for installation, but pip is also mentioned. To install Langchain with provider dependencies, use syntax like uv add langchain openai or pip install langchain openai. This installs the necessary provider packages (e.g., openai).
• API Keys: API keys from providers (OpenAI, Anthropic, Mistral AI, Google) are required for authentication. These are typically stored in a .env file and loaded into the environment using python-dotenv.
• First Simple Agent Example:
  • Imports: requests for API calls, load_dotenv from dotenv, create_agent from langchain.agents, and tool decorator from langchain.tools.
  • Tool Definition: A get_weather tool is defined using the @tool decorator. It takes a city (string) as input and returns weather information by making a request to an external weather API (https://wttr.in/{city}?format=j1). The return_direct=True parameter can be used to immediately return the tool's output to the user.
  • Agent Creation: The create_agent function is used to instantiate an agent. Key parameters include:
    • model: The AI model to use (e.g., "gpt-4.1-mini"). The provider is inferred from the model name.
    • tools: A list of defined tools (e.g., [get_weather]).
    • system_prompt: Instructions for the agent's persona and behavior (e.g., "You are a helpful weather assistant who always cracks jokes...").
  • Agent Invocation: The agent is invoked using agent.invoke(), passing a dictionary with a messages key containing a list of message dictionaries (e.g., {"role": "user", "content": "What is the weather like in Vienna?"}).
  • Output: The invoke method returns a response object. The actual message content can be accessed via response.messages[-1].content.

3. Standalone Models and Message History

• Standalone Models: For simpler interactions without agents, Langchain's init_chat_model function can be used.
  • Usage: model = init_chat_model(model="gpt-4.1-mini", temperature=0.1).
  • Invocation: response = model.invoke("Hello, what is Python?").
  • Model Swapping: The model can be easily swapped by changing the model parameter (e.g., to "mistral-medium"), with the rest of the code remaining the same.
• Message History: To maintain conversation context, Langchain supports structured message history.
  • Message Classes: HumanMessage, AIMessage, and SystemMessage from langchain.messages provide a more structured way to represent conversation turns.
  • Conversation List: A list of these message objects can be passed to model.invoke() to simulate a conversation.
  • Example:

    from langchain.messages import HumanMessage, AIMessage, SystemMessage
    
    conversation = [
        SystemMessage(content="You are a helpful assistant for questions regarding programming."),
        HumanMessage(content="What is Python?"),
        AIMessage(content="Python is an interpreted programming language."),
        HumanMessage(content="When was it released?")
    ]
    response = model.invoke(conversation)
    

• Streaming Responses: To display output in real-time, use model.stream() instead of model.invoke(). This returns a generator that can be iterated over to print chunks of text as they are generated.
  • Example:

    for chunk in model.stream("What is Python?"):
        print(chunk.content, end="", flush=True)
    

4. Advanced Agent Features: Structured Output, Context, and Memory

• Structured Output: Langchain allows forcing agents to output data in a specific format using data classes.
  • Data Classes: Define Python data classes (e.g., Context, ResponseFormat) to specify the expected input and output structures.
  • response_format Parameter: Pass the data class to the create_agent function via the response_format parameter. The agent will then attempt to generate output conforming to this schema.
• Contextual Information: Agents can leverage contextual data beyond the immediate prompt.
  • context_schema Parameter: Specify a data class for context in create_agent.
  • ToolRuntime: Tools can receive context via a ToolRuntime object, which contains contextual information like user_id.
  • Example: A locate_user tool uses a ToolRuntime with a context object containing a user_id to determine the user's city.
• Memory: Agents can retain memory of past interactions.
  • checkpointer Parameter: Use a checkpointer (e.g., InMemorySaver from langchain_core.checkpoint) to store and retrieve conversation history.
  • thread_id: A thread_id is used to identify and manage specific conversation threads.
• Combined Example: An agent is created with tools for get_weather and locate_user, a context_schema for user location, a response_format for structured weather data, and checkpointer for memory. When invoked, it uses the user_id from the context to find the city, then fetches the weather, and returns a structured response.

5. Multimodal Input

• Processing Images and Text: Langchain supports models that can process both text and image inputs.
• Content Structure: The content field in a message can be a list containing multiple items, each with a type (e.g., "text", "image") and corresponding data.
• Image Input Methods:
  • URL: Provide a direct URL to an image.
  • Base64 Encoded Bytes: Load an image from disk, encode it in Base64, and pass it as a string. This requires specifying the mime_type (e.g., "image/png").
• Example: An agent is invoked with a message containing both text ("Describe the contents of this image") and an image URL or Base64 data. The model then analyzes the image and provides a textual description.

6. Retrieval Augmented Generation (RAG)

• RAG Workflow: RAG involves using a vector store and an embedding model to find relevant information from a knowledge base, which is then used by a generative model to produce a response.
• Components:
  • Embedding Model: OpenAIEmbeddings is used with a specific model identifier (e.g., "text-embedding-3-large").
  • Vector Store: FAISS (Facebook AI Similarity Search) is used as the vector store. It requires installation (uv add faiss-cpu).
  • from_texts: The vector_store.from_texts() method creates a vector store from a list of text documents, using the provided embedding model.
  • similarity_search: The vector_store.similarity_search() method finds the most similar documents to a given query.
• Creating a Retriever Tool:
  • vector_store.as_retriever(): Converts the vector store into a retriever object.
  • create_retriever_tool: From langchain_core.tools, this function turns a retriever into a tool that an agent can use.
• RAG Agent Example:
  • An agent is created with a knowledge_base_search tool (created from the retriever).
  • The system prompt instructs the agent to use this tool to retrieve context before answering questions about fruits or operating systems.
  • The agent is invoked with a query like "What three fruits does the person like and what three fruits does the person dislike?". The agent uses the retriever tool to find relevant information and then generates a concise answer.

7. Middleware in Langchain

• Purpose: Middleware intercepts requests and responses, allowing for dynamic adjustments to the agent's behavior. This includes changing prompts, models, summarizing conversations, rate limiting, and more.
• Dynamic Prompt Middleware:
  • @dynamic_prompt Decorator: This decorator from langchain.agents.middleware allows defining functions that dynamically generate system prompts based on request context.
  • Contextual Prompts: The example shows how to adjust the system prompt based on a user_role (e.g., "expert", "beginner", "child") provided in the request context.
• Dynamic Model Selection Middleware:
  • @wrap_model_call Decorator: This decorator from langchain.agents.middleware allows intercepting model calls.
  • Conditional Model Switching: The example demonstrates switching between a "basic" and an "advanced" model based on the number of messages in the conversation history.
• Custom Middleware with Hooks:
  • AgentMiddleware Class: Developers can create custom middleware by inheriting from AgentMiddleware and overriding specific hook methods (e.g., before_agent, before_model, after_model, after_agent).
  • Timing and Logging: The example shows how to measure the execution time of different stages of the agent's workflow.
• Pre-built Middleware Examples:
  • Summarization Middleware: Automatically summarizes long conversations to manage token limits.
  • Human-in-the-Loop Middleware: Pauses execution for manual approval at critical steps (e.g., sending emails, payments).
  • Model/Tool Call Limits: Enforces limits on how often models or tools can be called.
  • Model Fallback: Switches to a backup model if the primary one fails.
  • PII Detection: Redacts personally identifiable information for compliance.
  • Planning Middleware: Enables agents to create and manage to-do lists.

Conclusion

Langchain provides a powerful and flexible framework for building sophisticated AI agents. Its abstraction layer simplifies integration with various models and tools, while features like structured output, memory, multimodal input, RAG, and middleware enable the creation of highly capable and dynamic AI systems. The recent 1.0 release further streamlines development with a more consolidated package structure and a stronger focus on agent creation.