Level Up Your AI Agents with Fine-Tuning (n8n)

Key Concepts

Fine-tuning, Prompt Engineering, Retrieval-Augmented Generation (RAG), Supervised Fine-tuning, Direct Preference Optimization, Reinforcement Fine-tuning, Embedding Models, Full Fine-tuning, Parameter Efficient Fine-tuning (PEFT), LoRA (Low Rank Adaptation), OpenAI Playground, JSONL, Hyperparameters (Batch Size, Learning Rate Multiplier, Epochs), AI Agents, LLM Chain, Vector Store, Tool Calling, Tone of Voice, Style, Format, Scalable System, Air Table, Google Sheets, Webhooks, N8N, OpenAI API.

Fine-tuning for AI Optimization

Introduction to Fine-tuning

Fine-tuning is a core method for optimizing AI responses by creating specialized models that influence the style, tone, structure, and format of the AI's output. It's not primarily for teaching new information, which is better suited for RAG systems.

Fine-tuning vs. Prompt Engineering and RAG

• Prompt Engineering: Crafting high-quality prompts to guide AI responses.
• RAG (Retrieval-Augmented Generation): Providing the AI with access to vast amounts of information.
• Fine-tuning: Creating a specialized model to influence the style, tone, structure, and format of the AI response.

Use Cases for Fine-tuning

• Matching writing style and tone by providing numerous examples.
• Enforcing specific jargon, terminology, or formats in industries like legal documentation.
• Optimizing embedding models for RAG applications.
• Using cheaper models for high-volume, task-specific workflows to reduce latency and cost.

Limitations of Fine-tuning

• Not suitable for teaching the AI new knowledge (RAG is better).
• Not ideal for infrequent model use or generalized applications across diverse domains.
• Requires high-quality, consistent, and varied training data.

Types and Strategies of Fine-tuning

• Full Fine-tuning: Updates all model parameters (GPU intensive).
• Parameter Efficient Fine-tuning (PEFT): Updates only a small amount of model parameters.
  • LoRA (Low Rank Adaptation): A common example of PEFT.

Models Available for Fine-tuning

• OpenAI models (via self-serve API or custom models program).
• Claude 3 Haiku (via Amazon Bedrock).
• Certain Gemini models (via Vertex AI).
• Open-source models (using tools like Axelottle or Lamaactory).

Supervised Fine-tuning Method

The video focuses on supervised fine-tuning, where the model is trained using simple user prompts and corresponding desired responses. Other methods include:

• Direct Preference Optimization: Providing both correct and incorrect answers to steer the model.
• Reinforcement Fine-tuning: Used for reasoning models.
• Vision Fine-tuning: Training the model using images.

Creating a Fine-tuned Model via OpenAI Playground

Step-by-Step Process

1. Access OpenAI Dashboard: Navigate to the fine-tuning section.
2. Create New Job: Select a base model (e.g., GPT-3.5 Turbo).
3. Add Suffix: Provide a descriptive name for the model.
4. Set Seed: Enter a number for reproducibility across jobs.
5. Add Training Data:
   • Create a JSONL file containing sample prompts and desired responses.
   • Use ChatGPT to convert data into the correct JSONL format.
   • Upload the file to OpenAI.
6. Validation Data (Optional): Upload a separate JSONL file to test the model's output.
7. Hyperparameters: OpenAI automatically sets hyperparameters like batch size, learning rate multiplier, and epochs.
8. Create: Initiate the fine-tuning process.

Hyperparameter Explanation

• Batch Size: Number of training examples considered in each training chunk.
• Learning Rate Multiplier: Determines how quickly the model learns. A high number can lead to poor generalization.
• Epochs: Number of full cycles through the entire dataset.

Integrating the Fine-tuned Model into an N8N Agent

1. Create AI Agent in N8N: Use an AI agent node with a chat trigger and simple memory.
2. Select OpenAI Chat Model: Choose the OpenAI chat model node.
3. Enter Fine-tuned Model Name: Replace the base model with the unique model name obtained after fine-tuning.
4. Test the Workflow: Send a message to the agent and verify the response.
5. System Message: Provide the same system message used in the training data.

Mitigating Incorrect Responses

• Provide more varied training data to handle edge cases.
• Implement a RAG system to ground the AI in correct information.

Data Preparation Guidelines

• Provide as many high-quality examples as possible.
• Ensure examples are consistent and non-contradictory.
• Prompt in a similar format to how the model will be used in practice.
• Use at least 10 examples, with 50-100 recommended for better results.
• Avoid overfitting by providing too few examples or losing context with too many.

Scalable Fine-tuning System Using Air Table and N8N

System Overview

A scalable system is created using Air Table and N8N to automate the fine-tuning process. Google Sheets containing training data are automatically processed and used to create fine-tuned models.

Step-by-Step Process

1. Google Drive Integration: A Google Drive trigger monitors a folder for new Google Sheets.
2. Air Table Record Creation: When a new sheet is detected, an Air Table record is created with the file name, status, base model, and Google Sheet URL.
3. Status Update: The status is changed to "ready for processing" to trigger the next flow.
4. N8N Workflow Execution: An N8N workflow is triggered to:
   • Get the Air Table record.
   • Get the data from the Google Sheet.
   • Transform the data into the correct JSONL format.
   • Upload the file to OpenAI.
   • Trigger the fine-tuning job via the OpenAI API.
5. Polling for Results: A separate N8N workflow polls the OpenAI API for the status of the fine-tuning job.
6. Air Table Update: Once the job is completed, the Air Table record is updated with the fine-tuned model name.

N8N Workflow Details

• Webhook Trigger: A webhook node in N8N is triggered by the Air Table sync button.
• Search Air Table Node: Searches for records with the status "ready for processing."
• Loop Over Items Node: Processes each Air Table record individually.
• Read Sheet Node: Reads data from the Google Sheet.
• Code Nodes:
  • Transforms the data into JSONL format.
  • Transforms the JSONL into a binary format for API upload.
• HTTP Request Node: Sends a POST request to the OpenAI API to trigger the fine-tuning job.
• Separate Workflow for Polling: A separate workflow checks the job status and updates the Air Table record accordingly.

Monitoring Workflow

• A separate workflow is triggered to poll the OpenAI service for results.
• The workflow checks the job status every minute until a maximum timeout of 90 minutes.
• The Air Table record is updated with the status (succeeded, failed, canceled, or timed out) and the fine-tuned model name.

Advanced Use Cases and Best Practices

Email Newsletter Generation

Fine-tuning can be used to generate email newsletters in a specific tone and format. The fine-tuned model is used within an LLM chain to generate the email content, which is then passed to the ConvertKit API.

Integrating Fine-tuning with AI Agents and RAG

• Limitation of Fine-tuning AI Agents Directly: AI agents need to be able to generalize and call tools, which can be difficult to achieve with fine-tuning alone.
• Recommended Approach: Use a base model for the AI agent and fine-tune the output after the agent has retrieved information from a vector store or other external tool.
• Step-by-Step Process:
  1. The AI agent receives a chat message.
  2. The agent retrieves information from a vector store.
  3. The retrieved information and the original question are passed to an LLM chain.
  4. The LLM chain uses the fine-tuned model to generate the final response.
  5. The response is sent back to the user.

Prompt Engineering Best Practices

• Keep the pattern of prompts as similar as possible to what will be used in practice.
• Include relevant context in the prompts.

Conclusion

Fine-tuning is a powerful technique for optimizing AI responses by influencing the style, tone, and format of the output. It is best used in conjunction with prompt engineering and RAG systems. By following the best practices outlined in this video, you can create specialized AI models that are more reliable, aligned with your brand, and optimized for specific use cases.