2+ Hour Course: Build & Sell Voice AI Agents with n8n & 11Labs (No Code)

Key Concepts

• Voice AI Agent: An AI-powered agent that interacts with users through voice, understanding and responding to spoken commands.
• 11 Labs: A platform used to build voice AI agents, offering features like voice cloning, multilingual support, and integration with external tools via webhooks.
• NADN (n8n): A workflow automation platform used as a backend to process data from and control the 11 Labs voice AI agent.
• System Prompt: Instructions given to the large language model (LLM) that define the agent's persona, behavior, and how it should respond to users.
• Knowledge Base: A collection of information (documents, URLs, text) that the voice AI agent can use to answer user questions.
• Tools: External functions or services (e.g., webhooks, Google Sheets, calendar APIs) that the voice AI agent can access to perform specific tasks.
• Webhooks: A way for 11 Labs and NADN to communicate with each other in real-time, sending data and triggering actions.
• LLM (Large Language Model): The AI model that powers the voice AI agent, responsible for understanding language and generating responses. Examples include Google's Gemini, GPT-4, and Claude.
• Query Parameters: URL parameters to pass optional data along with a URL request.
• Body Parameters: Data passed as the body of an HTTP request, typically used to send data to a server for processing.
• From AI Function: n8n’s function to dynamically map values from AI model response to other nodes, like Google Sheets.

11 Labs Voice AI Agent Masterclass: A Comprehensive Summary

Introduction to Voice AI Agents with 11 Labs and NADN

• The masterclass aims to guide users from beginner to pro in building voice AI agents.
• It utilizes 11 Labs for voice AI and NADN as the backend to process information and build agents.
• The series will cover the basics of voice AI, connecting 11 Labs to NADN, and building three different voice AI agents (beginner to advanced).

Setting Up 11 Labs

• Create an account on 11labs.io.
• Choose a pricing plan; the "Starter" or "Creator" plan is recommended for beginners due to ample credits and voice cloning capabilities, and first month is 50% off.
• Navigate to the "Agents" section after logging in.
• Click "Create an Agent" to start building a new agent.
• Name your agent. You can choose pre-built templates (e.g., support agent, math tutor) or start with a blank template for full customization.

Understanding the 11 Labs Agent Interface

Agent Tab:

• Agent Language: Select the language for the agent's interactions (supports multiple languages).
• First Message: Customize the initial message the agent speaks when a user initiates a call (e.g., "Hi, I'm Eric. How can I help you today?").
• System Prompt: Define the agent's persona, behavior, and response style. This is crucial for instructing the agent on how to interact with users and utilize available tools. Example: "You're a support agent named Eric. You're very friendly, enthusiastic, and you really want to help the customer get the help they need. Answer in three to seven sentences in most cases."
• LLM: Choose the Large Language Model (LLM) to power the agent. Google's Gemini 1.5 Flash is the fastest for real-time conversations but may not be the most robust for tool use. More powerful models like GPT-4 Turbo, mini, or Claude 2.5 U are better for complex tool use.
• Temperature: Controls the creativity or randomness of the LLM's responses.
• Limit Token Usage: Sets a limit on the number of tokens used per response. Leaving it at the standard value (-1) means no limit.
• Knowledge Base: Add data sources (files, URLs, text) for the agent to draw upon when answering questions. Supported file types: PDF, TXT, DOCS, HTML, EPUB (up to 21MB).
• Tools: Connect external functions or services via webhooks. This is key for integrating the voice AI agent with NADN and other systems.
• Secrets: Securely store API keys or other sensitive information for authentication.

Voice Tab:

• Voices: Select a pre-existing voice from 11 Labs or clone a custom voice.
• Use Flash: Enables low-latency models for faster response times.
• TTS Output Format: Choose the audio quality for text-to-speech output (e.g., 16,000 Hz for good pronunciation).
• Pronunciation: Upload lexicon dictionary files for pronunciation replacement.
• Optimization Streaming Latency: Adjust settings like stability and similarity for voice cloning.

Analysis Tab:

• Evaluation Criteria: Define metrics for evaluating the agent's performance and collect data during conversations.

Security Tab:

• Advanced features for enabling overrides (covered in later videos).

Limitations Tab:

• Set limits on conversation timeout and maximum duration.

Widget Tab:

• Customize the appearance of the embedded widget (color, font, avatar).
• Copy the code snippet to embed the agent on a website.

Building a Tech Support Agent (Simple Example)

1. Set up the Agent: Create a new agent in 11 Labs, setting the language to English.
2. Customize the First Message: Modify the initial greeting (e.g., "Hi, I'm Bill, the customer tech support agent. How can I help you today?").
3. Adjust the System Prompt: Define the agent's role and personality. For example, "You are a very professional and funny customer support agent. Customer support agent answer in three to seven sentences."
4. Add a Knowledge Base: Upload a PDF document containing FAQs related to AI workshop.
5. Modify the System Prompt to Use the Knowledge Base: Update the system prompt to instruct the agent to prioritize the knowledge base when answering questions about the AI workshop. For example: "When a customer asks about AI workshop then please only respond to the customer from your knowledge base."
6. Test the Agent: Use the "Test AI Agent" feature to interact with the agent and verify that it is behaving as expected.

Connecting to NADN via Webhooks

1. Create a NADN Workflow: Create a new workflow in NADN to handle requests from the 11 Labs voice AI agent.
2. Add a Webhook Trigger: Use the "Webhook" trigger node to listen for incoming requests from 11 Labs. Set the HTTP method to "POST" and choose "Respond to Webhook node" to allow NADN to send data back to 11 Labs. Change the path to something descriptive like “testing”.
3. Configure the 11 Labs Tool: In 11 Labs, go to the "Tools" section and add a new "Webhook" tool.
4. Name and Describe the Tool: Provide a name and description for the tool (e.g., "Test Tool," "Use this tool to call n8n").
5. Set the URL: Paste the webhook URL from NADN into the "URL" field in 11 Labs.
6. Enable Body Parameters: Enable body parameters for the webhook tool in 11 Labs. Add properties to define the data that the agent should collect from the user (e.g., "name" of type "string" with the description "Name of the customer.”).
7. Update the System Prompt: Instruct the agent to use the tool when a specific condition is met (e.g., "When a customer is asking about the weather, use the test tool.").
8. Respond to the Webhook: Back in NADN, use the "Respond to Webhook" node to send a response back to 11 Labs.
9. Test the Connection: Interact with the voice AI agent and verify that data is being sent to NADN via the webhook.

Building a Restaurant Reservation Voice AI Agent

1. Set up the Agent: Create a new agent in 11 Labs for Joe's Diner.
2. Customize the First Message: Set the initial greeting for the agent (e.g., "Hi, I'm Jackie. How can I help you today?").
3. Define the System Prompt: Provide instructions on the agent's role as a support agent for Joe's Diner (e.g., "You're a support agent named Jackie. You're very friendly and enthusiastic and really want to help the customer get the help they need. answer to three to seven sentences in most cases. You work at a restaurant called Joe's Diner.").
4. Select the LLM: Choose a suitable LLM (e.g., GPT4 or Claude 3.5).
5. Add a Knowledge Base: Upload a FAQ document for Joe's Diner and a text file containing the menu items and prices.
6. Create the Reservations Tool: Add a webhook tool in 11 Labs named "Reservations." Describe its purpose: "Use this tool to make a reservation for Joe's Diner."
7. Set Up the NADN Workflow: Create a NADN workflow with a webhook trigger to receive reservation requests. Set the path to reservations. Use the "Respond to Webhook" node to send a confirmation message back to 11 Labs.
8. Configure Body Parameters: Define the body parameters in the 11 Labs tool to collect the necessary information for a reservation (e.g., name, date, time, number of people). Set data types (e.g., string for name and date, number for number of people).
9. Add Google Sheets Integration: In NADN, add a "Google Sheets" node to append reservation data to a spreadsheet.
10. Map Data to Google Sheets: Manually map the incoming webhook data (name, date, time, number of people) to the corresponding columns in the Google Sheet.
11. Test the System: Interact with the voice AI agent and verify that reservation data is being correctly captured in the Google Sheet.

Enhancing the Reservation Agent with an AI Agent in NADN

1. Add AI Agent to NADN Workflow: Remove the direct connection from the webhook trigger to the Google Sheets node. Insert an AI Agent node in between.
2. Configure the AI Agent: Choose a chat model (e.g., OpenAI with GPT-4 mini).
3. Attach Google Sheets Tool: Connect the Google Sheets node as a tool to the AI Agent. Set “Append Row” to operation.
4. Create System Prompt: Define a system prompt for the AI Agent, instructing it to book reservations using the Google Sheets tool (e.g., "You're a customer support AI agent that works at a restaurant named Joe's Diner. Your primary role is to book reservation for Joe's Diner. You have access to multiple tools to help you fulfill this request.").
5. Map Data to Google Sheets with From AI Function: Use n8n’s from AI function to dynamically map the customer name, date, time and number of people from the AI model response to the corresponding columns in Google Sheets.
6. Stringify Response to Web Hook: For the “Response to Web Hook” set the status to “reservation is confirmed”.
7. Test and Refine: Test the entire workflow and refine the prompts to optimize performance.

Building a More Complex Voice AI Agent (AI Workshop Support Agent)

This section describes a more advanced setup, building upon the principles learned in the earlier examples.

• Goal: Create a voice AI agent for AI Workshop to provide information about the agency and school community, book consultation calls, and gather customer contact information.
• Tools: The agent utilizes multiple tools, including:
  • SER API: For general information retrieval from Google Search.
  • Appointments: For accessing the calendar.
  • Google Calendar: For booking appointments.
  • Availability: For checking available time slots.
  • Google Sheets: For storing customer contact information (name, email, phone number).
  • Vector Store Retriever Tool: To retrieve data from a Pinecone vector database containing information about the AI Workshop school community. A FAQ document is uploaded to Pinecone for this purpose.
• Query Parameters: A request query parameter with the same identifier is used across multiple tools to help the AI agent identify the purpose of each request.
• System Prompt: A detailed system prompt is used to instruct the agent on how to use each tool and handle various scenarios. The prompt includes:
  • General instructions on the agent's role and responsibilities.
  • Specific instructions on when to use each tool.
  • Instructions to use the school wisdom tool to retrieve information when customers ask about the school community.
  • Instructions to check for open time slots if a customer is leaning towards booking a consultation call.
  • Instructions to use the book meeting tool after agreeing on a time with the customer.
  • Instructions to let the customer know that they will receive an email with the details of the scheduled call after booking.
  • Instructions to apologize and suggest using the contact form if booking fails.
• User Request: Setting name, email, phone number, date and time that is coming from the parameter the voice AI agent has.

Becoming a Certified Voice AI Agent Partner

• Partner with Detail AI, a beginner-friendly voice AI platform.
• Take the exclusive course to learn how to build complex voice AI agents.
• Get certified to be listed on Retail AI's website as a certified partner.
• Access a step-by-step AI agency course to learn how to pick a niche, run discovery calls, price services, and start an AI agency.

Conclusion

The masterclass provides a comprehensive guide to building voice AI agents using 11 Labs and NADN. It covers the basics of setting up 11 Labs, connecting to NADN via webhooks, building simple and complex agents, and becoming a certified partner to sell voice AI agents to customers. Key takeaways include understanding the 11 Labs agent interface, crafting effective system prompts, and using tools to extend the agent's capabilities. The series emphasizes the importance of testing and refining prompts to optimize performance and deliver a natural and engaging user experience.