Build an AI Influencer Clone in WhatsApp in 30 Min | No-Code

Key Concepts

AI Persona Agent, WhatsApp Automation, Voice Cloning, 11 Labs, Make.com, OpenAI Assistant, Knowledge Base, WhatsApp Business API, Meta Business Verification, AI Voice Messages, No-Code Automation, AI Agents, AI Automations, Voice Cloning, WhatsApp Business API Setup.

AI Persona Agent on WhatsApp: Voice Message Automation

This video demonstrates how to build an AI Persona agent that responds to WhatsApp messages with voice messages in the user's voice. The agent is trained on a knowledge base and can answer questions related to that knowledge. The presenter, Ben, showcases a demo of his own AI clone on WhatsApp and provides a step-by-step guide on how to replicate the setup.

Demo and Use Cases

Ben demonstrates his AI clone by sending a voice message asking about his work. The AI transcribes the message, generates a response based on its knowledge base (transcripts of Ben's YouTube videos), converts the text response into a voice message using 11 Labs (voice cloning), and sends it back via WhatsApp.

Potential use cases include:

• Influencers engaging with fans in a personalized way.
• Founders reaching out to customers for support or marketing.
• Brands creating personalized customer experiences.

System Architecture and Workflow (Make.com)

The entire system is built on Make.com due to its native integrations with WhatsApp, 11 Labs, and OpenAI. The workflow is as follows:

1. WhatsApp Trigger: A "Watch Events" module in Make.com monitors a WhatsApp Business number for new messages. A webhook is set up in Meta to send new message data to Make.com.
2. Router: The workflow splits based on message type (text or audio).
   • Text Messages: The text is sent directly to the OpenAI Assistant.
   • Voice Messages: The audio is downloaded, transcribed to text using OpenAI's Whisper API, and then sent to the OpenAI Assistant.
3. OpenAI Assistant (AI Agent): The transcribed text is sent to an OpenAI Assistant, which generates a response based on its knowledge base.
4. Source Removal: A "Create a completion" module removes any source citations from the response (e.g., "Source: File 2") to ensure a natural-sounding voice message.
5. 11 Labs (Voice Cloning): The cleaned text response is sent to 11 Labs, which generates an audio file (MP3) in the user's cloned voice.
6. WhatsApp Media Upload: The generated MP3 is uploaded to the WhatsApp Business Cloud API.
7. Send WhatsApp Message: The uploaded media ID is used to send a voice message back to the user via the WhatsApp Business Cloud API.

Setting up the AI Agent (OpenAI Assistant)

1. Access OpenAI Platform: Navigate to platform.openai.com and log in.
2. Create an Assistant: Go to the "Assistants" tab and create a new assistant.
3. Define System Prompt: Provide a system prompt that defines the AI's role and personality (e.g., "You are Ben AI, a knowledgeable AI YouTuber...").
4. Add Knowledge Base: Enable "File Search" to allow the assistant to access a knowledge base.
5. Create Vector Store: In the "Storage" tab, upload files (e.g., transcripts of YouTube videos) to create a vector store. The files are vectorized for efficient knowledge retrieval.
6. Connect Vector Store to Assistant: Link the created vector store to the assistant.

Automating Knowledge Base Updates

The video explains an automation to automatically transcribe YouTube videos and add them to the OpenAI knowledge base:

1. Google Sheets: A Google Sheet contains a list of YouTube video links.
2. Dumpling AI: The Dumpling AI module extracts the transcript from each YouTube video.
3. Create File: A file is created from the YouTube transcript.
4. Download File: The file is downloaded.
5. Upload File to OpenAI: The file is uploaded to the OpenAI account using the "Upload a File" module.
6. Add File to Vector Store: The uploaded file is added to the vector store, updating the knowledge base.

Voice Cloning with 11 Labs

1. 11 Labs Account: Create an account on 11 Labs. A paid plan (Creator) is recommended for professional voice cloning.
2. Add New Voice: Go to "Voices" and click "Add a new voice."
3. Choose Voice Cloning Method: Select either "Instant Voice Clone" (available on the Starter plan) or "Professional Voice Clone" (requires the Creator plan). The Professional Voice Clone requires at least 30 minutes of high-quality audio.
4. Upload Audio Samples: Upload audio samples of your voice. Ensure the audio is high quality with minimal background noise.
5. Generate Voice Clone: 11 Labs will generate a voice clone based on the provided audio. The Professional Voice Clone takes longer (1-2 hours) as it fine-tunes a model.
6. Connect 11 Labs to Make.com: In Make.com, use the 11 Labs module ("Create a speech synthesis"). Create a connection using your 11 Labs API key (found in your 11 Labs account). Select your cloned voice from the list.

Setting up the WhatsApp Business API

1. Meta Developer Account: Create a Meta developer account at developers.facebook.com.
2. Create an App: Create a new app and select "Other" as the app type and "Business" as the use case.
3. Add WhatsApp Product: Add the WhatsApp product to your app.
4. Set up Business Portfolio: If you don't have one already, set up a Facebook Business Portfolio. The name of the portfolio is important as it can be used as your WhatsApp display name.
5. Add Phone Number: Add a phone number to your WhatsApp Business account. Do not use your personal WhatsApp number. Use a separate SIM card or online number.
6. Configure Webhook: In the app's configuration settings, set up the webhook URL. This is the webhook URL generated by the "Watch Events" module in Make.com.
7. Verify Token: Set a verify token in both Meta and Make.com.
8. Subscribe to Webhook Fields: Subscribe to the "messages" webhook field.
9. Generate Permanent Access Token: To get a permanent access token (which doesn't expire after 24 hours), you need to verify your business with Meta.
10. Verify Business: Go to "Business Info" in your Meta app and fill out all the details of your business. Then, click "View Details" and "Verify." You will need to provide documentation to prove your business legitimacy.
11. System Users: Add a system user with admin privileges.
12. Assign Assets: Assign the app you created to the system user and grant full control.
13. Generate Token: Generate a new token, selecting the app and setting the expiration to "Never." Select the "WhatsApp Business Management" and "WhatsApp Business Messaging" permissions.
14. Use Token in Make.com: Use the generated token in the Make.com WhatsApp module to create a connection.

Conclusion

The video provides a detailed guide on building an AI Persona agent that can respond to WhatsApp messages with voice messages in the user's voice. The system leverages Make.com for automation, OpenAI for AI processing and knowledge management, and 11 Labs for voice cloning. Setting up the WhatsApp Business API requires business verification with Meta to obtain a permanent access token. The presenter emphasizes the potential of this technology for personalized communication and customer engagement.