From Prompt to Voice-Narrated Story — Fully Automated with GPT-4o & n8n (FREE Template)

Key Concepts

GPT4 image model, Runway ML (image to video animation), 11 Labs (text-to-speech), Niten workflow, AI-generated short stories, social media automation, prompt engineering, API integration, image generation, video generation, voice narration.

Creating Animated Short Story Videos with AI

Overview

The video demonstrates how to create animated short story videos using AI tools: GPT4 for image generation, Runway ML for animation, and 11 Labs for voice narration. The process is automated using a Niten workflow, and a free template is provided for users to implement the solution. The generated content can be used for social media or personal use.

Workflow Template and Access

• A free Niten workflow template is available for download from the AI Workshop Light community (link in video description).
• The template can be imported into a Niten instance to automate the video creation process.
• A paid AI Workshop community offers a more detailed, step-by-step guide with links to resources, code, and prompts.
• The paid community also provides access to daily calls for support and collaboration with other members.

Example and Use Cases

• The video showcases examples of generated images and animations, including portraits for social media, anime-style characters, and realistic faces.
• Use cases include creating short, narrated stories for social media platforms like TikTok and Instagram.
• The workflow can be adapted for various themes, such as motivational videos or fictional narratives.

Step-by-Step Process

1. Chat Input: A simple prompt (e.g., "a busy street market on a sunny afternoon") is entered into the Niten workflow.
2. Prompt Generation (OpenAI): The simple prompt is fed into an OpenAI model (e.g., GPT-3.5 Turbo or GPT-4) with a detailed prompt instructing it to transform the simple topic into a detailed prompt suitable for image generation, video animation, and voice narration. The detailed prompt includes examples to guide the AI.
   • Example prompt: "Under a bright golden afternoon sun, a bustling market comes alive where vibrant stalls overflow with colorful fruits..."
3. Image Generation (GPT4 Image API): The detailed prompt is sent to the GPT4 image generation API to create an image.
   • The API endpoint is https://api.openai.com/v1/images/generations.
   • A POST request is sent with the API key in the authorization header.
   • The request body includes the model, prompt, and image size (e.g., 1024x536).
   • The API returns a base64 string representing the generated image.
4. Video Animation (Runway ML): The base64 image and the detailed prompt are sent to Runway ML's image-to-video API to animate the image.
   • The API endpoint is https://api.runwayml.com/v1/video/.
   • A POST request is sent with the API key in the authorization header.
   • The request body includes the image data (base64 string), prompt text, duration (e.g., 5 seconds), and ratio.
   • A "wait" node is added to pause the workflow for 60-80 seconds to allow Runway ML to process the request.
5. Video Retrieval (Runway ML): A GET request is sent to Runway ML to retrieve the generated video using the job ID obtained from the previous step.
   • The API endpoint is https://api.runwayml.com/v1/video/{job_id}.
   • The request includes the API key and the RunwayML-API-Version header set to 2024-11-16.
   • The API returns a video link if the job is successful.
6. Voice Narration (11 Labs): The detailed prompt is sent to 11 Labs' text-to-speech API to generate a voice narration.
   • The API endpoint is https://api.elevenlabs.io/v1/text-to-speech/{voice_id}.
   • A POST request is sent with the API key in the xi-api-key header.
   • The request body includes the text prompt.
   • The API returns an audio file containing the voice narration.
7. Email Notification: The video link and audio file are sent to the user's email address.
8. Manual Combination (iMovie or similar): The generated video and audio files are manually combined using video editing software like iMovie.
9. Social Media Posting (Blato API): The combined video can be posted to various social media platforms using the Blato API.
   • Blato allows posting to Facebook, Instagram, YouTube, TikTok, and other platforms with a single API call.

Technical Details

• GPT4 Image API: Requires an API key for authentication. Supports different image sizes.
• Runway ML: Requires an API key and a paid account. Charges per second of video generation (approximately $0.25 per 5 seconds). Accepts base64 image strings.
• 11 Labs: Requires an API key. Supports multiple voice IDs and custom voice cloning.
• Niten: A workflow automation platform that allows connecting different APIs and services.
• Base64: A binary-to-text encoding scheme used to represent images as strings.
• JSON: A lightweight data-interchange format used for sending data between APIs.

Key Arguments and Perspectives

• Automation vs. Manual Work: The video argues that while most of the process can be automated, manually combining the video and audio files in a video editor is more efficient than trying to automate that step.
• Prompt Engineering: The quality of the generated content depends heavily on the quality of the prompt. The video emphasizes the importance of creating detailed and specific prompts.

Notable Quotes

• "Just because you can automate something doesn't mean you always should." (Regarding the manual video editing step)

Conclusion

The video provides a practical guide to creating AI-generated animated short stories using a combination of GPT4, Runway ML, and 11 Labs. By leveraging a Niten workflow and the provided template, users can automate most of the content creation process and generate engaging videos for social media or personal use. The video also highlights the importance of prompt engineering and the strategic use of automation to maximize efficiency.