Create Your Own Private Local AI Cloud Stack in Under 20 Minutes

Local AI Package Cloud Deployment: A Comprehensive Guide

Key Concepts:

• Local AI Package: A bundled collection of local AI services (LLMs, databases, automation tools, etc.) for easy installation and configuration.
• Cloud Deployment: Hosting the Local AI Package on a cloud instance for accessibility, resource management, and 24/7 availability.
• Digital Ocean: A cloud provider used in the video for demonstration purposes.
• Caddy: A managed HTTPS and TLS server used for secure endpoints and subdomains.
• Docker: A containerization platform used to run the Local AI Package services.
• Subdomains: Custom domains (e.g., n8n.yourdomain.com) pointing to specific services within the Local AI Package.
• Firewall Rules: Network security settings that control access to specific ports on the cloud instance.
• DNS Records: Configurations that map domain names and subdomains to IP addresses.

1. Introduction: The Need for Cloud Deployment of Local AI

• The Local AI Package offers a free and easy way to run local AI services.
• Cloud deployment addresses limitations of running local AI solely on a personal computer:
  • Accessibility for internal teams.
  • 24/7 availability.
  • Resource constraints on personal machines.
  • Access to specialized hardware (e.g., GPUs).
• Managing the cloud instance yourself still qualifies it as "local AI."
• The video provides a step-by-step guide to deploying the Local AI Package to a private cloud instance.

2. The Local AI Package: Components and Features

• The Local AI Package, originally created by the n8n team, has been extended to include:
  • n8n (automation tool)
  • Superbase (database)
  • Ollama (LLM server)
  • Open Web UI (user interface for LLMs)
  • Flowise (visual flow builder)
  • Quadrant (vector database)
  • CRX NG (local web search with Redis caching)
  • Caddy (managed HTTPS and TLS)
• Caddy enables secure endpoints for all services, allowing access via subdomains (e.g., n8n.yourdomain.com).

3. Cloud Provider Selection and Hardware Requirements

• Digital Ocean: Recommended for CPU instances due to cost-effectiveness. Used for the demonstration.
• Ami Cloud (formerly Linode): Another good option for CPU instances.
• GPU Instance Platforms:
  • Lambda Labs, Vulture, Paper Space (Digital Ocean's GPU platform), AWS, Google Cloud, Microsoft Azure.
  • Recommendation: Conduct thorough research based on specific needs and budget.
• Platforms to Avoid:
  • TensorDock: Restricts port access required for the Local AI Package.
  • RunPod, Novita: Deploy everything as containers, making it difficult to run containers within containers.
• Hardware Requirements:
  • CPU instance (8 GB RAM, 2 CPUs) is sufficient for the entire Local AI Package, unless running large LLMs.
  • Dedicated GPU is required for LLMs larger than 7-8 billion parameters (e.g., 14B, 32B, 70B).

4. Step-by-Step Deployment on Digital Ocean

1. Create a Droplet:
   • Choose a region.
   • Select an instance type (e.g., $42/month with 8 GB RAM and 2 CPUs).
   • Select an image with Docker pre-installed (Ubuntu from the marketplace).
   • Set a password for authentication (SSH keys are more secure but password is used for simplicity).
   • Set a hostname (e.g., "YouTube local AI package").
2. Access the Droplet Console:
   • Use the root user for privileged access.
3. Firewall Configuration:
   • Enable firewall rules using ufw (Uncomplicated Firewall).
   • Open ports for all Local AI services (Superbase, Flowise, Open Web UI, n8n).
   • Optional: Open ports for CRX NG and Ollama, but note that these services are not password-protected by default.
   • Run ufw reload to apply the rules.
4. DNS Record Setup:
   • Obtain the IPv4 address of the Digital Ocean droplet.
   • Create A records in your DNS provider (e.g., Hostinger).
   • Map subdomains (e.g., yt-n8n.automator.ai) to the droplet's IP address.
   • Create A records for each service (n8n, Superbase, Open Web UI, Flowise).
5. Base Installation Steps:
   • Clone the Local AI Package repository using git clone <repository_url>.
   • Change directory into the repository using cd local-ai-package.
   • Copy the .env.example file to .env using cp .env.example .env.
   • Edit the .env file using nano .env.
6. Environment Variable Configuration:
   • Set the required environment variables in the .env file.
   • n8n: Set a secure alphanumeric string for the N8N_BASIC_AUTH_PASSWORD.
   • Superbase: Set the POSTGRES_PASSWORD, login details, JWT_SECRET, ANON_KEY, and SERVICE_ROLE_KEY (refer to the Superbase self-host guide).
   • Caddy:
     • Uncomment the subdomains you configured in your DNS provider.
     • Set the LETS_ENCRYPT_EMAIL to your email address.
     • Update the hostnames to match your subdomains and domain (e.g., yt-n8n.automator.ai).
   • Save and exit the .env file using Ctrl+X, then Y, then Enter.
7. Run the Docker Compose Script:
   • Choose the appropriate command based on your hardware (CPU or GPU).
   • For CPU instances: python3 scripts/run-local-ai.py --profile cpu
   • This script pulls the necessary Docker images, starts the containers, and sets up the Docker Compose network.
8. Verification and Testing:
   • Use docker ps to verify that all containers are running.
   • Use docker logs -f <container_name> to view the logs for a specific container.
   • Access the services through the configured subdomains in your browser (e.g., yt-n8n.automator.ai).

5. Troubleshooting and Management

• If containers are down or restarting, check the logs for errors.
• The -f flag in docker logs -f <container_name> follows the logs in real-time.

6. Conclusion

• The video demonstrates a fast and efficient way to deploy the Local AI Package to the cloud.
• The Local AI Package is continuously being improved with new services and features.
• Future content will focus on creating local AI agents using the package.