The BEST Local AI Setup – Run Everything FREE (LLMs + n8n + MCP, No Code)

Key Concepts

• Self-hosted AI
• Open-source AI tools
• Docker Desktop
• NN (Nadlan)
• Quadrant Vector Database
• Ollama (for running local LLMs)
• mCP (Modular Computation Protocol)
• AI Agents
• Docker Compose
• Community Nodes

Self-Hosted AI Starter Kit Installation

1. Accessing the GitHub Repository:
   • Navigate to the self-hosted AI starter kit GitHub repository: github.com/nn-/selfhosted-ai-starter-kit.
   • Alternatively, join the Nadlan community and find the link there.
2. Cloning the Repository:
   • Open a terminal (e.g., Mac Terminal).
   • Use the git clone command to download the repository: git clone https://github.com/nn-/selfhosted-ai-starter-kit.
3. Navigating to the Cloned Directory:
   • Use the cd command to enter the newly created directory: cd selfhosted-ai-starter-kit.
4. Important: Modifying docker-compose.yml for mCP:
   • Open the docker-compose.yml file in a text editor (e.g., Visual Studio Code, Cursor).
   • Crucial Step: Replace the existing nadlan section with the updated version from the Nadlan community resources. This includes:
     • nadlan_community_packages_allow_tool_use=true to enable tool usage for mCP.
     • A volumes section (specific details not provided in the transcript but essential for mCP functionality).
   • Save the modified docker-compose.yml file.
5. Running Docker Compose:
   • In the terminal, execute the following command: docker compose --profile cpu up.
   • This command uses Docker Compose to build and run the services defined in the docker-compose.yml file, utilizing the CPU profile.
6. Monitoring the Process:
   • Open the Docker Desktop application to monitor the download and setup process.
   • Images for Anan, AMA, PostgreSQL, and Quadrant will be downloaded.
   • Containers will be created and started.
   • The "selfhosted-ai-starter-kit" container should appear and run.

Nadlan (NN) Setup and mCP Integration

1. Accessing Nadlan Editor:
   • Once the Docker containers are running, access the Nadlan editor in your web browser at localhost:5678.
   • If it's your first time, you'll need to sign up for an account.
2. Installing the mCP Community Node:
   • In the Nadlan editor, go to "Settings" (three dots next to your name) -> "Community Nodes."
   • Click "Install" and enter the name of the Nadlan mCP Community node (the exact name is provided in the Nadlan community resources).
   • Confirm the installation by checking the "I understand the risk" box.
3. Verifying mCP Installation:
   • Create a new blank workflow.
   • Click the "+" button to add a node.
   • Search for "mCP."
   • The "mCP Client" node should appear with a community node icon (a small box). If not, refresh the page.

Ollama Model Management

1. Accessing Ollama:
   • Ollama is running within the Docker container.
   • The default model is Llama 3.2 latest.
2. Downloading Additional Models:
   • Go to the Ollama homepage (accessed through the Ollama container in Docker Desktop).
   • Browse the available models.
   • To download a model (e.g., Gemma 3 1B), open the Ollama container's terminal in Docker Desktop.
   • Use the olama pull <model_name> command (e.g., olama pull Gemma-3-1b).
3. Verifying Model Download:
   • After the download completes, the new model should appear in the Nadlan AI Agent's model selection dropdown.

Quadrant Vector Database Setup

1. Accessing Quadrant Dashboard:
   • Quadrant is running within the Docker container on port 6333.
   • Access the Quadrant dashboard in your web browser at localhost:6333/dashboard.
   • No login is required.
2. Adding Documents to Quadrant:
   • In Nadlan, create a workflow with the following nodes:
     • "Trigger on Chat" (or any other trigger).
     • "Quadrant Vector Store - Add Documents."
     • "Default Document Loader" (or any other document loader).
     • "Text Splitter" (e.g., Token Splitter with chunk size 500 and overlap 50).
     • "Embedding Model" (requires downloading an embedding model first).
3. Downloading an Embedding Model:
   • In the Ollama container's terminal in Docker Desktop, use the olama pull <embedding_model_name> command (e.g., olama pull snowflake-arctic-embed).
4. Configuring Quadrant Vector Store Node:
   • Select the downloaded embedding model in the "Embedding Model" node.
   • Set the "Connection" to "ID" and paste the session ID from the Nadlan editor.
5. Testing the Vector Database:
   • Upload a document (e.g., a resume) using the "Allow File Upload" option in the chat.
   • Ask a question related to the document.
   • Verify that the document is vectorized and added to the Quadrant collection.

mCP Configuration and Usage

1. Adding mCP Credentials:
   • In Nadlan, add an "AI Agent" node.
   • Select a chat model (e.g., Llama 3.2).
   • Add an "mCP Client Tool" node.
   • Configure the mCP credentials based on the desired mCP server (e.g., Airbnb, Brave Search).
2. Airbnb mCP Configuration:
   • Command: npx (followed by the specific command from the Airbnb mCP documentation).
   • Argument: A long string (obtained from the Airbnb mCP documentation).
   • No environment variables are needed for Airbnb.
3. Brave Search mCP Configuration:
   • Command: npx (followed by the specific command from the Brave Search mCP documentation).
   • Environment Variable: BRAVE_API_KEY=<your_api_key>.
4. Using mCP Tools:
   • Add an "mCP Execute Tool" node.
   • Set the "Operation" to "execute_tool."
   • Set the "Tool Name" to the desired tool (e.g., "airbnb_search").
   • Set the "Tool Parameters" to be defined automatically by the model (using the three-star icon).
5. Testing mCP:
   • Ask the AI agent what tools it has access to.
   • Ask the AI agent to perform a task using a specific mCP tool (e.g., search for Airbnb listings).

Conclusion

The video provides a detailed walkthrough of setting up a local, open-source AI environment using Docker, Nadlan, Ollama, Quadrant, and mCP. It emphasizes the importance of modifying the docker-compose.yml file for mCP integration and demonstrates how to download and manage models with Ollama, configure and use Quadrant as a vector database, and integrate mCP tools for enhanced AI agent capabilities. The key takeaway is the ability to run powerful AI tools and agents locally, offering privacy and control over your data and computations.