How I Built a Competitor Analysis AI System in n8n (No-Code)

Key Concepts

• Competitor Research: Tracking competitor job postings to gain insights into their strategic direction.
• Productized AI System: A ready-to-use AI system that can be easily implemented across different companies and industries.
• Air Table: Used as a front-end interface for data management and visualization.
• NAN (n8n): Used as the back-end automation platform for data scraping, processing, and report generation.
• Uni API (UniPow): An API used for accessing and interacting with LinkedIn data, including scraping job postings.
• LinkedIn Voyager API: LinkedIn's internal GraphQL API used to extract job posting data.
• LLM (Large Language Model): Used to analyze job posting data and generate competitor strategy reports.

Competitor Research System Overview

The video demonstrates a system built to perform automated competitor research by tracking LinkedIn job advertisements. The system scrapes job postings, categorizes them by department, and generates a weekly competitor strategy report. The system uses Air Table as a front end and NAN as the back end. The template is designed to be easily copied and pasted for use in various companies and industries without customization.

Demo of the System

1. Adding Competitors:
   • Competitors are added to Air Table with their LinkedIn URLs.
   • Example: Adding "Relevance AI" with its LinkedIn URL.
2. Initial Research:
   • Clicking "research" triggers the initial research process.
   • The system finds the company ID, creates a Google Drive folder, retrieves the company website, logo, and a brief description.
3. Automated Job Posting Tracking:
   • A scheduled trigger (weekly) initiates the job scraping process.
   • The system loops through each competitor, checks for new job postings, categorizes each job posting by department, saves the job posting to a Google Docs file in a dedicated Google Drive folder, and updates the Air Table database.
4. Report Generation:
   • After scraping and categorizing job postings, the system generates a competitor insights report.
   • The report is sent via email and updated in Air Table.
5. Report Content:
   • The report includes strategic insights for each competitor, such as:
     • Vapy: Focused on building out their core product platform and sales/customer support functions.
     • Relevance AI: Strategy centered around category creation and driving adoption through hands-on customer implementation.
     • NAN (n8n): Transitioning from product-led growth to a dedicated, regionally-focused sales engine.

Step-by-Step Breakdown of the NAN Workflow

1. Triggering the Automation:
   • The automation is triggered by a web hook in Air Table when a new company's status changes to "research".
   • Air Table automation runs a script to send the record ID to the NAN web hook URL.
2. Retrieving Company Details:
   • The NAN workflow receives the record ID and uses an Air Table node to retrieve the company's details (name, URL, status).
3. Cleaning the URL and Adding Company Name:
   • A code step cleans the LinkedIn URL and extracts the company name for use in subsequent steps.
4. Retrieving Company ID using Uni API:
   • The Uni API is used to retrieve the company ID from LinkedIn.
   • The API endpoint used is "retrieve a company profile".
   • Requires the company name, subdomain, port, and API key.
5. Google Drive Folder Management:
   • The system searches for an existing Google Drive folder for the competitor.
   • If the folder doesn't exist, it creates a new one.
6. Updating Air Table:
   • The Air Table is updated with the company ID, description, website, logo, and Google Drive folder ID.
7. Scheduled Job Posting Tracking:
   • A scheduled trigger (weekly) initiates the job scraping process.
   • A search air table node filters for companies with the status "add to schedule".
8. Looping Through Competitors:
   • A loop iterates through each competitor to scrape job postings.
9. Scraping Job Postings using Uni API and LinkedIn Voyager API:
   • The Uni API is used with the "get raw data from any endpoint" to scrape job postings from LinkedIn.
   • This involves using LinkedIn's internal Voyager API (GraphQL).
   • The request URL is obtained from the Voyager API by inspecting the page source of a LinkedIn job post.
   • The company ID is passed as a parameter to retrieve job postings for each competitor.
10. Data Cleaning and Transformation:
    • The scraped data is cleaned and transformed to extract relevant information (title, description, applicants, location, etc.).
11. Categorizing Job Postings by Department:
    • An LLM is used to categorize each job posting by department based on the job description.
    • A prompt is used to classify the job into predefined departments.
12. Creating Google Docs for Job Listings:
    • A Google Docs file is created for each job posting in the competitor's Google Drive folder.
    • The job description is appended to the Google Docs file.
13. Updating Air Table with Job Posting Data:
    • The Air Table is updated with the job posting data, including the title, description, applicants, location, file URL, department, and a link to the original company record.
14. Report Generation:
    • The scraped job postings are aggregated into a single JSON object.
    • An LLM is used to generate a competitor strategy report based on the aggregated job posting data.
    • A prompt is used to guide the LLM in analyzing the data and generating insights.
15. Emailing the Report:
    • The generated report is converted to HTML and sent via email.
16. Updating Air Table with the Report:
    • The Air Table is updated with the generated report.

Technical Details and API Usage

• Uni API (UniPow): Used for LinkedIn automation, including retrieving company profiles and scraping job postings. Requires an account and API key.
• LinkedIn Voyager API: LinkedIn's internal GraphQL API used to extract job posting data. Accessed through the Uni API's "get raw data from any endpoint" feature.
• Air Table API: Used to create, read, update, and delete records in Air Table.
• Google Drive API: Used to create folders and store job posting documents.
• LLM (Large Language Model): Used for job posting categorization and report generation. Google Gemini 2.5 Flash is used in the example.

Key Arguments and Perspectives

• Job Ads as Strategic Indicators: Job advertisements provide valuable insights into a company's strategic future moves and departmental priorities.
• Automation for Efficiency: Automating competitor research saves time and provides consistent, up-to-date insights.
• Productization for Scalability: The system is designed to be easily productized and deployed across different companies and industries.

Notable Quotes

• "The reason why tracking job ads is such a good way to get competitor insights is because job ads tell you something about the strategic future moves of companies."
• "If you want you can copy and paste my template inside of my community."
• "The UniL API is is a game changer and I highly recommend trying it out because you can do a lot with it."

Conclusion

The video provides a detailed walkthrough of an automated competitor research system that leverages Air Table, NAN, and the Uni API to scrape, categorize, and analyze LinkedIn job postings. The system generates weekly competitor strategy reports, providing valuable insights into their strategic direction and departmental priorities. The productized nature of the system allows for easy implementation across various companies and industries, making it a valuable tool for staying ahead of the competition.