Automate Competitor Research Tool Using Make.com, Airtable, and Scrape Ninja

Want to speed up your competitive research so you can keep an eye more competitors with ease? One way to achieve this is by automating the research process using a powerful combination of tools like Make.com, Airtable, Scrape Ninja, and AI models like ChatGPT and Claude. In this guide, I’ll show you how to build a competitor research tool from scratch using these platforms.

Step 1: Setting Up Your Workflow

To get started, you’ll need accounts for Make.com, Airtable, Scrape Ninja, and access to AI models like ChatGPT or Claude. Here’s a breakdown of how each tool contributes to the process:

- Make.com: Manages the workflow automation.

- Airtable: Stores the competitor sitemap and summarised data.

- Scrape Ninja: Extracts content from competitor websites.

- ChatGPT/Claude: Processes and summarises the scraped content.

Once you have these ready, the first step is to pull the competitor’s sitemap. For this example, I’m using a local electrical company, Emergency Electrical, to pull their sitemap and extract the relevant URLs.

Step 2: Extracting Competitor Pages

The next phase is integrating **Scrape Ninja** into the workflow to retrieve the competitor’s page URLs. Using Make.com, you can set up a "watch records" module to monitor new entries in your Airtable. Each new competitor sitemap entry triggers Scrape Ninja to extract the subpages, pulling information such as:

- Subpage name

- Content summary

- Page URL

This data is then stored in Airtable, creating a comprehensive overview of your competitor's site structure.

Step 3: Cleaning and Organising Data with ChatGPT

Once the pages are scraped, the content needs to be cleaned up before processing. This is where ChatGPT comes into play. It helps summarise the content from Scrape Ninja by cleaning and structuring the text into digestible chunks, making it easier to process later. You can automate this step in Make.com by linking the cleaned data back to Airtable, where each page's name, URL, and content summary are saved.

Step 4: Summarising Competitor Insights with Claude

Here comes the fun part—analysing your competitor’s data. While ChatGPT helps with initial content summaries, **Claude** excels at providing human-like insights. In this workflow, Claude is used to summarise the entire competitor’s website, focusing on key areas like:

- Strengths and weaknesses

- Unique selling points

- Target audience

- Pricing strategy

- Marketing approach

Claude processes this information and converts it into a Markdown file, which can be easily pasted into a Google Doc for future reference.

Step 5: Automating the Workflow

Once the competitor data has been processed, the final stage involves outputting the results. The data can be formatted into a competitive research document, summarising all key findings in one place. You can also set up alerts via Google Chat or email to notify you when a new competitor analysis is complete.

Step 6: Managing Large Websites

A crucial point to keep in mind is that larger websites might require additional filtering to avoid overwhelming the system. By tweaking the queries in Make.com, you can instruct ChatGPT or Claude to prioritise specific URLs that are most relevant to your analysis, saving time and reducing API costs.

Conclusion

Building a competitor research tool using Make.com, Airtable, Scrape Ninja, and AI models is an excellent way to automate and streamline your market analysis. Whether you’re tracking a small local business or a larger competitor, this workflow helps you stay ahead with minimal effort. You can even download this template from Gumroad to tweak it for your own needs.

Feel free to build upon this, customise the inputs, and enjoy automating your competitive research!