Skip to main content

How to Scrape Any Website in Minutes!

www.make.com
www.make.com

Step 1: Setting up the Foundation

  • Create a new scenario on Make.com. It can be generated at make.com by selecting the “Create a scenario” button.
  • Lets add an http module in order to make requests to the website that we want to scrape.

    http Module
    http Module
  • Add an HTML to text parser module to remove HTML from the scraped data.

    HTML to Text Parser
    HTML to Text Parser
  • To parse the scraped data into individual listings use a regular expression.

    R.E: \[\/(home\/.)\]

    Text Parser (Match Pattern)
    Text Parser (Match Pattern)
  • Add one more HTTP module to make a request to individual listings. Add headers to the request so that the request appears it is coming from a real person visiting the website.

    http Module (Single Request)
    http Module (Single Request)
  • Put a sleep delay to avoid the website thinking that you are a bot.

    Sleep
    Sleep

Step 2: Extracting the Price and Price per Square Feet

  • Use regular expressions to extract the price and price per square feet from the individual listings.

    R.E: Price:\s\$.*P

    Extract Price per sqft
    Extract from Individual Listings
  • Add a text parser module to get the price and price per square feet.
    R.E: Per\sSqft:\s\$.*\[

    Getting Exact Price
    Getting Exact Price
  • Use the extracted data to calculate the price per square feet.

Step 3: Writing the Data to Google Sheets

  • Add a new module in your scenario; Google Sheets. Choose ‘Update a row’ and for the spreadsheet choose the same as before; for the sheet also choose the same.
  • Create the columns that you wish to fill with the extracted data in the table. Organize the extracted data to fill the columns.

    Writing Data to sheets
    Writing Data to sheets

That’s it! By following these steps, you should be able to create your own specific scraping automation through Make.com. Be also aware of the terms of use and robots.txt file to make sure web scraping is allowed.

0
    0
    Your Cart
    Your cart is emptyReturn to Courses