Scraping products from

Scraping products from

In this project, I've built a Scraper/Bot to get equipment data from using Scrapy Web Crawling Framework.

The Problem

The client needs data from thousands of equipment of National Instruments corporation type from to perform data analysis.


Create an automated and fast solution to navigate the website, extract all the data of the National Instruments corporation type, and save it in a user-friendly format (CSV, JSON, and XML).


I've used the Scrapy Web Crawling Framework to build a bot to scrape (extract) the data of all equipment found in the National Instruments corporation type.


The client was able to quickly download the data of almost 50,000 different types of equipment from


The data was used for data analysis and add great value to the client business.

Source code

The solution is available at Github.


How to use

  • You will need Python 3.5+ to run the scripts. Python can be downloaded here.

  • You have to install the Scrapy framework and other required packages:

  • In command prompt/Terminal: pip install -r requirements.txt

  • Once you have installed the Scrapy framework, just clone/download this project:

git clone

Access the folder in command prompt/Terminal and run the following command:

scrapy crawl gsamart_nic -o equipaments.csv

You can change the output format to JSON or XML by change the output file extension (ex: equipaments.json).