Search for a command to run...
This project contains a set of scripts used to scrape Ebay's products data using Scrapy Web Crawling Framework.
I help companies to leverage Machine Learning to create innovative products through an end-to-end machine learning development process that designs, builds, and manages reproducible, testable, scalable, and evolvable ML-powered software with minimal cost.
No comments yet. Be the first to comment.
Here I show a summary description of the projects I've been working on in the last few years. Most of the projects have the source code publicly available.
In this project, I've built a Scraper/Bot to get equipment data from testmart.com using Scrapy Web Crawling Framework. The Problem The client needs data from thousands of equipment of National Instruments corporation type from testmart.com to perform...
How AI-Driven Programming is Transforming the Tech Industry and Reshaping Developer Roles
How Automated Tests Can Transform Your Nights From Stressful to Restful
Streamlining Business Communication for Enhanced Collaboration, Customer Support, Marketing, and Decision-Making.
Machine Learning Operations is a process of automating and productionalizing machine learning applications and workflows.
In this project, I've built a Computer Vision model to find basket icons in e-commerces.
This project contains a set of scripts used to scrape Ebay's products data using Scrapy Web Crawling Framework.
The client needs data (name, status, price, stars, ratings) from thousands of products of ebay.com to perform data analysis.
Create an automated and fast solution to navigate the website, find the products by a search text, extract all the data, and save it in a user-friendly format (CSV or JSON).
I've used the Scrapy Web Crawling Framework to build a Python script to search and scrape (extract) the data of products found in Etsy.
The client was able to quickly (in minutes) download the data in CSV and JSON format of more than 100,000 products from ebay.com and used this data to perform data-driven decisions.
The solution is available at Github.
In the current stage, the list of products scraped is defined by a search string (the same used on the eBay web page).
An example of the scraped data can be found in the data/ folder.
The image below shows a scraped data for the "iPhone X 256GB" search string in ebay.com
You will need Python 3.x to run the scripts. Python can be downloaded here.
You have to install scrapy framework:
pip install scrapyconda install -c conda-forge scrapyOnce you have installed Scrapy framework, just clone/download this project, access the folder in command prompt/Terminal and run the following command:
scrapy crawl ebay -o products.csv
You can change the output format to JSON or XML by change the output file extension (ex: products.json).
The default search string is Nintendo switch console and it can be changed in the command line with the -a flag. For example, to search to Xbox one X you can use:
scrapy crawl ebay -o products.csv -a search="Xbox one X"