Web scraping the Bikez.com website to create a dataset of 38.000 bike models.
Bizez.com is a website that features an extensive list of bike models old and new.
For each bike this website features a variety of stats and details about different aspects and attributes of the bike.
To extract the data I used the Python programming language as well as some of its modules.
- For requests I used the Requests module.
- For the web scraping I used the Beautiful Soup module
- For cleaning and transforming the data I used the Pandas module.
- To save the raw data I used a json file. To save the cleaner data I used a csv file.
Each model is uniquely defined by the model name as well as the year of release.
The same model can be relased in multiple years, thus several model names may appear multiple times.
For many columns where I expected to get a few distinct string values, I was met instead with a variety of string values with major inconsitencies that were difficult and time consumming to standardize.
Given how most of the entries had missing values for these columns, I decided to not go through the trouble cleaning them.
On top of that, there are several columns of the data that I wasn't able to clean due to a lack of knowledge on bikes.
In short, the user may need to perform additional cleaning before performing an analysis.
The dataset can be found here: https://www.kaggle.com/victormegir/bikes-from-bikezcom
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent