infinitay / shoe-scrapper

Retrieves relevant data of a shoe such as shoe name, gender, sizes, colorway, and much more from Stadium Goods given the model, or sku, of a shoe.

Shoe Scrapper

Retrieves relevant data of a shoe such as shoe name, gender, sizes, colorway, and much more from Stadium Goods given the model, or sku, of a shoe.

Example · Report An Issue · Request Feature

Table of Contents

• About the Project
  • Built With
• Getting Started
  • Prerequisites
  • Installation
• Usage
• Roadmap
• Contributing
• License
• Contact
• Acknowledgements

About The Project

This package was made when I was looking for a way to grab the UPC-A of shoes and I came across Stadium Goods's website as they also had the GTIN-12 of shoes. As far as I know, both UPC-A and GTIN-12 are identical in that UPC-A is just a different representation of a GTIN-12. In addition to retrieving the GTIN-12 of a shoe, because this package pulls the data from Stadium Goods, there is additional information such as market data for the shoe and other general information of the product.

Built With

• @rane/web-auto-extractor
• axios

Getting Started

To get a local copy up and running follow these simple steps.

Installation

1. Clone the shoe-scrapper

git clone https://github.com/infinitay/shoe-scrapper.git

2. Install NPM packages

npm install

Usage

Fetch a product's data by its sku, or model, and print it out.

const skuScrapper = require("./index.js");

const sku = "CU0449-601";

(async () => {
  console.log(JSON.stringify(await skuScrapper.getProductData(sku), null, 4));
})();

Fetch a product's data by its sku, or model, filters out sizes that don't have a UPC, and returns an array of objects { size: String, upc: String }

const skuScrapper = require("./index.js");

const sku = "CU0449-601";

// Retrieve all sizes and its respective upc-a/gtin-12
skuScrapper.getProductData(sku).then((productData) => {
  const transmuted = productData.marketData.offers
    .filter((offer) => !!offer["gtin12"])
    .map((offer) => {
      return { size: offer.size, upc: offer["gtin12"] };
    });
  console.log(transmuted);
});

Roadmap

• Help contribute to The Sneaker Database in order to limit and possibly avoid using Stadium Goods in return for an API intended for fetching data.
• Use this project to help gather GTIN-12s and start a database of my own to use in a future app and hopefully use that to crowd-source future GTIN-12 collection.

Potential Additions

• Provide methods to allow users to easily retrieve sizings and GTIN-12s rather than parsing it themselves.
• Provide a uniform parsing system to avoid inconsistencies and make sizing more clear on some trading platforms such as StockX.
  • Currently, AJ1 Retros on StockX such as the Royal Toe (555088-041) list gs sizes (3.5 to 7) under their adult/mens shoe. I would like to make the product data returned more clear which sizing it refers to. * Note however, there are still some inconsistencies even on Stadium Goods such as on the AJ1 Retro Satin Snakes PS (CU0449-601) where it has both toddler and preschool sizings.

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Twitter: @CallMeInfinitay

Project Link: https://github.com/infinitay/shoe-scrapper

Acknowledgements

• Stadium Goods - Without them this wouldn't be possible, so thank you.

Disclaimer

Since this depends on scrapping Stadium Goods, if you, Stadium Goods, has any issues with this, please let me know and I will take it down. For everyone else, I would like to request of you to not flood their site with requests and try to limit the scraping you do to not negatively impact their site and therefore anyone elses experience. Unfortunately they are the only ones that publicly provide GTIN-12s and it's best to avoid forcing them to remove it due to people mass scrapping. That's why in the future I'd like to use this project to help The Sneaker Database as I describe in the roadmap. That way, people won't have to depend on this package entirely, and instead call The Sneaker Database's API.