- Using the Terminal or Anaconda Prompt.
- git clone this repository.
- cd billboards_crawler
- Run the following : ‘scrapy crawl billboards -a nweeks=100 -a base_date=2018-07-28 -a timedelta=7 -a save_dir='./billboard_data’
- Note: the arguments are optional, shown are the default values. Just use 'scrapy crawl billboards' to use defaults
- Check the CSV files generated in the save directory.
TODO: Handle HTTP 429 exception