khanshaheb34 / prothomaloscraper

A python script to scrap articles from Prothom Alo with the Headline, Category, URL, and Summary

ProthomAloScraper

A python script to scrape all article with title, category, summary (if available) and URL from Prothom Alo

To get help run: python scraper.py -h or python scraper.py --help

Documentation:

usage: python scraper.py [-h] [--write-interval WRITE_INTERVAL]
                         [--verbose VERBOSE]
                         start end filename

Scrape articles from Prothom Alo

positional arguments:
  start                 Starting article ID from Prothom Alo
  end                   Ending article ID from Prothom Alo
  filename              Filename for the csv file

optional arguments:
  -h, --help            show this help message and exit
  --write-interval WRITE_INTERVAL
                        Write to file after every interval (default 25)
  --verbose VERBOSE     0 = False, 1 = True (default 1)

Give an starting id, ending id and a filename, the script will scrape all the articles from given ID and save them to filename.csv