mtdukes / custom-scrapers

A collection of custom scrapers with substandard documentation.

#Custom scrapers Here are some scrapers I made. Mostly in Python.

##Scraper template A template to quickly build scrapers and store information. Accepts a csv filename and a url to scrape. More improvement TK. Will eventually output some json. Copy it and build your own scraper.

Usage (for testing):

cp scraper-template.py your-new-scraper.py

python your-new-scraper.py test_csv.csv http://google.com

##Law enforcement personnel (v2) A scraper to collect data on law enforcement personnel from the N.C. State Bureau of Investigation's Uniform Crime Reporting Program, located here.

Usage:

python lea-personnel_v2.py

##Law enforcement assaults A scraper to collect data on assaults on law enforcement from the N.C. State Bureau of Investigation's Uniform Crime Reporting Program, located here.

Usage:

python lea-assaults.py

##Mental health beds A scraper to collect data on available mental health beds from the N.C. Division of Mental Health, Developmental Disabilities and Substance Abuse Services' Bed availablility inventory.

Usage:

python track_beds.py

##Commerce press releases A scraper to collect data from N.C. Department of Commerce press releases.

Usage:

python commerce-releases.py