webscraper in nim
NOTE THAT THIS IS BETA SOFTWARE, NOT FINISHED, etc, etc
- scrapers for multiple sites
- a generic media scraper / site mapper
- a proxy scraper and checker
- a rpc
- a file indexer webapi
nim c src/aberrant.nim
nim c -d:release src/aberrant.nim
get help
aberrant.exe --help
get and check some proxies
aberrant.exe proxay
get some images/videos from some url
aberrant.exe "http://img.jpg4.net/cat/pic1.html"
create a site map
aberrant.exe mcrawl "https://angusnicneven.com/" --map --cregex=angusnicneven.com --level=20
crawl some json api
aberrant.exe jcrawl "https://api.reddit.com/r/all/new?raw_json=1&limit=100" --verbose --remove-query --include=\.mp4 --exclude=DASH_96 --keys=**/permalink --refresh=5000