samnissen / awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

Awesome Web Scraping

The list of tools, programming libraries and web services used in web scraping.

Web scraping chats: @grablab (English) and @grablab_ru (Russian)

Programming Libraries

• Python
• PHP
• Ruby
• JavaScript
• Go

Other Things

• Web Scraping Services
• Console tools
• dhamaniasad / HeadlessBrowsers - a list of (almost) all headless web browsers in existence
• lorien / awesome-python-dev - a list of tools for debugging, profiling and analyzing python programs.

Captcha Solving Services

• 2captcha.com - 2captcha.com
• anti-gate.com - anti-gate.com

Proxy Services

• proxyrack.com - Over 1,250,000 Unique Private IPs With Instant Setup
• webshare.io - 5000+ HTTP server proxies
• proxium.ru - extremely cheap small proxy lists
• scrapinghub.com/crawlera - Smart Proxy Network that automatically rotates IPs

About awesome-web-scraping

Make this list better! Your contributions are always welcome! See contributing how-to. To add new language to Programming Libraries section use new_language_template as starting point.

The list is based initially on some data from these sources awesome-python, awesome-php, awesome-ruby, ruby-nlp, awesome-javascript