webscrapist Goto Github PK

daily-xkcd

Full-stack web app that lets you sign up to receive daily comics (sent through text) from xkcd! See README for in-depth description.

data_mining_system

demo-cheerio-ejs

grab web page using cheerio and render the data with ejs.

dirhunt

Find web directories without bruteforce

docsearch-scraper

DocSearch - Scraper

easy-scraping-tutorial

Simple but useful Python web scraping tutorial code.

embed

Get info from any web service or page

emhi-temperature

EMHI temperature scraper

example-scrape-websites

Examples scripts to scrape websites

facebook-page-post-scraper

Data scraper for Facebook Pages, and also code accompanying the blog post How to Scrape Data From Facebook Page Posts for Statistical Analysis

fantasy-football-projections

How accurate are fantasy football projections?

finance-scraper

Scrapes transactions from Capital One and Venmo via a chrome extension and then automatically writes that data to a personal Excel Spreadsheet

flipkart_price_tracker

This is a simple web scrapper which scrapes prices of the e-commerce site "flipkart". It sends user a message after the price reaches a certain amount set by user. User should provide the complete flipkart link for the product to be watched.

forexretailsentimentanalysis

Forex Retail Sentiment Analysis is a tool to scrape data from Myfxbook's Community Outlook and show it in a simple way.

frontend-downloader

Simple Node script to automate the login and download of course videos from http://frontendmasters.com for Offline Viewing on Desktop/Laptop.

fundscraper

Collection of web crawlers to scrape fund data using Scrapy

get-latest-episodes

Script that takes a show name as an input and opens a link to the stream of the shows latest episode

get-repo-images

An extremely fast repository crawler to find images across multiple repos

google-play-store-spider-bs4-excel

Google-Play-Store-spider use Beautiful Soup on Python to EXCEL

goose

Html Content / Article Extractor in Scala - open sourced from Gravity Labs

headless-chrome-crawler

Distributed crawler powered by Headless Chrome

heritrix3

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

image-crawler

A polite image crawler that can thumbnail and extract metadata from images at scale

insta-quotes-generator

This is a simple script used to scrape a random quote from internet and Create An Instagram Format Post For The Same.

instagram-profilecrawl

📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.

intro_to_webscraping

Code for my "Introduction to Web Scraping" talk at Capital One | Programmatically Finding Arbitrage Opportunities Between Amazon and eBay

javaspotlightscraper

A small java application to scrape Windows Spotlight images

js-crawler

A short and simple python crawler, that uses Webkit and executes Javascript

kickscraper

API library for Kickstarter.com

kinstagram