18520339 / image-search-downloader

Download images including URLs from Google, Bing, Flickr and Instagram hashtags with given keyword

Download Images From Search Engine

Download images including urls from Google, Bing, Flickr, Instagram with given keyword

Demo: https://www.youtube.com/watch?v=zfMBE3D9IG4

Preparation

Copy these following information to corresponding fields in the secret.ini file

1. Bing: https://azure.microsoft.com/en-us/try/cognitive-services/my-apis/?api=bing-image-search-api

2. Flickr: https://www.flickr.com/services/apps/create/noncommercial/

3. Instagram: https://www.instagram.com/explore/tags/?__a=1

• Press F12 and go to Network Panel
• In the Name column, click csp_report/ or the one that has maximum time in the Time column
• Copy value of the cookie param in Request Headers
• Note that:
  • Because of using Cookie, signing in is required
  • Instagram images will be expired after a certain period of time which can be found in the oe param of image url. For example: oe=604645EF.
  • The expired time is an Unix hexadecimal timestamp and can be convert to human-readable date

Usage

1. Google:

• Search your images and start scrolling to the end
• Copy + paste the function in the get_google_urls.js file into the console

2. Bing:

• Example: python get_bing_urls.py -o 'images' -k 'cat'

usage: get_bing_urls.py [-h] -o OUT -k KEYWORD [-p PER] [-m MAX]

optional arguments:
  -h, --help                      show this help message and exit
  -o OUT, --out OUT               path to images directory
  -k KEYWORD, --keyword KEYWORD   search query for API
  -p PER, --per PER               number of results per requests
  -m MAX, --max MAX               total results

3. Flickr:

• Example: python get_flickr_urls.py -o 'images' -k 'cat'

usage: get_flickr_urls.py [-h] -o OUT -k KEYWORD [-p PER] [-m MAX]

optional arguments:
  -h, --help                      show this help message and exit
  -o OUT, --out OUT               path to images directory
  -k KEYWORD, --keyword KEYWORD   search query for API
  -p PER, --per PER               number of results per requests
  -m MAX, --max MAX               total results

4. Instagram:

• Example: python get_instagram_urls.py -o 'images' -k 'cat'

usage: get_instagram_urls.py [-h] -o OUT -k KEYWORD [-m MAX]

optional arguments:
  -h, --help                      show this help message and exit
  -o OUT, --out OUT               path to images directory
  -k KEYWORD, --keyword KEYWORD   search query for API
  -m MAX, --max MAX               total results

5. Download images from url file:

• Example: python download_images.py -o 'images' -u 'cat.txt'

usage: download_images.py [-h] -o OUT -u URLS [-s START]

optional arguments:
  -h, --help                show this help message and exit
  -o OUT, --out OUT         path to images directory
  -u URLS, --urls URLS      path to image urls file
  -s START, --start START   start number of image name

Many thanks to

• https://www.pyimagesearch.com/2018/04/09/how-to-quickly-build-a-deep-learning-image-dataset/
• https://www.pyimagesearch.com/2017/12/04/how-to-create-a-deep-learning-dataset-using-google-images/