lmammino / flickr-set-get

A simple command line app to download photos from a flickr set

flickr-set-get

Flickr-set-get is a command line application written in node that allows you to download an entire set of photos from Flickr (a gallery). Once you have it installed, to start the download you just need to:

$ flickr-set-get get <setid> <userid>

Install

As simple as installing any other global node package. Be sure to have npm and node (>= 0.8 version, or io.js >= 1.0) installed and launch:

$ npm install -g flickr-set-get

Configuration

Flickr-set-get is already configured with a default api key to allow you to use it immediately. Anyway it's highly recommended to create your own Flickr api key in the Flickr app garden especially if you intend to make authenticated requests to download private photos and photo sets. Having your own api key will allow you to avoid rate limiting and to use a more secure authenticated communication.

To configure flickr-set-get to use your own api key you need to launch the following command:

$ flickr-set-get config

An interactive guide will then help you to complete the configuration process.

Start the download

At this stage you should be ready to download a Flickr photo set. A set is identified by two parameters: a setId and a userId. Given the url of a set (gallery) you can easily spot these two parameters as shown in the image below:

So, given the url https://www.flickr.com/photos/21272841@N05/sets/72157623488969696, to start the download you need to run:

$ flickr-set-get get 72157623488969696 21272841@N05

And then just watch the command to do the hard work for you!

Download private photos and sets

Flickr supports private photos and private sets (sets that contains only private photos). To be able to download these photos from your sets you need to authenticate yourself and to send authenticated api requests. In order to do so you need to do configure your authentication options and get an authentication token with the flickr-set-get config command. Then you need to use the option --authenticated when using the flickr-set-get get command.

It's highly recommended to use your own api key and secret to achieve an optimal level of security. In fact the default api secret is shared in plain text in the code base of flickr-set-get, so it's not really a secret. Someone that might intercept a flickr-set-get authentication token for your Flickr profile (generated with the default api key and api secret) will be able to download all your private photos. Anyway it's not mandatory to provide personal api key and api secret and you can use the default values, especially if you just want to quick test the application to download some photos (ensure to be in a safe network in that case, and to revoke your authentication token at the end of the test).

Sub-commands, options and help

Flickr set get supports different sub-commands. You can access a comprehensive documentation with:

$ flickr-set-get --help

This will display all the available sub-commands and options. You can also access the specific help of a sub-command with:

$ flickr-set-get <sub-command> --help

For example flickr-set-get get --help will show:

  Usage: get|g [options] <setId> <userId>

  download a set of photos

  Options:

    -h, --help             output usage information
    --apiKey <value>       The flickr API key
    --authToken <value>    The flickr auth token
    --authenticated        Use authenticate request to access private photos (ensure to provide a valid `authToken` as option or in your config file)
    --config <value>       Define the config file to use
    -c, --concurrency <n>  The number of concurrent requests
    -o, --outputDir <s>    The directory where to save the downloaded images
    -s, --size <s>         The size of the image to download (eg. "Original", "Large", "Medium", etc.)
    -vs, --videoSize <s>   The size of the video to download (eg. "Video Original", "HD MP4", "Mobile MP4", etc.)
    -n, --noOverwrite      If set does not overwrite existing files

Current status

This project is currently on its early days so it's expected to have (a lot of) bugs and imperfections. Feel free to contribute to its development and to report bugs.

Programmatic API

Developers can integrate part of the code into their own apps (want to build a GUI for this command line app? :P) Here's a small documentation to get you going in these cases.

If you install the package as a dependency into an existing project (npm install --save flickr-set-get) you can require it's main module:

var Flickr = require('flickr-set-get');

var apiKey = 'someApiKey';
var options = {};

var client = new Flickr(apiKey, options);

Here follows a comprehensive documentation of the Flickr class.

Class: Flickr

A class that defines a set of methods to download an entire set of photos (gallery or set) from Flickr.

Available options:

• concurrency number the maximum number of concurrent http requests (default: 10)
• outputDir string the path where to save the images (default: ".")
• size string The size of the image to download, eg. "Original", "Large", "Medium", etc. (default "Original")
• noOverwrite boolean if true avoids to override already existing files (default: false)
• auth boolean|object if false it will not use authentication. Otherwise it should be an object containing the keys secret and authToken (or miniToken) (default: false)

This class extends from {EventEmitter} and emits several events:

• error Error in case of error
• setInfo Object when info about a given set are successfully retrieved
• photoSizes Object when the info about a photo (url to download the sizes) are retrieved
• photoDownloaded Object when a photo is successfully downloaded
• photoSkipped Object when a photo is skipped (already downloaded)
• done Object when all the photo of the set are downloaded

Flickr.downloadSet(setId, userId)

Starts the download of the photos from a given Flickr set. Triggers events during the whole process

Parameters

setId: string, Starts the download of the photos from a given Flickr set. Triggers events during the whole process

userId: string, Starts the download of the photos from a given Flickr set. Triggers events during the whole process

Returns: Flickr

Flickr.getAuthToken(cb)

Get the authToken. Requires that the auth option has been correctly configured providing secret and miniToken

Parameters

cb: function, a callback function that gets called (with error and data arguments) once the request is finished

Returns: Flickr

Contributing

Everyone is very welcome to contribute to this project. You can contribute just by submitting bugs or suggesting improvements by opening an issue on GitHub.

You can also submit PRs as long as you adhere with the code standards and write tests for the proposed changes.

You can read a dedicated guide on how to contribute.

License

Licensed under MIT License. © Luciano Mammino.

flickr-set-get's Issues

Read program version directly from package file

var pkg = require( path.join(__dirname, 'package.json') );

var program = require('commander');

program
    .version(pkg.version)
;

//...

Occasional UNABLE_TO_VERIFY_LEAF_SIGNATURE error

Code Coverage

Manage errors in every asynchronous part of the code

Add support to download specific Size of the photos

Improve generic code quality

Some hints received by the NodeJs community:

• Reduce the size of the Flickr class (http://blog.izs.me/post/48281998870/unix-philosophy-and-node-js)
• Flickr class methods getSetInfo and getSetInfoPage can be unified into a single method where getSetInfo is the base case that downloads the first page
• Use the EventEmitter wisely: emits new tasks when a page is loaded and start to download the images as soon as possible (avoid to block the entire flow while pages gets collected). Also see #22 .
• Use prototipe to attach functions to the Flickr class

Config issue

Hello sir, I am fairly new to computer language but this looked fun and that's why want to try it out.
I am having 2 problems.

1. How do I change the api key,secret and auth url..how do i get my own auth url.? I have my own key and secret but how do i generate authurl. I have uploaded a screenshot. is it the right way of doing it?

2. getting error on few files stating Error: Size "Original" not available. how to change size?.

I downloaded few images with the default setting and it downloaded fine but only downloaded public files.on few of them it had size errors.

Please sir, I need your help on this one.

Redesign to leverage streams and better command line composition

Some reflections:

• The entire flow of events can be re-architected using streams
• The output should be able to list all the download file to allow composition with other tools (eg. compression)

Error with flickr API: Photoset not found

hi! thanks for writing flickr-set-get! i'm getting this error when i try to download a private photo set that i have a guest pass for. is that not supported?

$ flickr-set-get 72157653461216005 49022499@N00
ERROR: Error: Error with flickr API: Photoset not found (https://api.flickr.com/services/rest?photoset_id=72157653461216005&user_id=49022499%40N00&page=1&nojsoncallback=1&format=json&api_key=80ff7d163768e25928db697db1783e35&method=flickr.photosets.getPhotos)

Default "Original" size of the photo is not available anymore

Original is used as the default label of the photo to download. Unfortunately, it looks like such a label is no longer a thing. I tried couple of albums and it usually returns:

"Square"
"Large Square"
"Thumbnail"
"Small"
"Small 320"
"Medium"
"Medium 640"
"Medium 800"
"Large"
"Large 1600"
"Large 2048"

Is there a setting an author can choose to make "Original" label show up?

I'm aware there is a -s setting and it gets the job done, but it seems having a default Original no longer cuts it, so maybe I'd be worth adding an interactive choice or making size setting required.

Switch to "commander" lib for better command line management and output

ERROR: Error: Error with flickr API: Photoset not found

I get the above error when trying to download an AutoUpload album (the only album I have). Not sure if this has anything to do with the change in URI from Set to Album in the API or whether its a special condition on the AutoUpload album.

Any help appreciated. Thanks.

Limited number of photos in larger set found

When trying to run a flickr-set-get get its only seeing 443 photos in the set, rather than the full 10,127 photos that are actually there. Any ideas where to start debugging?

INFO: Downloading 443 photos from...

Add support to skip already downloaded files

Retain original filename of photos

It would be great if the downloaded photos retain their original filenames.

Downloaded photos are watermarked

Is there any way to get a version of the photos without the Flickr watermark at the bottom?

Specify output folder as option

add -o or --outputFolder command line option to specify where to save the downloaded images

Publish to NPM

Use global apikey from user profile file

eg .flickr-set-get-rc.js

error message

Hello,

thank you for flickr-set-get.

I have an error message when trying to download this flickr https://www.flickr.com/photos/n_ipper/sets/72157624752126886

ERROR: Error: Error with flickr API: User not found (https://api.flickr.com/services/rest?photoset_id=72157624752126886&user_id=n_ipper&page=1&nojsoncallback=1&format=json&method=flickr.photosets.getPhotos&auth_token=xxx70583920373-f3f6942b1199b347&api_key=xxxxx354998d776fbaac7d87c55c0203&api_sig=xxxxa349854d47089c92364d05eba)
arnauld@arnauld-Alienware-18:~$ flickr-set-get get--authenticated 72157624752126886 n_ipper

thank you for any help,

Just the first page of the set is fetched from the API

FR: Download seach results?

It would be nice if this app can download flickr searches.

thanks

Write Readme

Increase coverage

Improve code coverage in tests

Avoid/reduce pyramid of doom where possible (use promises?)

Add JSCS

Remove semicolons

Just because everyone seems to hate them!

{
  "requireSemicolons": false,
  "disallowSemicolons": true
}

in jscs config

Use temp files until the download is complete

Support for private photo sets

As of problem reported in issue #19

Support for set url instead of two parameters (user id and set id)

Add code comments

Make tests compatible with node version >= 4

See errors here: https://travis-ci.org/lmammino/flickr-set-get/jobs/150110049

Add tests

Add ESLint support to make advanced checks on code quality

Update dependencies to solve security warnings

Make installable with NPM