thumbsup / downsize

Convert / resize / transcode / down-sample photos & videos to be web-friendly

thumbsup

---

https://thumbsup.github.io

---

Turn any folder with photos & videos into a web gallery.

• thumbnails & multiple resolutions for fast previews
• mobile friendly website with customisable themes
• only rebuilds changed files: it's fast!
• uses relative paths so you can deploy the pages anywhere
• works great with Amazon S3 for static hosting

Quick start

Simply point thumbsup to a folder with photos & videos. All nested folders become separate albums.

npm install -g thumbsup
thumbsup --input ./photos --output ./gallery

There are many command line arguments to customise the output. See the website for the full documentation: https://thumbsup.github.io.

Sample gallery

See a sample gallery online at https://thumbsup.github.io/demos/themes/mosaic/

Requirements

Thumbsup requires the following dependencies:

• Node.js: brew install node
• exiftool: brew install exiftool
• GraphicsMagick: brew install graphicsmagick

And optionally:

• FFmpeg to process videos: brew install ffmpeg
• Gifsicle to process animated GIFs: brew install gifsicle
• dcraw to process RAW photos: brew install dcraw
• ImageMagick for HEIC support (needs to be compiled with --with-heic)

You can run thumbsup as a Docker container (ghcr.io/thumbsup/thumbsup) which pre-packages all the dependencies above. Read the thumbsup on Docker documentation for more detail.

docker run -v `pwd`:/work ghcr.io/thumbsup/thumbsup [...]

Command line arguments

This reflects the CLI for the latest code on master. For the latest published version please refer to the docs on the website.

Usages:
  thumbsup [required] [options]
  thumbsup --config config.json


Required:
  --input   Path to the folder with all photos/videos  [string] [required]
  --output  Output path for the static website  [string] [required]

Input options:
  --scan-mode           How files are indexed  [choices: "full", "partial", "incremental"] [default: "full"]
  --include-photos      Include photos in the gallery  [boolean] [default: true]
  --include-videos      Include videos in the gallery  [boolean] [default: true]
  --include-raw-photos  Include raw photos in the gallery  [boolean] [default: false]
  --include             Glob pattern of files to include  [array]
  --exclude             Glob pattern of files to exclude  [array]

Output options:
  --thumb-size          Pixel size of the square thumbnails  [number] [default: 120]
  --small-size          Pixel height of the small photos  [number] [default: 300]
  --large-size          Pixel height of the fullscreen photos  [number] [default: 1000]
  --photo-quality       Quality of the resized/converted photos  [number] [default: 90]
  --video-quality       Quality of the converted video (percent)  [number] [default: 75]
  --video-bitrate       Bitrate of the converted videos (e.g. 120k)  [string] [default: null]
  --video-format        Video output format  [choices: "mp4", "webm"] [default: "mp4"]
  --video-hwaccel       Use hardware acceleration (requires bitrate)  [choices: "none", "vaapi"] [default: "none"]
  --video-stills        Where the video still frame is taken  [choices: "seek", "middle"] [default: "seek"]
  --video-stills-seek   Number of seconds where the still frame is taken  [number] [default: 1]
  --photo-preview       How lightbox photos are generated  [choices: "resize", "copy", "symlink", "link"] [default: "resize"]
  --video-preview       How lightbox videos are generated  [choices: "resize", "copy", "symlink", "link"] [default: "resize"]
  --photo-download      How downloadable photos are generated  [choices: "resize", "copy", "symlink", "link"] [default: "resize"]
  --video-download      How downloadable videos are generated  [choices: "resize", "copy", "symlink", "link"] [default: "resize"]
  --link-prefix         Path or URL prefix for "linked" photos and videos  [string]
  --cleanup             Remove any output file that's no longer needed  [boolean] [default: false]
  --concurrency         Number of parallel parsing/processing operations  [number] [default: 2]
  --output-structure    File and folder structure for output media  [choices: "folders", "suffix"] [default: "folders"]
  --gm-args             Custom image processing arguments for GraphicsMagick  [array]
  --watermark           Path to a transparent PNG to be overlaid on all images  [string]
  --watermark-position  Position of the watermark  [choices: "Repeat", "Center", "NorthWest", "North", "NorthEast", "West", "East", "SouthWest", "South", "SouthEast"]

Album options:
  --albums-from            How files are grouped into albums  [array] [default: ["%path"]]
  --sort-albums-by         How to sort albums  [choices: "title", "start-date", "end-date"] [default: "start-date"]
  --sort-albums-direction  Album sorting direction  [choices: "asc", "desc"] [default: "asc"]
  --sort-media-by          How to sort photos and videos  [choices: "filename", "date"] [default: "date"]
  --sort-media-direction   Media sorting direction  [choices: "asc", "desc"] [default: "asc"]
  --home-album-name        Name of the top-level album  [string] [default: "Home"]
  --album-page-size        Max number of files displayed on a page  [number] [default: null]
  --album-zip-files        Create a ZIP file per album  [boolean] [default: false]
  --include-keywords       Keywords to include in %keywords  [array]
  --exclude-keywords       Keywords to exclude from %keywords  [array]
  --include-people         Names to include in %people  [array]
  --exclude-people         Names to exclude from %people  [array]
  --album-previews         How previews are selected  [choices: "first", "spread", "random"] [default: "first"]

Website options:
  --index                 Filename of the home page  [string] [default: "index.html"]
  --albums-output-folder  Output subfolder for HTML albums (default: website root)  [string] [default: "."]
  --theme                 Name of a built-in gallery theme  [choices: "classic", "cards", "mosaic", "flow"] [default: "classic"]
  --theme-path            Path to a custom theme  [string]
  --theme-style           Path to a custom LESS/CSS file for additional styles  [string]
  --theme-settings        Path to a JSON file with theme settings  [string]
  --title                 Website title  [string] [default: "Photo album"]
  --footer                Text or HTML footer  [string] [default: null]
  --google-analytics      Code for Google Analytics tracking  [string]
  --embed-exif            Embed the exif metadata for each image into the gallery page  [boolean] [default: false]
  --locale                Locale for regional settings like dates  [string] [default: "en"]
  --seo-location          Location where the site will be hosted. If provided, sitemap.xml and robots.txt will be created.  [string] [default: null]

Misc options:
  --config         JSON config file (one key per argument)  [string]
  --database-file  Path to the database file  [string]
  --log-file       Path to the log file  [string]
  --log            Print a detailed text log  [choices: "default", "info", "debug", "trace"] [default: "default"]
  --dry-run        Update the index, but don't create the media files / website  [boolean] [default: false]

Deprecated:
  --original-photos       Copy and allow download of full-size photos (use --photo-download=copy)  [boolean]
  --original-videos       Copy and allow download of full-size videos (use --video-download=copy)  [boolean]
  --albums-date-format    How albums are named in <date> mode [moment.js pattern]
  --css                   Path to a custom provided CSS/LESS file for styling  [string]
  --download-photos       Target of the photo download links  [choices: "large", "copy", "symlink", "link"]
  --download-videos       Target of the video download links  [choices: "large", "copy", "symlink", "link"]
  --download-link-prefix  Path or URL prefix for linked downloads  [string]
  --usage-stats           Enable anonymous usage statistics  [boolean]

Options:
  --version  Show version number  [boolean]
  --help     Show help  [boolean]


 The optional JSON config should contain a single object with one key
 per argument, not including the leading "--". For example:
 { "sort-albums-by": "start-date" }

Contributing

We welcome all issues and pull requests!

If you are facing any issues or getting crashes, please try the following options to help troubleshoot:

thumbsup [options] --log debug
# [16:04:56] media/thumbs/photo-1446822622709-e1c7ad6e82d52.jpg [started]
# [16:04:57] media/thumbs/photo-1446822622709-e1c7ad6e82d52.jpg [completed]

thumbsup [options] --log trace
# [16:04:56] media/thumbs/photo-1446822622709-e1c7ad6e82d52.jpg [started]
# gm "identify" "-ping" "-format" "%[EXIF:Orientation]" [...]
# gm "convert" "-quality" "90" "-resize" "x400>" "+profile" [...]
# [16:04:57] media/thumbs/photo-1446822622709-e1c7ad6e82d52.jpg [completed]

If you want to contribute some code, please check out the contributing guidelines for an overview of the design and a run-through of the different automated/manual tests.

Disclaimer

While a lot of effort is put into testing Thumbsup (over 400 automated tests), the software is provided as-is under the MIT license. The authors cannot be held responsible for any unintended behaviours.

We recommend running Thumbsup with the least appropriate privilege, such as giving read-only access to the source images. The Docker setup detailed in the documentation follows this advice.

downsize's Issues

Consider "-movflags +faststart"

Normally, metadata in an MP4 container is stored in a single place (the moov atom) at the end of the file. Thus, the playback cannot start until the whole video is downloaded. Traditionally, special tools where needed to move the metadata to the beginning, allowing web browser to immediately start playback (qt-faststart, MP4Box).

A few years ago, ffmpeg has added support for this using the -movflags +faststart option. I'm not sure if most users have recent enough versions of ffmpeg installed (even the version in Debian stable is recent enough) or if there is a fallback needed, but it seems reasonable to add it.

Support for HEIC photos

As per thumbsup/thumbsup#131, it would be good to implement support for HEIC photos. This might require switching from GraphicsMagick to ImageMagick who seems to be more actively maintained, and on Github: https://github.com/ImageMagick/ImageMagick

Preserve exif data

I know I have been a bit quiet on the thumbsup exif sidepanel thread (thumbsup/thumbsup#80) lately! I just finished up a very very beta prototype of preserving exif info with the downsize package.

It turns out that using gm.autoOrient() is the culprit for stripping exif info. The reasoning behind this is because if you did not strip the orientation tag, it would double rotate things. While I am not a fan of that solution, it would probably be more work to move these changes into GraphicsMagick.

Instead I am proposing that instead we just autorotate ourselves and then strip exif (either partially or fully based on user choice) in a later step. The autorotate will only strip the rotation info while keeping everything else intact.

I have the changes working on a local repo here: master...amcolash:exif_rotation

I wasn't sure if this would be the best solution (and it isn't quite set to go) so I didn't open a PR yet. It does add the exiftool dependency to this package to strip out that specific info after rotation.

heic conversion fails on Debian/Ubuntu

Hi,

the ImageMagick debian package does not contain the "magick" command in PATH. This makes all thumbsup's heic conversions fail with an ENOENT error.

A quick and dirty solution for my system (where I installed thumbsup globally) was to simply edit the file:
/usr/local/lib/node_modules/thumbsup/node_modules/thumbsup-downsize/lib/image/heic.js

modifying the convertToJpeg function from

function convertToJpeg (source, target, done) { 
  const args = [ 'convert', source, target ]
  exec('magick', args, done)
} 

to

function convertToJpeg (source, target, done) { 
  const args = [ source, target ]
  exec('convert', args, done)
} 

and similarly the function convertToSRGB:

function convertToSRGB (target, done) {
  const args = [ '-profile', SRGB_ICM_PATH, target ]
  exec('mogrify', args, done)
}

With this fix, thumbsup correctly processes my gallery.

Add smart crop option to crop/zoom on the interesting part of a photo

Can we integrate with https://github.com/jwagner/smartcrop.js ?

add exiftool as a prerequisit

exiftool is mandatory for running tests. It should be added as a prerequisit in the README file, along GraphicsMagick, FFMpeg and Gifsicle.

ffprobe.js doesn't quote filenames properly

The new getDuration function passes filenames directly to the shell without any quoting. This means filenames with spaces or quotes in them don't work, and (if this is run on untrusted filenames, which I think is unlikely) it might mean that malicious filenames can run other stuff.

Feature Request: HW Acceleration support for VAAPI

Great continuing work on this project!

I'm wondering if you would consider enabling the option to use the ffmpeg options for vaapi hardware acceleration for video decoding/encoding.

On my Synology/Linux environment, there is VAAPI HW accelerator support that can be enabled through some additional ffmpeg switches. It increases performance an order of magnitude. In the example below, I'm capping the output size at 1920 to downsize from 4K too. vaapi doesn't support directly the quality setting, but it does allow bitrate adjustments.

ffmpeg -y -nostdin -hwaccel_device /dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -i "$FILE_ORIG" -movflags +faststart -b:a 160k -vf "format=nv12|vaapi,hwupload,fps=29.97,scale_vaapi=1920:-1" -c:v h264_vaapi -f mp4 "$FILE_DEST"

Doesn't always work for .MOV files on Mac

I have some .mov files which are not being downsized correctly on my Mac. That is, the output won't play in the Finder or Quicktime. I found I could fix this by calling ffmpeg with -pix_fmt yuv420p as described in the ffmpeg docs at http://trac.ffmpeg.org/wiki/Encode/H.264. For the moment, I patched my local copy of ffmpeg by adding this to exports.prepare() in ffargs.js (mimicking, and just after, the code for .mts files):

} else if (path.extname(source).toLowerCase() === '.mov') {
args.push('-pix_fmt', 'yuv420p')
}

Animated gif as video thumbnail

Thumbsup generates a jpeg thumbnail for a video file. It would be cool, if it could be possible to generate a gif with some frames from original video and play them in an endless loop