creativelive / mkimage-server

On-demand image resize server based on 'express'

mkimage-server

On-demand image resize server based on 'express'

Installation & running

1: assuming you have proper node 0.8.x installed

2: user that runs the server is 'mkimage'

3: set desired settings in config directory

# install imagemagick
sudo apt-get install --no-install-recommends build-essential imagemagick

# clone the repo
git clone git://github.com/Boxee/mkimage-server.git

# install node dependencies
cd mkimage-server && npm install

# create `cache_dir`
mkdir -p /mnt/cache/mkimage-server
chown mkimage.mkimage /mnt/cache/mkimage-server

# run it (in production)
NODE_ENV=production bin/mkimage

##CreativeLive specific setup in Ngninx setup

add to cdn.creativelive.com
  location ~* ^/(fill|fit|crop|chop)/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-NginX-Proxy true;
    proxy_pass http://mkserver;
    proxy_redirect off;
  }

add to creativelive.com
upstream mkserver {
  server 127.0.0.1:3020;
}

  location ~* ^/(fill|fit|crop|chop)/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-NginX-Proxy true;
    proxy_pass http://mkserver;
    proxy_redirect off;
  }

Haproxy will need to redirect those same routes fill/, fit/, and crop/ to whatever server is running the mkimage-server.

Basic usage

• url - the url of remote image to be converted (and cached)
• w - width of resized image
• h - height of resized image
• q - quality of resized image (default is 92)

the url parameter must be encoded, otherwise unexpected behaviour may occur.

• the url http://www.google.com/images/icons/product/apps-128.png
• becomes http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png

Note: lib/oldapi.js is actually the new API, don't let it fool you.

Examples

# resize an image with keeping ratio (width)
# /fit/{url}/{width}/{height?}
http://example.com/fit/http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png/200

## resize an image with keeping ratio (width) and changing image quality (use q as a query param)
# /fit/{url}/{width}?q={quality}
http://example.com/fit/http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png/200?q=60

# resize an image with keeping ratio (width & height)
http://example.com/fit/http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png/100/200

# resize an image to given size without keeping the ratio
# /fill/{url}/{width}/{height}
http://example.com/fill/http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png/200/200

# very basic crop for now, will add more options in future
# /crop/{url}/{width}/{height}/{xOffset}/{yOffset}
http://example.com/crop/http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png/200/200/50/50

# a centered chop, shrink & crop cutting off edges - good for square thumbnails...
# /chop/{url}/{width}/{height}
http://example.com/chop/http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png/100/100

# return resized/cropped/etc. image in webp format
# /{method}/{url}/webp/{width}/{height?}/{xOffset?}/{yOffset?}
http://example.com/fit/http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png/webp/200
http://example.com/crop/http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png/webp/200/200/50/50

Old API

# resize an image with keeping ratio (width)
http://example.com/resize?url=http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png&w=200

# resize an image with keeping ratio (height)
http://example.com/resize?url=http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png&h=200

# resize an image to given size without keeping the ratio
http://example.com/stretch?url=http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png&w=200&h=200

# crop an image from the CENTER (height is optional),
# very basic crop for now, will add more options in future
http://example.com/crop?url=http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png&w=200

# return image with same size as original (just cache it)
http://example.com/cache?url=http%3A%2F%2Fwww.google.com%2Fimages%2Ficons%2Fproduct%2Fapps-128.png

---

More docs comming soon

mkimage-server's Issues

add route to clear cached images

add route to clear cached images

need health check page

We're going to need a health check page for this app, so that we can run multiple instances behind HAproxy. Something like the /tigris health check in tigris or what used to be magicmangler. Doesn't need to be fancy, just an "OK" page is fine, so that HAproxy knows when the app is up and functioning.

Define cache folder location on stage and prod environments

fix attempt to open non-existent log file

Error: ENOENT, open '/mnt/logs/mkimage-access.log'
worker 9996, respawning in 1 second ...
worker 10002 running

events.js:72
throw er; // Unhandled 'error' event
^
Error: ENOENT, open '/mnt/logs/mkimage-access.log'
worker 10002, respawning in 1 second ...
worker 10008 running

events.js:72
throw er; // Unhandled 'error' event
^
Error: ENOENT, open '/mnt/logs/mkimage-access.log'
worker 10008, respawning in 1 second ...
worker 10015 running

No such destination exists in non-dev environments.

Ensure EXIF data is not stripped from resulting image.

update app.js to use production config for our prod AND stage environments

update app.js to use production config for our prod AND stage environments

mkimage-server does not report it's full execution path

In the process table, the master process for mkimage-server only shows:

www-data 6147 1 0 Feb19 ? 00:00:00 mkimage: master process /var/nodejs/mkimage-ser

contrast to something like nginx (which it appears mkimage's process listing is based on):

root 28135 1 0 Feb21 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf

In fact, mkimage-server is the only app in the process table that does NOT report it's full execution path, which is a problem. It must be some code issue or something, because our other node apps do not have this issue either. This is actually fairly urgent as our normal management tools are unable to manage this process without a good process table listing.

URLs beginning with /fit have http:// added to them causing nginx warnings.

Nginx consistently reports issues with mkimage URLs that begin with /fit/ which may be due to imageresizer.js adding an extra http:// into the request URL. An example URL would look something like:

http://localhost:3020/fit/http://assets.creativelive.com/sites/default/files/HOP-09q4-BNR-CDI-640x360px.png

which if you attempt to curl give an Unauthorized error. Removing the second http:// from the URL and doing the same replies with "could'd download from http://assets.creativelive.com, http error: 403" which seems more reasonable.

Status page URL is getting passed on to the image resizer.

The URL for the mkimage status pages gets routed along to the image resizer component which causes "WARN request /mkimage/status is not valid" to get dumped into the logs along with the normal status checks which are ok.

change default port to 3020

change default port to 3020 for all NODE_ENV