Coder Social home page Coder Social logo

martinheidegger / mapslice Goto Github PK

View Code? Open in Web Editor NEW
47.0 7.0 10.0 2.6 MB

Slices a given image into tiles to be used for a interactive map display. (including command-line tool)

License: MIT License

JavaScript 95.41% Shell 1.03% TypeScript 3.55%
image-processing maps tiles

mapslice's Introduction

mapslice - Image crop & slice tool

Build Status js-standard-style

mapslice is a tool to cut images into slices of various zoom levels for use in interactive maps.

Javascript tools for high-performant viewing of huge images are plenty available, yet cropping and slicing images can be a pain with the given tools. mapslice automatically detects which tile-sizes your input-material supports and creates all possible tiles to be used by a common javascript map tool like polymaps, kartograph or PanoJS.

Installation

via npm:

$ npm install mapslice

Command line

After installing the latest node, you can use mapslice as a command-line tool by installing it with:

$ npm install mapslice -g

Also make sure that you have GraphicsMagick or ImageMagick installed and available in your command-line!

Once the prerequisites are given, run mapslice using:

$ mapslice -f test.jpg

For more documentation run mapslice without arguments:

$ mapslice

Script usage

const MapSlicer = require('mapslice')

// The parameters passed here are equal to the command-line parameters
const mapSlicer = new MapSlicer({
  file: `myImage.jpg`,               // (required) Huge image to slice
  output: `myImage/{z}/{y}/{x}.png`, // (default: derived from file path) Output file pattern
  outputFolder: './output',          // (default: derived from file path) Output to be used for. Use either output or outputFolder, not both!
  tileSize: 512,                     // (default: 256) Tile-size to be used
  imageMagick: true,                 // (default: false) If (true) then use ImageMagick instead of GraphicsMagick
  background: '#00000000',           // (default: '#FFFFFFFF') Background color to be used for the tiles. More: http://ow.ly/rsluD
  tmp: './temp',                     // (default: '.tmp') Temporary directory to be used to store helper files
  parallelLimit: 3,                  // (default: 5) Maximum parallel tasks to be run at the same time (warning: processes can consume a lot of memory!)
  minWidth: 200,                     // See explanation about Size detection below
  skipEmptyTiles: true,              // Skips all the empty tiles
  bitdepth: 8,                       // (optional) See http://aheckmann.github.io/gm/docs.html#dither
  dither: true,                      // (optional) See http://aheckmann.github.io/gm/docs.html#bitdepth
  colors: 128,                       // (optional) See http://aheckmann.github.io/gm/docs.html#colors
  gm: require('gm'),                 // (optional) Alternative way to specify the GraphicsMagic library
  signal: new (require('abort-controller'))().signal // (optional) Signal to abort the map slicing process
})

mapSlicer.on('start', (files, options) => console.info(`Starting to process ${files} files.`))
mapSlicer.on('imageSize', (width, height) => console.info(`Image size: ${width}x${height}`))
mapSlicer.on('levels', (levels) => { console.info(`Level Data: ${levels}`) }) // see TypeScript declaration for more details
mapSlicer.on('warning', err => console.warn(err))
mapSlicer.on('progress', (progress, total, current, path)  => console.info(`Progress: ${Math.round(progress*100)}%`))
mapSlicer.on('end', () => console.info('Finished processing slices.') )
mapSlicer.start().catch(err => console.error(err))

## Size detection and scaling

To render the image in its fullest glory, mapslice assumes that you want to preserve the original image-quality and chooses input-size as its starting point from which the quality should be reduced. However: If you have a fixed-size map-user-interface then you might want the smallest image quality to fit this user-interface-design in order to assure that its is beautifully visible. To produce tiles that fit this needs you can use the "minWidth" or "minHeight" property which fits the map to have its lowest size matching exactly your required size:

```console
$ mapslice -f test.jpg -w=1000

Will fit the smallest size to be exactly 1000 pixels wide and zoom up from there.

Note

To speed up performance mapslice stores a prescaled version of the each zoom-level in a temorary folder and then just crops off of that. These temporary files can become quite big as they are stored with low compression and high quality in sgi files.

License

MIT

mapslice's People

Contributors

aaronblondeau avatar edjr avatar frulo avatar martinheidegger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

mapslice's Issues

Unrecognized color

Getting this error no matter whether using imagemagick or graphicsmagick.

Error: Command failed: gm convert: Unrecognized color (0000000).

I fixed it by setting the background color to rgb(0,0,0) instead and everything works. Node 6.3.1, both graphics libraries installed via brew if it matters.

Error fetching size of file and em identify not returning a file

i just installed this library on Frdora and whether i use the global mapslice install via terminal or run or via Node similar to the sample code provided in the doc i get these errors:

$ node index.js 
Error: Error while fetching size of file
    at gm.<anonymous> (/var/www/sandbox/map/node_modules/mapslice/lib/getImageSize.js:15:33)
    at gm.emit (events.js:311:20)
    at gm.<anonymous> (/var/www/sandbox/map/node_modules/gm/lib/getters.js:129:14)
    at cb (/var/www/sandbox/map/node_modules/gm/lib/command.js:322:16)
    at ChildProcess.onExit (/var/www/sandbox/map/node_modules/gm/lib/command.js:305:9)
    at ChildProcess.emit (events.js:311:20)
    at maybeClose (internal/child_process.js:1021:16)
    at Socket.<anonymous> (internal/child_process.js:443:11)
    at Socket.emit (events.js:311:20)
    at Pipe.<anonymous> (net.js:668:12) {
  code: 'MMAPIDENT',
  cause: Error: Command failed: gm identify: No decode delegate for this image format (/var/www/sandbox/map/map.jpg).
  gm identify: Request did not return an image.
  
      at ChildProcess.onExit (/var/www/sandbox/map/node_modules/gm/lib/command.js:301:17)
      at ChildProcess.emit (events.js:311:20)
      at maybeClose (internal/child_process.js:1021:16)
      at Socket.<anonymous> (internal/child_process.js:443:11)
      at Socket.emit (events.js:311:20)
      at Pipe.<anonymous> (net.js:668:12) {
    code: 1,
    signal: null
  },
  file: '/var/www/sandbox/map/map.jpg',
  hint: undefined
}

or

mapslice -f map.jpg 

Error: Error while fetching size of file
---
code: MMAPIDENT
cause: Error: Command failed: gm identify: No decode delegate for this image format (map.jpg).
gm identify: Request did not return an image.

file: map.jpg

the map.jpg is a valid jpeg file and i have installed 'GraphicsMagick' but don't have imagicmagick. any idea why these errors?

thanks

Conversion instances are executed beyond the number of parallelLimit

The conversion process (ImageMagick, GraphicsMagic) will be executed regardless of the value defined by parallelLimit.
Perhaps it doesn't make sense because the next process runs before the above process ends, or it runs out of memory endlessly.

The image to be tiled is a 32-bit PNG of 12288x12288 with a capacity of about 7.34M, but the memory will be exhausted from around 25% and the operation will not be accepted until 1 hour when the processing is completed.

Support tile overlap

Some tile systems require tile overlap to produce a map that doesn't look ugly (particular problem of svg maps)

out of memory issue

I have uploaded an image with this spec:
13.62MB
original dimension: 15822 x 1897

basemap-1469553899965

The issue that is running out memory on each every time. With the heroku free dyno it gives me R14 error on the first try when the RAM exceed 500MB. Then I pull the code down and run it on windows with 8GB ram 64bit.

I use parallel limit =5 which is the default setting and i found that will crash every time on the zoom level 6. I think the better design show give the gm to release the ram when starting the next zoomlevel. that will squeeze out some memory from the process. i can think of another way using serial async process.

my suggestion or idea:

  1. parallelLimit can be calculated based on tiles number
  2. using serial sync process

maybe if that has another callback calculation strategy before firing up the command on gm or im would be great. because you dont know what is the computation restrain in different environments.

Trying to process more than one image: crypto HashUpdate fail

I'm trying to implement a page with express and multer where uploaded images should be immediately processed by mapslice. It works fine with the first image, but fails for the second one, whether I try to upload both at the same time or sequentially:

crypto.js:70
  this._handle.update(data, encoding);
               ^
TypeError: HashUpdate fail
[....]

In nodejs/node-v0.x-archive#1415 it is stated that "crypto instances are not reusable", so I guess the script is trying to reuse the same object for the second image. Is this a bug in mapslice or am I doing something wrong?

The following code reproduces the problem, although it's not exactly what I'm using:

var mapslice = require('mapslice');

var mapSlicer = mapslice({
  file: 'myimg.png'
})
  .on("start", function(files, options) {
    console.info("Processing first image....");
})
  .on("end", function() {
    console.info("Finished processing first image.\n\nProcessing second image....");

    var mapSlicer2 = mapslice({
      file: 'myimg.png'
    }).start();
}).start();

Error: invalid options argument

Hi,

I've installed mapslice using npm but, whenever I try to run it, I get an error from the mkdirp dependency saying: invalid options argument

You can see the output below.

What am I doing wrong?

$ mapslice -f highres.jpeg
Input File: highres.jpeg
Input Size: 5047x2794
Level #0: 256x256, 157x87
Level #1: 512x512, 314x174
Level #2: 1024x1024, 628x348
Level #3: 2048x2048, 1256x696
Level #4: 4096x4096, 2512x1392
Level #5: 8192x8192, 5024x2784
Single Tile Size: 256x256px
Files to process: 1371
Progress:                       ▏0% ▏0/1371 ▏/usr/local/lib/node_modules/mapslice/node_modules/mkdirp/lib/opts-arg.js:13
    throw new TypeError('invalid options argument')
    ^

TypeError: invalid options argument
    at optsArg (/usr/local/lib/node_modules/mapslice/node_modules/mkdirp/lib/opts-arg.js:13:11)
    at mkdirp (/usr/local/lib/node_modules/mapslice/node_modules/mkdirp/index.js:11:10)
    at processLevel (/usr/local/lib/node_modules/mapslice/lib/processLevel.js:7:3)
    at /usr/local/lib/node_modules/mapslice/lib/MapSlicer.js:78:13
    at /usr/local/lib/node_modules/mapslice/node_modules/async/dist/async.js:2948:28
    at replenish (/usr/local/lib/node_modules/mapslice/node_modules/async/dist/async.js:440:21)
    at /usr/local/lib/node_modules/mapslice/node_modules/async/dist/async.js:445:13
    at eachOfLimit$1 (/usr/local/lib/node_modules/mapslice/node_modules/async/dist/async.js:471:34)
    at awaitable (/usr/local/lib/node_modules/mapslice/node_modules/async/dist/async.js:208:32)
    at eachOfSeries (/usr/local/lib/node_modules/mapslice/node_modules/async/dist/async.js:658:16)

source is not defined

I used this package as instructed and I got sourc is not defined at on line 23. I have ImageMagick installed globally.

update to support node5 and node6

are you planning on support with version? if so there will need to make separate version of the library because the emitEvent is using different code now.

More tests

I wrote some initial unit tests for a few sections, but there are still quite a few missing...

Request - Tile size NOT power of 2

Hi, my Panorama images are 8000 x 4000, so I think it would make sense to have 500x500 tiles?

This size is the standard output of the new Iris360 Google Street View Camera.

Automated tile slicing for a specific item (eg. campus building)

Hi @martinheidegger, I found your mapslice library nifty for automated slicing of tiles.

I do have a question, i am relatively new to tile slicing and would like to get your input.

I've used your mapslicer library, and is it doing the following, correct me if i'm wrong :-

  1. Assumes that you are slicing a full map and not a specific item that i wish to place on a map (eg. Campus building)
  2. Because of that the auto-generated directory or z/x/y starts from 0/0/0 ? If i would like to place the campus building on a specific building, this library does not support that right?

If that's the case, do you know what i should be looking at in order to get an automated tile slicing script running?

Image Cropping

Hi,

my actual image size : 4096 x 2048
I m trying to slice the image with tileSize : 512,

in this case it should create "32" tiles. but instead creating 89 files with blank images. how can i prevent that.

Error thrown in couldnot execute ImageMagick "identify"

C:\Users\susingan\Desktop\MapSlice>mapslice -f a_base.png

Error: Error#2: Error while fetching size of File: a_base.png; Error: Error: Could not execute GraphicsMagick/ImageMagick: gm "identify" "-ping" "-format" "%wx%h" "a_base.png" this most likely means the gm/convert binaries can't be found
    at MapSlicer.<anonymous> (C:\Users\susingan\AppData\Roaming\npm\node_modules\mapslice\lib\MapSlicer.js:79:40)
    at emitMany (events.js:127:13)
    at gm.emit (events.js:201:7)
    at gm.<anonymous> (C:\Users\susingan\AppData\Roaming\npm\node_modules\mapslice\node_modules\gm\lib\getters.js:70:16)
    at cb (C:\Users\susingan\AppData\Roaming\npm\node_modules\mapslice\node_modules\gm\lib\command.js:322:16)
    at ChildProcess.<anonymous> (C:\Users\susingan\AppData\Roaming\npm\node_modules\mapslice\node_modules\gm\lib\command.js:232:9)
    at emitOne (events.js:96:13)
    at ChildProcess.emit (events.js:188:7)
    at ChildProcess.cp.emit (C:\Users\susingan\AppData\Roaming\npm\node_modules\mapslice\node_modules\cross-spawn\lib\enoent.js:36:37)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:204:12)

C:\Users\susingan\Desktop\MapSlice>identify -ping -format %wx%h a_base.png
5120x2880
C:\Users\susingan\Desktop\MapSlice>

The binary command identify -ping -format %wx%h a_base.png which is throwing the error is accessible and gives the result as 5120x2880 but map slice fails

zoom level customization

how to tune the zoom level on each zoom and the number of zooms to be configured? i have found zoomMin in the source.

skip empty tiles

actually Actually mapslice generate lot of empty/blanc tiles. That is a good feature to ensure the same image position for different zoom level. In some cases the user do not need this empty tiles. I make some changes in the code that give the possibility to skip this tiles. I would like to push the changes...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.