twolfson / gif-encoder

Streaming GIF encoder

gif-encoder

Streaming GIF encoder

This is built as part of the gifsockets project. It is forked from gif.js to allow for a streaming API and performance optimization.

Getting Started

Install the module with: npm install gif-encoder

// Create a 10 x 10 gif
var GifEncoder = require('gif-encoder');
var gif = new GifEncoder(10, 10);

// using an rgba array of pixels [r, g, b, a, ... continues on for every pixel]
// This can be collected from a <canvas> via context.getImageData(0, 0, width, height).data
var pixels = [0, 0, 0, 255/*, ...*/];

// Collect output
var file = require('fs').createWriteStream('img.gif');
gif.pipe(file);

// Write out the image into memory
gif.writeHeader();
gif.addFrame(pixels);
// gif.addFrame(pixels); // Write subsequent rgba arrays for more frames
gif.finish();

Documentation

gif-encoder exports GifEncoder, a constructor function which extends readable-stream@~1.1.9. This means you can use any streams1/streams2 functionality. I will re-iterate what this means below.

// streams1
var gif = new GifEncoder(10, 10);
gif.on('data', console.log);
gif.on('end', process.exit);

// streams2
var gif = new GifEncoder(10, 10);
gif.on('readable', function () {
  console.log(gif.read());
});

new GifEncoder(width, height, [options])

Constructor for a new GifEncoder

• width Number - Width, in pixels, of the GIF to output
• height Number - Height, in pixels, of the GIF to output
• options Object - Optional container for any options
  • highWaterMark Number - Number, in bytes, to store in internal buffer. Defaults to 64kB.

NEVER CALL .removeAllListeners(). NO DATA EVENTS WILL BE ABLE TO EMIT.

We implement the GIF89a specification which can be found at

http://www.w3.org/Graphics/GIF/spec-gif89a.txt

Events

Event: data

function (buffer) {}

Emits a Buffer containing either header bytes, frame bytes, or footer bytes.

Event: end

function () {}

Signifies end of the encoding has been reached. This will be emitted once .finish() is called.

Event: error

function (error) {}

Emits an Error when internal buffer is exceeded. This occurs when you do not read (either via .on('data') or .read()) and we cannot flush prepared data.

If you have a very large GIF, you can update options.highWaterMark via the Constructor.

Event: readable

function () {}

Emits when the stream is ready to be .read() from.

Event: writeHeader#start/stop

function () {}

Emits when at the start and end of .writeHeader().

Event: frame#start/stop

function () {}

Emits when at the start and end of .addFrame()

Event: finish#start/stop

function () {}

Emits when at the start and end of .finish()

Settings

gif.setDelay(ms)

Set milliseconds to wait between frames

• ms Number - Amount of milliseconds to delay between frames

setFrameRate(framesPerSecond)

Set delay based on amount of frames per second. Cannot be used with gif.setDelay.

• framesPerSecond Number - Amount of frames per second

setDispose(disposalCode)

Set the disposal code

• disposalCode Number - Alters behavior of how to render between frames
  • If no transparent color has been set, defaults to 0.
  • Otherwise, defaults to 2.

Values :    0 -   No disposal specified. The decoder is
                  not required to take any action.
            1 -   Do not dispose. The graphic is to be left
                  in place.
            2 -   Restore to background color. The area used by the
                  graphic must be restored to the background color.
            3 -   Restore to previous. The decoder is required to
                  restore the area overwritten by the graphic with
                  what was there prior to rendering the graphic.
         4-7 -    To be defined.

Taken from http://www.w3.org/Graphics/GIF/spec-gif89a.txt

setRepeat(n)

Sets amount of times to repeat GIF

• n Number
  • If n is -1, play once.
  • If n is 0, loop indefinitely.
  • If n is a positive number, loop n times.

setTransparent(color)

Define the color which represents transparency in the GIF.

• color Hexadecimal Number - Color to represent transparent background
  • Example: 0x00FF00

setQuality(quality)

Set the quality (computational/performance trade-off).

• quality Positive number
  • 1 is best colors, worst performance.
  • 20 is suggested maximum but there is no limit.
  • 10 is the default, provided an even trade-off.

Input/output

read([size])

Read out size bytes or until the end of the buffer. This is implemented by readable-stream.

• size Number - Optional number of bytes to read out

writeHeader()

Write out header bytes. We are following GIF89a specification.

addFrame(imageData, options)

Write out a new frame to the GIF.

• imageData Array - Array of pixels for the new frame. It should follow the sequence of r, g, b, a and be 4 * height * width in length.
  • If used with the options palette and indexedPixels, then this becomes the index in the palette (e.g. 0 for color #0)
• options Object - Optional container for options
  • palette Array - Array of pixels to use as palette for the frame. It should follow the sequence of r, g, b, a
    • At the moment, this must be used with options.indexedPixels
  • indexedPixels Boolean - Indicator to treat imageData as RGBA values (false) or indicies in palette (true)

finish()

Write out footer bytes.

Low-level

For performance in gifsockets, we needed to open up some lower level methods for fancy tricks.

Don't use these unless you know what you are doing.

flushData()

We have a secondary internal buffer that collects each byte from writeByte. This is to prevent create a new Buffer and data event for every byte of data.

This method empties the internal buffer and pushes it out to the stream buffer for reading.

pixels

Internal store for imageData passed in by addFrame.

analyzeImage(imageData, options)

First part of addFrame; runs setImagePixels(removeAlphaChannel(imageData)) and runs analyzePixels().

• imageData Array - Same as that in addFrame
• options Object - Optional container for options
  • indexedPixels Boolean - Indicator to treat imageData as RGBA values (false) or indicies in palette (true)

removeAlphaChannel(imageData)

Reduces imageData into a Uint8Array of length 3 * width * height containing sequences of r, g, b; removing the alpha channel.

• imageData Array - Same as that in addFrame; array containing r, g, b, a sequences.

setImagePixels(pixels)

Save pixels as this.pixels for image analysis.

• pixels Array - Same as imageData from addFrame
  • GifEncoder will mutate the original data.

setImagePalette(palette)

Save palette as this.userPalette for frame writing.

• palette Array - Same as options.palette from addFrame

writeImageInfo()

Second part of addFrame; behavior varies on if it is the first frame or following frame.

In either case, it writes out a bunch of bytes about the image (e.g. palette, color tables).

outputImage()

Third part of addFrame; encodes the analyzed/indexed pixels for the GIF format.

Donating

Support this project and others by twolfson via donations.

http://twolfson.com/support-me

Contributing

In lieu of a formal styleguide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint via grunt and test via npm test.

UNLICENSE

As of Nov 11 2013, Todd Wolfson has released all code differences since initial fork from gif.js to the public domain.

These differences have been released under the UNLICENSE.

At the gif.js time of forking, gif.js was using the MIT license.

gif-encoder's Issues

about GIF memory limit exceeded

I'm a novice of nodejs and I have some question about GIF memory limit exceeded

• what is the maximum size of 'read'?
• how to remove the limit?

events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: GIF memory limit exceeded. Please `read` from GIF before writing additional frames/information.
    at GIFEncoder.ByteCapacitor.flushData (C:\Users\home\Desktop\pixivDLerNode\node_modules\gif-encoder\lib
\GIFEncoder.js:43:15)
    at GIFEncoder.flushData (C:\Users\home\Desktop\pixivDLerNode\node_modules\gif-encoder\lib\GIFEncoder.js
:110:10)
    at emitNone (events.js:86:13)
    at GIFEncoder.emit (events.js:185:7)
    at GIFEncoder.addFrame (C:\Users\home\Desktop\pixivDLerNode\node_modules\gif-encoder\lib\GIFEncoder.js:
216:8)
    at zipEntries.forEach (C:\Users\home\Desktop\pixivDLerNode\index.js:65:9)
    at Array.forEach (native)
    at makeGif (C:\Users\home\Desktop\pixivDLerNode\index.js:62:14)
    at WriteStream.<anonymous> (C:\Users\home\Desktop\pixivDLerNode\index.js:23:5)
    at emitNone (events.js:91:20)
    at WriteStream.emit (events.js:185:7)
    at fs.js:2037:14
    at FSReqWrap.oncomplete (fs.js:123:15)

Quality and File Size Issues(Doesn't look like gif limitations)

I would have liked to make this a discussion rather than an issue but there is no discussion tab for this repository so I'll just initially bring it up here.

Here is a gif I made in photoshop (You will need to zoom in for a better look):

Here is the same gif but created using the gif-encoder:

The photoshop created gif has a consistent palette for the background image throughout the animation. While as even on the highest quality gif-encoder has some trouble keeping a consistent palette resulting in some flickering.
This doesn't happen for every image, for example using a simpler version of the background(no shading on face), I can create a gif with no flickering with gif-encoder:

Before I reproduced the gif in photoshop, I thought this was just a limitation of gifs having a reduced colour palette but after creating it manually on photoshop and having someone else create it using their own method, I had to make sure so I used a colour analyser aside from photoshop to count the different colours on each frame. Needless to say, there are less than 70 individual colours used overall for the shaded version and even less for the unshaded. Neither could use the maximum 256 but for some reason, when I use gif-encoder, I cannot accurately get the correct colours per frame.

On another note, there is a massive disparity in file size between gifs made in photoshop and gif-encoder. As you can check for yourselves, the shaded version has an exported file size of 4kb from photoshop and with gif-encoder it is 68kb.
I know it is not fair for me to compare gif-encoder and photoshop, so... I tried using an online gif maker and there were no problems with flickering even though the file size was around 8kb which is twice the size as photoshop but still better size compared to the library.
I'm not an image expert otherwise I would make contributions here but maybe the author is still willing to improve this library after all these years?

Gif has low quality / noise

Ive generated a gif out of 30 .png files. But if you look closely you can see some quality issues there.
The source Files are "clean". They look exactly the same (well except for that animated thing).
This was the result:

Files;
gif.tar.gz
use tar xfz gif.tar.gz to unpack the files

Code:

        let pathA = PathService.getInstance().combinePaths([outPutPath, `????.png`])
        let pathB = PathService.getInstance().combinePaths([outPutPath, `done.gif`])
        var pngFileStream = require('png-file-stream');
        var GifEncoder = require('gif-encoder');
        var file = fs.createWriteStream(pathB);
        let gif = new GifEncoder(dimensions[0], dimensions[1]);
        gif.setRepeat(0);
        gif.setQuality(10);
        gif.pipe(file);
        gif.writeHeader();

        pngFileStream(pathA).on('data', (pixelsBuffer) => {
            gif.addFrame(pixelsBuffer)
        }).on('finish', () => {
            gif.finish();
            console.log("done")
        })

Am i doing it wrong? Just started using your libary and followed your workaround for missing png-file-stream. described in #11 .

Broken gifs with jimp images?

When I'm taking some screenshots with puppeteer and encode them the result is okay but if I would like to edit these shots with JIMP then encode things goes messy and results are only broken gifs.

Size very big

I have 10 images. Each imag size = +-5kb. Summary this 50 kb, but my gif have 500rb. What to do ? I set quality = 20

Bugfix for broken transparency

I've discovered what I believe to be a bug with the transparency. Specifically, in the function that finds the nearest color in the palette. Right now transparency simply does not work at all because this function doesn't work.

This function will fail to find any color because it's checking that this.usedEntry has a true value at the index i / 3. But i is never a factor of 3 and so it's checking float index values.

The fix is changing line 302 to this:

var db = b - (this.colorTab[i++] & 0xff);

The previous 2 lines have the ++, and this line needs it as well so that i is incremented 3 times before being divided so an integer results. For some reason, the increment is at the end of the loop, so line 309 needs to be removed.

After making these changes, transparency works again.

Allow for streaming write into `gif-encoder`

As inspired by the pngjs API, we should allow streaming in of rgba values. I am curious how we will make pipe out work sans methods though.

How to Keep origin gif transparent

I am trying to split a gif to multi frames, then write some words on it by canvas, finally combine them to a new gif.
But when origin gif contains transparent, then new gif is not with transparent no longer. How should I fix it? It seems like that setTransparent() is not work for it.

setRepeat not working

set gif.setRepeat(-1) but infinity. All code lint from gist code

Relocate NeuQuant and LZWEncoder to their own repositories

For well targeted tests, one thing well, and re-usable code, it would be ideal to relocate NeuQuant and LZWEncoder to their own repositories. Then, we can add tests and move into performance driven development as in #1.

question regarding the palette

Hi again,
I am new to image treatment, so maybe you'll find my question stupid, and maybe you won't want to spend time answering me, I'd totally understand.

As far as I understand, we can use a palette and indexed pixels to this palette instead of a brut array of pixels. I don't know exactly the reason, but I guess it is supposed to reduce the size of the final GIF file (is it ?).

So my question is the following : in your README, you mention this :

palette Array - Array of pixels to use as palette for the frame. It should follow the sequence of r, g, b, a and be 4 * height * width in length.

I don't understand this sentence "[...] and be 4 * height * width in length." : if the palette is supposed to have a set of colors to distribute to the pixel indexed frame, why should it have the same length of the (non-pixel-indexed) frame ?

In the gif above, for example, there are four colors, so I assume the palette would only be [34,41,173,1,163,206,73,1,197,1,27,1,0,0,0,1 ], wouldn't it ?

Add image frame to gif

var file = require('fs').createWriteStream('img.gif');
gif.pipe(file);
gif.writeHeader();    

fs.createReadStream('mygif.gif')
.pipe(new GIFDecoder)
.pipe(concat(function(frames) {
    console.log(frames)
    for (var i=0; i<frames.length; i++) {
        gif.addFrame(frames[i]);
    }
    gif.finish();
}));

so I am trying to add a single frame to the gif, that is my image/other gif/whatever but I cant at best I get a black gif, any ideas? I have been trying to do so for the past 2-3 days with this and other modules but with no success, any help will be really appreciated, thanks.

ps: I also tried converting a png to base64 but as I came to understand it doesnt accept base 64.

Create bin script for encoding of images into gifs

This either should be in the same repo or another one which depends on gif-encoder (probably the latter), such that we can write gif into to stdout or pipe anywhere

gif-encoder *.png --delay 500 | hello.gif

Is the correct value UNLICENSED for the licenses field in package.json?

I noticed the licenses field in package.json is MIT. Is the correct value UNLICENSED ?
https://github.com/twolfson/gif-encoder/blob/master/package.json#L20-L21

PNG file streams?

Can I use this library with png-file-stream which is intended for use with gifencoder (not gif-encoder)?

Seems that gifencoder has broken transparency, so I can't use that.

Create a promise with pixels as entry, gif as resolve

Hi there,

I want to thank you for your lib, but I can't yet because I couldn't make it work :)

I am trying to create a gif on client-side, starting with the DOM.
I use for that the lib dom-to-image which, thanks to it's method toPixelData, return some pixels.
Then I want these pixels to give me a gif.
I thought about using a Promise for that because the last step is to put this gif in a ZIP file created by JSZip.

Anyway, so here is my Promise, and it's not returning what I want, could you help please ?

const pixelsToGIF = (pixels, fileName, width, height) =>
  new Promise((resolve, reject) => {
    const gif = new GifEncoder(width, height);
    const gifFile = fs.createWriteStream(fileName);
    gif.pipe(gifFile);
    gif.writeHeader();
    gif.addFrame(pixels);
    gif.finish();
    gif.on('end', () => resolve(gif))
    gif.on('error', reject)
    // gif.on('data', console.log)
  })

programmatically improve GIF size

Hello,
I try to improve the GIF size I am creating by playing with the setDispose and/or setTransparent parameters, but it doesn't have any effect at all on the GIF I created.

Let say I have this easy GIF file, where each frame is a Uint8ClampedArray of pixels of a screenshot of a DOM Node ; it has 40 frames, coming from 40 screenshots ; what would be the options I should choose to reduce the size of the output GIF ?

Another question : is there a way I can improve the GIF size by playing around with the Uint8ClampedArray of pixels I am feeding the GIFEncoder with ? I mean, if I was filtering each frames, giving the "transparent color" index to each pixel identical on previous frame, would it improve the size the GIF output ?

Thanks very much for your previous help, and for this one too I hope !
Cheers,
Arnaud

Add performance tests

During the development of gifsockets, performance was a major priority. We should embrace performance driven development as a core principle for this repository.

Things to perf:

• Verify using ByteCapacitor over pushing for every byte is more efficient
  • Test via both .read() and .on('data')
• Switch to ndarray over typed arrays for NeuQuant

image size

I am combining 6 jpg images about 70K a piece into an animated gif using 20 as the compression value... The resulting GIF is about 3MB. Does this sound normal to you? It seems like it's excessively large. Maybe it has something to do with the JPG compression. Thanks!

Wow. What ?

Gif 1100x200. What happened ? Image is broken. link image

Weird gif

Thanks for building this lib, I am facing some problems generating a gif. Is it my data that is corrupted?

sharp(jpeg)
      .raw()
      .toBuffer({ resolveWithObject: true })
      .then((value) => {
        const pixels = new Uint8ClampedArray(value.data.buffer);
        console.log(pixels);
        var gif = new GifEncoder(600, 900);
        var file = require("fs").createWriteStream("img.gif");
        gif.pipe(file);
        gif.writeHeader();
        gif.addFrame(pixels);
        gif.finish();
      });

The end result is this 1 frame gif: