animated gif support? about imagor

[jcupitt]

Hi again @cshum,

You can watermark an animated GIF in one operation using replicate: make a one-frame watermark, use replicate to duplicate it vertically for N frames, then composite that over the GIF. replicate duplicates images by copying pointers, so it's very fast.

You don't need to chop into frames for resize -- just round the resize factor so that pixels land on frame boundaries. thumbnail does this for you.

You would need to cut into frames from crop and rotate. It's not as bad as it sounds -- you can reassemble them quickly using arrayjoin (it just copies pointers, so it's very fast), and when you have a single image again, libvips is able to compute the whole thing (including all the crops etc.) in one parallel loop, so it's no slower. Really, the only downsize is the annoyance of dealing with an array of images instead of a single frame.

You can handle many-page PDFs and TIFFs, plus animated webp and heic in the same code path.

[cshum]

Imagor v0.8.0, docker image shumc/imagor:0.8.0now added animation support!

original image

https://raw.githubusercontent.com/cshum/imagor/master/testdata/dancing-banana.gif

fit-in, fill and watermark:

http://localhost:8000/unsafe/fit-in/200x150/filters:fill(yellow):watermark(raw.githubusercontent.com/cshum/imagor/master/testdata/gopher-front.png,repeat,bottom,0,40,40)/raw.githubusercontent.com/cshum/imagor/master/testdata/dancing-banana.gif

crop and fill:

http://localhost:8000/unsafe/30x40:100x150/filters:fill(cyan)/raw.githubusercontent.com/cshum/imagor/master/testdata/dancing-banana.gif

[cshum]

@gingerlime nice question :)

I did keep a close eye on animated gif, especially after the upstream library govips recently tried to add support for animated gif: davidbyttow/govips@623c60d

What their code does is to expand every gif frames into a grid of a large image, process it then turn it back to the animated gif. But at the moment, only resize and rotate is implemented.

While I presume similar treatment can be done for vips_thumbnail, trim or crop using vips_extract_area is seemingly difficult to implement. I could not think of any ways other than exhaustively looping the same operation for each frame, which will be really slow. Similar case goes for watermark.

Hi @jcupitt do you have any insight on implementing vips_extract_area and vips_composite for animated gif? Or are there vips functions or clever fast path that I have missed? Thanks :)

[gingerlime]

Thanks again @cshum !

I think thumbor only provides partial support for animated gifs. For example, I think if you try to add a watermark, it just ignores it. Not sure about resizing etc. However, it still returns an animated gif, rather than just a still frame. So I would say whatever can be supported is nice, but even starting off by ignoring options that are too complicated, but still returning an animated gif would be a big improvement :) currently with imagor, even if I don't do any transformations, I get back a static image...

[gingerlime]

Wow this looks amazing! And I think takes it a step above what thumbor offers if I'm not mistaken (though I don't use animated gifs much). Thank you so much @cshum 🕺

[gingerlime]

Somehow it's not working for me though, but I need to dig a bit deeper ... I'll try to double check and post something reproducible

[jcupitt]

That's very impressive @cshum!

Any feelings re. performance? Will it be fast enough?

(I guess this is using the new libvips 8.12 gifsave operation, is that right?)

[gingerlime]

Sorry for the (somewhat) false alarm... I bumped into two issues:

1. We used a filter :format(jpeg), which thumbor ignores with animated gifs, but imagor doesn't ... I guess it's more accurate to freeze it into a jpeg, but if we're looking at compatibility, then I'm not sure :)
2. The watermark we use seems to be positioned differently than with thumbor... we're using watermark(img,-10,-10,0) which with thumbor seems to get positioned on the bottom right, but somehow with imagor it appears on the bottom middle? strictly speaking, thumbor ignores watermarks for animated gifs anyway, so it's not a "fair" comparison anyway, but I think this issue might appear with other image format? maybe a separate bug, or a difference in behaviour? (EDIT: see #18)

I'm just quickly hacking some stuff, but might need more time to extract sample images to reproduce. Hope I'll find more time in the next couple of days.

[cshum]

@jcupitt yes it is currently using vips 8.12.1. For simple resize I did try to utilize vips_thumbnail_buffer whenever possible.

vips thumbnail on animated gif works for resize that does not involve crop, also works for -s force. However when there is crop factor, the animation stops working:

vips thumbnail dancing-banana.gif[n=-1] test.gif 100 -h 200 -c high

Is this expected? For now I made a hack around the animated thumbnail function that,

loads image -> thumbnail resize -> extract area with arrayjoin
https://github.com/cshum/imagor/blob/master/processor/vipsprocessor/vips.go#L173
https://github.com/davidbyttow/govips/pull/259/files#diff-34586cc4ebe1265830eef27bd765045fb9818bc677870d9913f940100d42d7c8R42

which is obviously slower though.

[cshum]

@gingerlime the watermark issue is a regression on v0.8.0. Fixed now on v0.8.1 shumc/imagor:0.8.1 :).

For the animated gif feature, I don't use it either in my production work environment, which is why it wasn't implemented in the early iterations. However as we are now doing it, let's make it as complete as possible. While the animated version missed a bit of performance, it certainly has some really fun use cases. There is also a config variable to turn if off:

  -vips-max-animation-frames int
        VIPS maximum of animated image frames to be processed. Set 1 to disable animation, -1 for unlimited. (default -1)

[jcupitt]

Yes, I never tried to make the crop stuff in thumbnail work with many-page images. Perhaps it should.

[gingerlime]

Thanks again @cshum and @jcupitt. It works great now with the watermark fixed.

I still wonder a bit about the filter(jpeg) case and how imagor should behave. One the one hand, perhaps someone meant to convert an animated gif to a jpeg, and in this case, taking the first frame makes the most sense? on the other hand, maybe it's a kind of filter that cannot logically be applied and should therefore be ignored?

I'm not sure if thumbor is ignoring it deliberately or if it's some kind of a bug? and also not sure how far the compatibility path imagor wants to go. I understand it's not the aim to reach full compatibility.

[gingerlime]

... and yeah, just seeing the dancing banana with the watermark or background makes me smile :)

[cshum]

@gingerlime for thumbor behavior, I believe it is more of a bug? I try to resize a transparent png url, it converts to jpeg and dropped transparency. So the format(jpeg) does enforce an image type:

http://thumbor.thumborize.me/unsafe/fit-in/500x150/filters:format(jpeg)/https://raw.githubusercontent.com/cshum/imagor/master/testdata/gopher.png

[gingerlime]

I definitely don't know, but maybe they deliberately ignore format filters for (animated) gifs? it can be a (reasonable?) design choice ... we can perhaps ask the thumbor team, but still you can make your own design decisions of course. Just trying to say that there's a reasonable argument to decide that format filters can be ignored for animated gifs... it might not be the best decision, but it's not completely invalid.

[gingerlime]

in any case, imagor already works better with animated gifs than thumbor, so I'm super happy. I think we can close this one? unless there are some further improvements you want to work on...

[cshum]

Thanks @gingerlime @jcupitt !