Coder Social home page Coder Social logo

discord / lilliput Goto Github PK

View Code? Open in Web Editor NEW
1.9K 54.0 121.0 223.26 MB

Resize images and animated GIFs in Go

Home Page: https://discord.com/blog/how-discord-resizes-150-million-images-every-day-with-go-and-c

License: Other

C++ 59.33% Go 40.67%
resize-images gif golang cgo images imageops image resized-images thumbnail image-resizer

lilliput's Introduction

lilliput

lilliput resizes images in Go.

Lilliput relies on mature, high-performance C libraries to do most of the work of decompressing, resizing and compressing images. It aims to do as little memory allocation as possible and especially not to create garbage in Go. As a result, it is suitable for very high throughput image resizing services.

Lilliput supports resizing JPEG, PNG, WEBP and animated GIFs. It can also convert formats. Lilliput also has some support for getting the first frame from MOV and WEBM videos.

Lilliput presently only supports OSX and Linux.

Example

Lilliput comes with a fully working example that runs on the command line. The example takes a user supplied filename and prints some basic info about the file. It then resizes and transcodes the image (if flags are supplied) and saves the resulting file.

To use the example, go get github.com/discord/lilliput and then run go build from the examples/ directory.

License

Lilliput is released under MIT license (see LICENSE). Additionally, lilliput ships with other libraries, each provided under its own license. See third-party-licenses for more info.

Usage

First, import "github.com/discord/lilliput".

Decoder

Lilliput is concerned with in-memory images, so the decoder requires image data to be in a []byte buffer.

func lilliput.NewDecoder([]byte buf) (lilliput.Decoder, error)

Create a new Decoder object from the compressed image contained by buf. This will return an error when the magic bytes of the buffer don't match one of the supported image types.

func (d lilliput.Decoder) Header() (lilliput.ImageHeader, error)

Read and return the image's header. The header contains the image's metadata. Returns error if the image has a malformed header. An image with a malformed header cannot be decoded.

func (d lilliput.Decoder) Description() string

Returns a string describing the image's type, e.g. "JPEG" or "PNG".

func (h lilliput.Decoder) Duration() time.Duration

Returns the length of the content. Returns 0 for static images and animated GIFs.

func (d lilliput.Decoder) DecodeTo(f *lilliput.Framebuffer) error

Fully decodes the image and writes its pixel data to f. Returns an error if the decoding process fails. If the image contains multiple frames, then each call returns a subsequent frame. io.EOF is returned when the image does not contain any more data to be decoded.

Users of lilliput generally should not call DecodeTo and should instead use an ImageOps object.

func (d lilliput.Decoder) Close()

Closes the decoder and releases resources. The Decoder object must have .Close() called when it is no longer in use.

ImageOps

Lilliput provides a convenience object to handle image resizing and encoding from an open Decoder object. The ImageOps object can be created and then reused, which reduces memory allocations. Generally, users should prefer the ImageOps object over manually controlling the resize and encode process.

func lilliput.NewImageOps(dimension int) *lilliput.ImageOps

Create an ImageOps object that can operate on images up to dimension x dimension pixels in size. This object can be reused for multiple operations.

func (o *lilliput.ImageOps) Transform(decoder lilliput.Decoder, opts *lilliput.ImageOptions, dst []byte) ([]byte, error)

Transform the compressed image contained in a Decoder object into the desired output type. The decoder must not have DecodeTo() called on it already. However, it is ok to call decoder.Header() if you would like to check image properties before transforming the image. Returns an error if the resize or encoding process fails.

The resulting compressed image will be written into dst. The returned []byte slice will point to the same region as dst but with a different length, so that you can tell where the image ends.

Fields for lilliput.ImageOptions are as follows

  • FileType: file extension type, e.g. ".jpeg"

  • Width: number of pixels of width of output image

  • Height: number of pixels of height of output image

  • ResizeMethod: one of lilliput.ImageOpsNoResize or lilliput.ImageOpsFit. Fit behavior is the same as Framebuffer.Fit() -- it performs a cropping resize that does not stretch the image.

  • NormalizeOrientation: If true, Transform() will inspect the image orientation and normalize the output so that it is facing in the standard orientation. This will undo JPEG EXIF-based orientation.

  • EncodeOptions: Of type map[int]int, same options accepted as Encoder.Encode(). This controls output encode quality.

func (o *lilliput.ImageOps) Clear()

Clear out all pixel data contained in ImageOps object from any previous operations. This function does not need to be called between Transform() calls. The user may choose to do this if they want to remove image data from memory.

func (o *lilliput.ImageOps) Close()

Close the ImageOps object and release resources. The ImageOps object must have .Close() called when it is no longer in use.

ImageHeader

This interface returns basic metadata about an image. It is created by calling Decoder.Header().

func (h lilliput.ImageHeader) Width() int

Returns the image's width in number of pixels.

func (h lilliput.ImageHeader) Height() int

Returns the image's height in number of pixels.

func (h lilliput.ImageHeader) PixelType() lilliput.PixelType

Returns the basic pixel type for the image's pixels.

func (h lilliput.ImageHeader) Orientation() lilliput.ImageOrientation

Returns the metadata-based orientation of the image. This function can be called on all image types but presently only detects orientation in JPEG images. An orientation value of 1 indicates default orientation. All other values indicate some kind of rotation or mirroring.

PixelType

func (p lilliput.PixelType) Depth() int

Returns the number of bits per pixel.

func (p lilliput.PixelType) Channels() int

Returns the number of channels per pixel, e.g. 3 for RGB or 4 for RGBA.

Framebuffer

This type contains a raw array of pixels, decompressed from an image. In general, you will want to use the ImageOps object instead of operating on Framebuffers manually.

func lilliput.NewFramebuffer(width, height int) *lilliput.Framebuffer

Create a new Framebuffer with given dimensions without any pixel data.

func (f *lilliput.Framebuffer) Clear()

Set contents of framebuffer to 0, clearing out any previous pixel data.

func (f *lilliput.Framebuffer) Width() int

Returns the width in number of pixels of the contained pixel data, if any. This does not return the capacity of the buffer.

func (f *lilliput.Framebuffer) Height() int

Returns the height in number of pixels of the contained pixel data, if any. This does not return the capacity of the buffer.

func (f *lilliput.Framebuffer) PixelType() lilliput.PixelType

Returns the PixelType of the contained pixel data, if any.

func (f *lilliput.Framebuffer) OrientationTransform(orientation lilliput.ImageOrientation)

Rotate and/or mirror framebuffer according to orientation value. If you pass the orientation value given by the image's ImageHeader, then the resulting image has its orientation normalized to the default orientation.

func (f *lilliput.Framebuffer) ResizeTo(width, height int, dst *lilliput.Framebuffer) error

Perform a resize into dst of f according to given dimensions. This function does not preserve the source's aspect ratio if the new dimensions have a different ratio. The resize can fail if the destination is not large enough to hold the new image.

func (f *lilliput.Framebuffer) Fit(width, height int, dst *lilliput.Framebuffer) error

Perform a cropping resize into dst of f according to given dimensions. This function does preserve the source's aspect ratio. The image will be cropped along one axis if the new dimensions have a different ratio than the source. The cropping will occur equally on the edges, e.g. if the source image is too tall for the new ratio, then the destination will have rows of pixels from the top and bottom removed. Returns error if the destination is not large enough to contain the resized image.

func (f *lilliput.Framebuffer) Close()

Closes the framebuffer and releases resources. The Framebuffer object must have .Close() called when it is no longer in use.

Encoder

The Encoder takes a Framebuffer and writes the pixels into a compressed format.

func lilliput.NewEncoder(extension string, decodedBy lilliput.Decoder, dst []byte) (lilliput.Encoder, error)

Create a new Encoder object that writes to dst. extension should be a file extension-like string, e.g. ".jpeg" or ".png". decodedBy should be the Decoder used to decompress the image, if any. decodedBy may be left as nil in most cases but is required when creating a .gif encoder. That is, .gif outputs can only be created from source GIFs.

func (e lilliput.Encoder) Encode(buffer lilliput.Framebuffer, opts map[int]int) ([]byte, error)

Encodes the Framebuffer supplied into the output dst given when the Encoder was created. The returned []byte will point to the same buffer as dst but can be a shorter slice, so that if dst has 50MB of capacity but the image only occupies 30KB, you can tell where the image data ends. This function returns an error if the encoding process fails.

opts is optional and may be left nil. It is used to control encoder behavior e.g. map[int]int{lilliput.JpegQuality: 80} to set JPEG outputquality to 80.

Valid keys/values for opts are

  • JpegQuality (1 - 100)
  • PngCompression (0 - 9)
  • WebpQuality (0 - 100).
func (e lilliput.Encoder) Close()

Close the Encoder and release resources. The Encoder object must have .Close() called when it is no longer in use.

Building Dependencies

Go does not provide any mechanism for arbitrary building of dependencies, e.g. invoking make or cmake. In order to make lilliput usable as a standard Go package, prebuilt static libraries have been provided for all of lilliput's dependencies on Linux and OSX. In order to automate this process, lilliput ships with build scripts alongside compressed archives of the sources of its dependencies. These build scripts are provided for OSX and Linux.

lilliput's People

Contributors

alexrsagen avatar brian-armstrong-discord avatar discord-automation avatar eiz avatar iredelmeier avatar nxnfufunezn avatar samschlegel avatar skidder avatar slam avatar tpcstld avatar vishnevskiy avatar walruscow avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lilliput's Issues

recompile with -fPIC in debian

Using a golang docker image (which I believe is based on Debian), I get the errors described in #6. Presumably they are appearing again because of #45.

16:12 $ docker run -it golang:1.11 /bin/bash
root@46ba7a1e3c11:/go# ls
bin  src
root@46ba7a1e3c11:/go# go get github.com/discordapp/lilliput
# github.com/discordapp/lilliput
/usr/bin/ld: src/github.com/discordapp/lilliput/deps/linux/lib/libswscale.a(swscale.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: src/github.com/discordapp/lilliput/deps/linux/lib/libswscale.a(utils.o): relocation R_X86_64_32S against `.bss' can not be used when making a shared object; recompile with -fPIC

Here's my thoughts:
Things work fine in the latest centos docker image. After some research, it looks like recent versions of gcc (6+) provide a "-pie" option by default that requires shared libraries be compiled with "fPIC". I found that if I added -no-pie to all the #cgo linux LDFLAGS: lines (and added -no-pie to CGO_LDFLAGS_ALLOW) it would build just fine in the golang:1.11 image.

I suppose we could also downgrade the version of gcc too.

My question: What do you guys do? Do you have build recommendations? It would be nice to put some stuff in the README so that future users don't stumble like I did. Or am I missing something obvious?

Cross compile

Im having issues with cross compiling. Compiling normally on OSX works fine. However when i try the following it produces errors. Any ideas?

$ GOOS=linux go build
# github.com/discordapp/lilliput
../../discordapp/lilliput/lilliput.go:26: undefined: ImageHeader
../../discordapp/lilliput/lilliput.go:41: undefined: Framebuffer
../../discordapp/lilliput/lilliput.go:48: undefined: Framebuffer
../../discordapp/lilliput/ops.go:43: undefined: Framebuffer

Edit:
I get this error when building the deps for osx or linux.

checking for object file format of host system... Mach-O64
checking for object file format specifier (NAFLAGS) ... -fmacho64 -DMACHO -D__x86_64__
checking whether the assembler (nasm -fmacho64 -DMACHO -D__x86_64__) works... no
configure: error: installation or configuration problem: assembler cannot create object files.

Strange distortion/artifacts

Hi, this lib is the best for generating thumbnails that I tested so far.
Thanks for the great job! :)

Unfortunately I have few files that give strange results.
Probably it's my fault (file may be broken somehow) but in players it looks ok.
Maybe it's worth to look at it?

I tried turning on and off NormalizeOrientation and used all 3 ResizeMethod options in my code but that doesn't have any effect on this. As I tested - FFmpeg gets first frame correctly.

My setup:
Ubuntu Desktop 16.04 64bit
go version go1.9.2 linux/amd64

Steps to reproduce:

  1. Clone repo with commit c2b3884 (latest when I'm writing this)
  2. Get example .webm file to reproduce and unzip it 0d2cfdb1b7e04d66f4e4bf712932a03591696bbf515eb03d4a1a0663b97637d0.webm.zip
wget https://github.com/SystemZ/gotag/files/1514352/0d2cfdb1b7e04d66f4e4bf712932a03591696bbf515eb03d4a1a0663b97637d0.webm.zip
  1. Use examples/main.go to convert .webm to .png (or .jpg)
go run main.go -input test.webm -output test.png
  1. Thumbnail is not looking correctly

zxejh9d6cng5jl4c

Some .gifs are incorrectly "smearing"

Input Output
input output

To reproduce:

  1. go get github.com/discord/lilliput
  2. cd ~/go/src/github.com/discord/lilliput/examples
  3. go build
  4. ./examples -input input.gif -output output.gif

Alternatively, upload the input gif to Discord to reproduce.

Using GIMP to inspect the output gif, it seems like Lilliput is incorrectly outputting transparent pixels when it shouldn't be.

(The gif provided here is a minimal reproducible example, which is spliced from a longer gif - that's why there are transparent bars in the gif. I can upload the full gif if needed.)

Issue when building

I keep getting issues pertaining to the C libraries when trying to build this as part of my project.

I have used both go mod and glide as dependency management tools.


# github.com/discordapp/lilliput
--
208 | /usr/bin/ld: cannot find -lpng
209 | /usr/bin/ld: cannot find -lpng
210 | collect2: error: ld returned 1 exit status

Build command:

CGO_ENABLED=1 GOOS=linux go build -a -tags netgo -ldflags '-w' .

10 concurrent goroutines for resizing causes: Corrupt JPEG data: premature end of data segment (sometimes)

The following gist includes the code that runs the concurrent processes that resizes images that ranges from 8mb to 14mb all in jpg format:
https://gist.github.com/MohsenElgendy/5188013d269ccf9ffacf6902805efd8f

Steps the program do:

  1. Reads all files in a directory named _ ./bulk-in_.
  2. Sends goroutines for each file found.
  3. the method that the goroutine runs is basically the example provided by lilliput with the addition of execution time calculation.

and the images that outputs the error mentioned above are glitched or gray, the exact same source code is used with gocv and it works without producing this issue.

Information:

  • Processor: 3.2 GHz Intel Core i5
  • Memory: 8 GB 1600 MHz DDR3
  • OpenCV version is: 3.4.2
  • Go version is: 1.10.3

Any ideas would be appreciated, thanks!

Build failed: lilliput don't work with go mod

this is an error. when I use go mod and try to compile.

# github.com/discordapp/lilliput
/usr/bin/ld: cannot find -lpng
/usr/bin/ld: cannot find -lpng
collect2: error: ld returned 1 exit status

Statically linked binaries?

Hi, first off thanks a lot for making & open-sourcing this library! Extremely helpful, very much looking forward to seeing it perform in production!

Question: I am assuming it's not (easily) possible, but is there a way to build a static go binary from code that uses the lilliput library as a dependency?

Background of the question is that I'd love to have a nice and slim Docker image for running my image resizing server in production, and producing a statically linked binary would be an easy path there. Happy to hear about any other ways of accomplishing that!

Thanks!

More flexibility for ImageResize

As a user of this project, I'd like to be able to align my resized image to a specified direction relative to the original image.

This is handled in #35.

Memory management using Lilliput

Hi,

We are using lilliput as custom backend in picfit, it's a better way to handle gif than golang image standard library.

Since Lilliput handles memory using buffer, the allocated memory of the image server is increasing over time with the following config:

// DefaultMaxBufferSize is the maximum size of buffer for lilliput
const DefaultMaxBufferSize = 8192

// DefaultImageBufferSize is the default image buffer size for lilliput
const DefaultImageBufferSize = 50 * 1024 * 1024

memory

What approach do you recommend in this case?

Thank you

Image corruption with several workers running in parallel and randomly excessive memory usage

While playing with lilliput, I ran into the "strangely corrupted images" issue mentioned in How Discord Resizes 150 Million Images Every Day with Go and C++ on my MacBook Pro running macOS (not sure if things will be different under Linux).

My demo code (slightly modified version of examples/main.go@bab63a0) is here: https://github.com/discordapp/lilliput/compare/master...arthow4n:demo-excessive-memory-usage-and-corruption?expand=1

The corrupted images issue can be reproduced with following steps:

  1. Build the modified demo code with go build on a Mac
  2. ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output demo.jpg -width 300 -height 200 -runs 3
    • the Sample-jpg-image-10mb.jpg can be downloaded from http://www.sample-videos.com/img/Sample-jpg-image-10mb.jpg, I'm hosting it with a local file server with python -m SimpleHTTPServer solely for reducing transfer time.
    • Output (unexpected Corrupt JPEG data: premature end of data segment from OpenCV?):
      $ ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output demo.jpg -width 300 -height 200 -runs 3
      file type: JPEG
      7724px x 5148px
      file type: JPEG
      7724px x 5148px
      file type: JPEG
      7724px x 5148px
      Corrupt JPEG data: premature end of data segment
      Corrupt JPEG data: premature end of data segment
      image written to 0-demo.jpg
      image written to 2-demo.jpg
      image written to 1-demo.jpg
      
  3. When Corrupt JPEG data: premature end of data segment happens, at least one of the output images would be a corrupted one.
    • I cannot reproduce the issue with -runs 1 => ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output demo.jpg -width 300 -height 200 -runs 1

This can also be reproduced with -input with a tons of image bomb which trys to eat all memory in the system:
./examples -input Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 100 (or higher run)
But I don't know why it is easier to reproduce it (can happen with significatly lower -runs) with -remoteInput.

A normal output looks like this:
1-gg
While corrupted outputs look like this:
0-gg
2-gg

Also, sometimes the modified example program uses excessive memory with ops.Transform when pairing with -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -runs 1.

$ rm -f *-gg.jpg && /usr/bin/time -l ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 1
file type: JPEG
7724px x 5148px
image written to 0-gg.jpg
        0.68 real         0.50 user         0.10 sys
 175280128  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
     42836  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         1  messages sent
       538  messages received
         0  signals received
       976  voluntary context switches
      1406  involuntary context switches

After several runs of the exact same command, an unexpected high peak memory usage can be observed.

$ rm -f *-gg.jpg && /usr/bin/time -l ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 1
file type: JPEG
7724px x 5148px
image written to 0-gg.jpg
        0.97 real         0.65 user         0.29 sys
 651296768  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    159051  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         1  messages sent
       772  messages received
         0  signals received
       397  voluntary context switches
      1562  involuntary context switches

This can also be observed with -run 3 or more

$ rm -f *-gg.jpg && /usr/bin/time -l ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 3
file type: JPEG
7724px x 5148px
file type: JPEG
7724px x 5148px
file type: JPEG
7724px x 5148px
Corrupt JPEG data: premature end of data segment
Corrupt JPEG data: premature end of data segment
image written to 0-gg.jpg
image written to 2-gg.jpg
image written to 1-gg.jpg
        0.98 real         1.85 user         0.30 sys
 460050432  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    112338  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         3  messages sent
      2409  messages received
         0  signals received
      3507  voluntary context switches
      7663  involuntary context switches
$ rm -f *-gg.jpg && /usr/bin/time -l ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 3
file type: JPEG
7724px x 5148px
file type: JPEG
7724px x 5148px
file type: JPEG
7724px x 5148px
Corrupt JPEG data: premature end of data segment
image written to 0-gg.jpg
image written to 2-gg.jpg
image written to 1-gg.jpg
        2.21 real         2.16 user         0.99 sys
1902272512  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    464427  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         1  block output operations
         3  messages sent
      2005  messages received
         0  signals received
       940  voluntary context switches
     23701  involuntary context switches

Anyway, thanks for open sourcing this insanely fast library!

invalid flag in #cgo CFLAGS: -msse

If you are getting this message, it's because you're on a version of Go that forbids certain Cgo flags due to a recent security issue discovered in Go. Upgrading to Go 1.10 or newer will fix this issue, as the Go team relaxed the set of flags allowed after fixing the security issue

See golang/go#23749 for more details.

No Windows support

Could someone kindly outline the challenges with supporting windows? Whether it works on windows, but unsupported, or doesnt work and for what reasons? This would be a great help in assessing whether we can use this.

Getting error in building library

Hi I am using Lilliput to resize image. I have imported it like this

"github.com/discordapp/lilliput"
but I keep getting following error when building code.

#7 49.65 /root/go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libopencv_imgcodecs.a(grfmt_png.cpp.o): In function cv::PngDecoder::readData(cv::Mat&)': #7 49.65 grfmt_png.cpp:(.text._ZN2cv10PngDecoder8readDataERNS_3MatE+0x266): undefined reference to png_set_longjmp_fn'
#7 49.65 /root/go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libopencv_imgcodecs.a(grfmt_png.cpp.o): In function cv::PngDecoder::readHeader()': #7 49.65 grfmt_png.cpp:(.text._ZN2cv10PngDecoder10readHeaderEv+0x1e5): undefined reference to png_set_longjmp_fn'
#7 49.65 /root/go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libopencv_imgcodecs.a(grfmt_png.cpp.o): In function cv::PngEncoder::write(cv::Mat const&, cv::_InputArray const&)': #7 49.65 grfmt_png.cpp:(.text._ZN2cv10PngEncoder5writeERKNS_3MatERKNS_11_InputArrayE+0x385): undefined reference to png_set_longjmp_fn'

Cancelling image transformation

Sometimes we have very large images which take a long time to transform; we'd like to be able to cancel the transformation if a time limit is reached.

I've tried implementing this using time.After and running the image transformation in a separate goroutine. Then cancelling the goroutine if the timeout is reached; however this causes a runtime error:

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30d pc=0x7fff639a9d82]

runtime stack:
runtime.throw(0x4fec013, 0x2a)
        /usr/local/Cellar/go/1.13.3/libexec/src/runtime/panic.go:774 +0x72
runtime.sigpanic()
        /usr/local/Cellar/go/1.13.3/libexec/src/runtime/signal_unix.go:378 +0x47c

goroutine 83 [syscall]:
runtime.cgocall(0x44eed90, 0xc00027cdb8, 0xc00011d340)
        /usr/local/Cellar/go/1.13.3/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc00027cd88 sp=0xc00027cd50 pc=0x4004bdb
github.com/discordapp/lilliput._Cfunc_giflib_decoder_decode_frame_header(0x8b003e0, 0x0)
        _cgo_gotypes.go:303 +0x49 fp=0xc00027cdb8 sp=0xc00027cd88 pc=0x43e6229

Is there a graceful way to cancel a call to ImageOps.Transform?

resizing gif with custom transparency color breaks

If a gif has a custom transparency color, it will not be resized correctly. I have only tested this using Discord, so I suppose it's possible it's been fixed here already. I don't know how often you update the version you're using.

Example created using ImageMagick with -transparent-color #0A0295.

Original
original

Converted
converted

go get error

go get -u github.com/discord/lilliput@dbb0328
# github.com/discord/lilliput
../../../../pkg/mod/github.com/discord/[email protected]/lilliput.go:31:13: undefined: ImageHeader
../../../../pkg/mod/github.com/discord/[email protected]/lilliput.go:46:14: undefined: Framebuffer
../../../../pkg/mod/github.com/discord/[email protected]/lilliput.go:57:12: undefined: Framebuffer
../../../../pkg/mod/github.com/discord/[email protected]/lilliput.go:87:10: undefined: newGifDecoder
../../../../pkg/mod/github.com/discord/[email protected]/lilliput.go:90:23: undefined: newOpenCVDecoder
../../../../pkg/mod/github.com/discord/[email protected]/lilliput.go:95:9: undefined: newAVCodecDecoder
../../../../pkg/mod/github.com/discord/[email protected]/ops.go:50:16: undefined: Framebuffer
../../../../pkg/mod/github.com/discord/[email protected]/ops.go:66:30: undefined: Framebuffer
../../../../pkg/mod/github.com/discord/[email protected]/ops.go:70:33: undefined: Framebuffer
../../../../pkg/mod/github.com/discord/[email protected]/ops.go:119:53: undefined: ImageOrientation
../../../../pkg/mod/github.com/discord/[email protected]/lilliput.go:95:9: too many errors

PNG compression algorithm causing desaturation

It seems lilliput's algorithm for compressing PNGs is causing them to become noticeably desaturated compared to the original image. I also tested Google's Hangout's compression algorithm which didn't result in desaturation (incidentally lilliput's file size is also ~25% larger than Google's)

Examples

Original image (105 kb):
original-wintermelon

lilliput compression (via Discord) (95 kb) (notice how the red and green are desaturated vs the original):
discord-wintermelon

Google compression (via Google Hangouts) (76 kb):
gchat-wintermelon

PNGs with transparent borders have an artifact after resizing

We're having an issue with certain types of PNG images. For example, this 320x320 image of a tomato has a white background, but has transparent borders on the top and bottom of the image.

tomato.png

After resizing to 250x250 with lilliput we get this image

tomato_out

We get a visible dark line where the transparent borders begin on the original image. We've messed about with different PNG compression levels to no avail. Any ideas on how we can fix this?

Big speed difference on Mac vs CentOS

I've compiled the examples/main.go on both a MacBook Pro (Early 2015 i5) and CentOS (Amazon t2.large) and the OSX build is much faster

Mac

time ./main-osx --input=tlou2ps4pro1.jpg --output=test123.webp                                                                        10:40:01
file type: JPEG
3840px x 2160px
image written to test123.webp
        1.20 real         1.10 user         0.07 sys

t2.large

[ec2-user@ip-172-31-35-112 examples]$ time ./main --input=tlou2ps4pro1.jpg --output=test113.webp
file type: JPEG
3840px x 2160px
image written to test113.webp

real	0m8.039s
user	0m8.001s
sys	0m0.040s

Is there anything obvious I'm missing? I've tried the build-deps-linux.sh out of the box, as well as adding the following flags to the OpenCV cmake:

-DCMAKE_BUILD_TYPE=RELEASE -DWITH_TBB=ON -DWITH_IPP=ON -DWITH_V4L=ON -DENABLE_AVX=ON -DENABLE_SSSE3=ON -DENABLE_SSE41=ON -DENABLE_SSE42=ON -DENABLE_POPCNT=ON -DENABLE_FAST_MATH=ON -DBUILD_EXAMPLES=OFF -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF

But nothing is bringing it close to the Mac speed.

Cheers.

Nondeterministic corrupted jpeg -- mostly grey output image

When I Transform a jpeg (resize to a thumbnail) sometimes (5% or so of the time) the resulting image is a mostly grey output image ... often entirely grey. 95% of the time the same image succeeds in being resized.

There's usually an error code spit out to the logs when this happens: Corrupt JPEG data: 31283 extraneous bytes before marker 0xc4 however no error is returned from Transform

Here's the relevant code. Am I doing anything that I shouldn't be?

    inputBuf, err := ioutil.ReadAll(file)
    if err != nil {
        log.Printf("failed to read input file, %s\n", err)
        return err
    }

    decoder, err := lilliput.NewDecoder(inputBuf)
    if err != nil {
        log.Printf("error decoding image, %s\n", err)
        return err
    }
    defer decoder.Close()

    _, err = decoder.Header()
    if err != nil {
        log.Printf("error reading image header, %s\n", err)
        return err
    }

    ops := lilliput.NewImageOps(func(x, y int) int {
        if x > y {
            return x
        }
        return y
    }(image.Width, image.Height))
    defer ops.Close()

    width := int(float64(args.Height) * (float64(image.Width)/float64(image.Height)))

    opts := &lilliput.ImageOptions{
        FileType:             "." + image.Format,
        Width:                width,
        Height:               int(args.Height),
        ResizeMethod:         lilliput.ImageOpsResize,
        NormalizeOrientation: false,
        EncodeOptions:        map[string]map[int]int{
            "jpeg": map[int]int{lilliput.JpegQuality: 75},
            "png":  map[int]int{lilliput.PngCompression: 7},
            "webp": map[int]int{lilliput.WebpQuality: 75},
        }[image.Format],
    }

    // create a buffer to store the output image ... for some reason this
    // sometimes bigger than the input ...
    outputImg := make([]byte, len(inputBuf)*2)

    outputImg, err = ops.Transform(decoder, opts, outputImg)
    if err != nil {
        log.Printf("error transforming image, %s\n", err)
        return err
    }

Buffer too small

Hi team,

I've been encountering this issue when using the library: buffer too small to hold image. Is there a way to know how big of a buffer is needed or a way to bypass passing in a buffer?

Thanks!

Libraries in deps/linux/lib/ can not be used when making a shared object

Numerous errors arise when building the example main.go, such as:

/usr/bin/ld: src/github.com/discordapp/lilliput/deps/linux/lib/libavcodec.a(bsf.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: src/github.com/discordapp/lilliput/deps/linux/lib/libavcodec.a(vp8dsp_init.o): relocation R_X86_64_32S against hidden symbol `ff_put_vp8_pixels8_mmx' can not be used when making a shared object

Tested environments:

  • go1.7.4 linux/amd64 + gcc version 6.3.0 20170516 (Debian 6.3.0-18)
  • go1.9.2 linux/amd64 + gcc version 6.4.0 (Gentoo Hardened 6.4.0 p1.1)

Thanks.

The effect of Discord increasingly relying on rust on this project

Since Discord is increasingly relying on rust for performance critical use-cases, the question arises whether or not there are plans for lilliput to be oxidised as well.

Lilliput is also a very performance sensitive application.
Plus lilliput is a library, and calling go libs from other languages have a non-trivial overhead due to the runtime and the gc and all that.

https://blog.discord.com/why-discord-is-switching-from-go-to-rust-a190bbca2b1f

How to compile and deploy a go binary using lilliput

So I am working on an image resizer that runs on serverless on a lambda function.
When I run my go server on local machine using go run *.go the code works fine. But when I build a binary from the code I get errors related to see that "usr/bin/ld -lpng" not found

I am sure that lilliput uses some c bindings inorder to work with the image transformations but I am not able to get the way to properly build a single binary which I could deploy to my lambda function.

New release build

Hi lilliput team!

I was wondering when the next release version will be released? Currently, if I use go get..., my project works, but if I use dep ensure, it fails due to a C++ issue. I believe this was solved in one of the commits that got merged after v1.1. Could you release an update release to solve this?

Thanks!

nondeterministic libpng error: IDAT: unknown compression method

I occasionally get this error message libpng error: IDAT: unknown compression method in my logs, followed by this error returned by Transform failed to decode image ... This is after I've successfully called NewDecoder on the image bytes and after calling decoder.Header without error.

Strangely, when the background job is attempted again after a few seconds with the same image, Transform succeeds without error.

Could anyone offer any help about what might be happening? Relevant code:

    inputBuf, err := ioutil.ReadAll(file)
    if err != nil {
        log.Printf("failed to read input file, %s\n", err)
        return err
    }

    decoder, err := lilliput.NewDecoder(inputBuf)
    if err != nil {
        log.Printf("error decoding image, %s\n", err)
        return err
    }
    defer decoder.Close()

    _, err = decoder.Header()
    if err != nil {
        log.Printf("error reading image header, %s\n", err)
        return err
    }

    ops := lilliput.NewImageOps(func(x, y int) int {
        if x > y {
            return x
        }
        return y
    }(image.Width, image.Height))
    defer ops.Close()

    width := int(float64(args.Height) * (float64(image.Width)/float64(image.Height)))

    opts := &lilliput.ImageOptions{
        FileType:             "." + image.Format,
        Width:                width,
        Height:               int(args.Height),
        ResizeMethod:         lilliput.ImageOpsResize,
        NormalizeOrientation: false,
        EncodeOptions:        map[string]map[int]int{
            "jpeg": map[int]int{lilliput.JpegQuality: 75},
            "png":  map[int]int{lilliput.PngCompression: 7},
            "webp": map[int]int{lilliput.WebpQuality: 75},
        }[image.Format],
    }

    // create a buffer to store the output image ... for some reason this
    // sometimes bigger than the input ...
    outputImg := make([]byte, len(inputBuf)*2)

    outputImg, err = ops.Transform(decoder, opts, outputImg)
    if err != nil {
        log.Printf("error transforming image, %s\n", err)
        return err
    }

Difference ImageOpsFit vs ImageOpsResize

resizeMethod := lilliput.ImageOpsFit
	if stretch {
		resizeMethod = lilliput.ImageOpsResize
	}

In your example you have this bit of code.
When I used the example to understand how Lilliput works, I couldn't really see a difference in the output whether I used ImageOpsFit or ImageOpsResize

What is the difference supposed to be?

[Dep ensure problem]

Hi, I got this error when import the library from vendor folder.
`
make go-build-http
make[1]: Entering directory '/home/nakama/Documents/go_projects/src/github.com/tokopedia/uploadpedia'
github.com/tokopedia/uploadpedia/vendor/github.com/discordapp/lilliput

github.com/tokopedia/uploadpedia/vendor/github.com/discordapp/lilliput

In file included from ./avcodec.hpp:4:0,
from vendor/github.com/discordapp/lilliput/avcodec.go:12:
./opencv.hpp:7:10: fatal error: opencv2/core/fast_math.hpp: No such file or directory
#include <opencv2/core/fast_math.hpp>
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:12: recipe for target 'go-build-http' failed
make[1]: *** [go-build-http] Error 2
make[1]: Leaving directory '/home/nakama/Documents/go_projects/src/github.com/tokopedia/uploadpedia'
Makefile:22: recipe for target 'go-run-http' failed
make: *** [go-run-http] Error 2
`

Any Idea?

Thank you.

Can not build the lib

Hello there.
I am stack for a couple of days

can not build the lilliput under the alpine linux in docker

here is the full error : https://pastebin.com/ZZrK12Xh

this is what I fetching in container

RUN apk add --no-cache --virtual .fetch-deps \ libtirpc-dev \ libtirpc \ procps \ imlib2 \ gcc \ ca-certificates \ curl \ nano \ file \ ffmpeg \ gawk \ build-base \ linux-headers \ autoconf \ nasm \ cmake \ libtool \ automake

but still had no luck :(

is there anything else I need to do ?

Buffer too small to hold image

Hello,
https://gist.github.com/lookapanda/39577ec2880f3f8765fb6e20e793fb08

when I try to transform this image with Lilliput, I get the error message in the title.
I don't really know what's wrong with that image. My program does work with other base64 images.

buf, err := b64.StdEncoding.DecodeString(res.Image) // res.Image being the base64 string
// ...
img, err := ProcessImage(buf, 256, 256, ".png", false)
// ...
func ProcessImage(buf []byte, outputWidth int, outputHeight int, outputType string, stretch bool) ([]byte, error) {
	decoder, err := lilliput.NewDecoder(buf)
	if err != nil {
		fmt.Printf("error decoding image, %s\n", err)
		return nil, errors.New("error decoding image")
	}
	defer decoder.Close()

	header, err := decoder.Header()
	if err != nil {
		fmt.Printf("error reading image header, %s\n", err)
		return nil, errors.New("error reading image header")
	}

	if EncodeOptions[outputType] == nil {
		return nil, errors.New("invalid outputType")
	}

	ops := lilliput.NewImageOps(512)
	defer ops.Close()

	// create a buffer to store the output image, 50MB in this case
	outputImg := make([]byte, 0.5*1024*1024)

	if outputWidth == 0 {
		outputWidth = header.Width()
	}

	if outputHeight == 0 {
		outputHeight = header.Height()
	}

	resizeMethod := lilliput.ImageOpsFit
	if stretch {
		resizeMethod = lilliput.ImageOpsResize
	}

	opts := &lilliput.ImageOptions{
		FileType:             outputType,
		Width:                outputWidth,
		Height:               outputHeight,
		ResizeMethod:         resizeMethod,
		NormalizeOrientation: true,
		EncodeOptions:        EncodeOptions[outputType],
	}

	// resize and transcode image
	return ops.Transform(decoder, opts, outputImg)
}

It definitely happens inside the Transform method, the logs above are not the same error message.

edit: okay, I just checked it. It fails in the opencv.gofile:

newMat := C.opencv_mat_create_from_data(C.int(width), C.int(height), C.int(pixelType), unsafe.Pointer(&f.buf[0]), C.size_t(len(f.buf)))
	if newMat == nil {
		return ErrBufTooSmall
	}

I guess you can help me more since you've built this library ๐Ÿ“ฆ

Dockerfile Example

Is there an example of a Dockerfile or docker-compose file that can be used?

Couldnt build lilliput

`CGO_ENABLED=1 GOOS=linux \

go build -o ./bin/gouploader main.go
# github.com/discordapp/lilliput

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libavformat.a(mov.o): in function `snprintf':

/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libavformat.a(mov.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: more undefined references to `__snprintf_chk' follow

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libavformat.a(mov.o): in function `memcpy':

/usr/include/x86_64-linux-gnu/bits/string_fortified.h:34: undefined reference to `__memcpy_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libavformat.a(mov.o): in function `snprintf':

/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'`
...

`collect2: error: ld returned 1 exit status

# github.com/discordapp/lilliput

In file included from ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavutil/common.h:464,

                 from ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavutil/avutil.h:296,

                 from ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavutil/samplefmt.h:24,

                 from ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavcodec/avcodec.h:31,

                 from avcodec.cpp:7:

../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavutil/mem.h:341:79: warning: 'alloc_size' attribute ignored on a function returning 'int' [-Wattributes]
  341 | av_alloc_size(2, 3) int av_reallocp_array(void *ptr, size_t nmemb, size_t size);                                                                ^

make: *** [Makefile:13: build] Error 2`

I am using docker with alpine:3.12

RUN apk add --no-cache --virtual .fetch-deps \ ca-certificates \ make \ git \ wget \ which \ openssh-client \ libgcc \ musl-dev \ gcc \ g++

could someone point me to the right direction ?!

Build failed: fail to compile on macOS Mojave 10.14

Build is failing on macOS Mojave 10.14

# github.com/discordapp/lilliput
ld: file too small (length=10) file '/Users/sherifabdlnaby/go/pkg/mod/github.com/discordapp/[email protected]/deps/osx/lib/libpng.a' for architecture x86_64
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)

Compilation finished with exit code 2

I tried uninstalling Command Line Tools 10.14 and Install High Sierra's but still failing to build.
It build on High Sierra successfully.

when build errors

Linux version 3.10.0-1062.1.2.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Mon Sep 30 14:19:46 UTC 2019

when go get github.com/discordapp/lilliput

/root/go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libopencv_core.a(out.cpp.o):out.cpp:(.text._ZN2cv6detail12PtrOwnerImplINS_15MatlabFormatterENS_14DefaultDeleterIS2_EEED0Ev[_ZN2cv6detail12PtrOwnerImplINS_15MatlabFormatterENS_14DefaultDeleterIS2_EEED5Ev]+0x6): more undefined references to `operator delete(void*, unsigned long)' follow collect2: error: ld returned 1 exit status

error transforming image, Framebuffer contains no pixels. When putting decoder into struct

Firstly many thanks for open sourcing this library!

I do have an issue however. I'm wanting to read the file only once and pass the decoder and header into a struct for use later on.

The problem that I am having. Is that once an image gets processed the decoder inside the struct gets flushed and I get the error: "error transforming image, Framebuffer contains no pixels".

I have created a simple gist to show what I am doing:

https://gist.github.com/paulm17/4bd17334660101139a973921924a00b3

I really don't want to read the file for every resize at scale. What can I do here?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.