Hi, I'm trying to run the example code on ubuntu 16.04 but getting

illegal instruction (core dumped)

error, when trying to install.Do I need to install the c dependencies

Hi, this lib is the best for generating thumbnails that I tested so far.
Thanks for the great job! :)

Unfortunately I have few files that give strange results.
Probably it's my fault (file may be broken somehow) but in players it looks ok.
Maybe it's worth to look at it?

I tried turning on and off NormalizeOrientation and used all 3 ResizeMethod options in my code but that doesn't have any effect on this. As I tested - FFmpeg gets first frame correctly.

My setup:
Ubuntu Desktop 16.04 64bit
go version go1.9.2 linux/amd64

Steps to reproduce:

1. Clone repo with commit c2b3884 (latest when I'm writing this)
2. Get example .webm file to reproduce and unzip it 0d2cfdb1b7e04d66f4e4bf712932a03591696bbf515eb03d4a1a0663b97637d0.webm.zip

wget https://github.com/SystemZ/gotag/files/1514352/0d2cfdb1b7e04d66f4e4bf712932a03591696bbf515eb03d4a1a0663b97637d0.webm.zip

3. Use examples/main.go to convert .webm to .png (or .jpg)

go run main.go -input test.webm -output test.png

4. Thumbnail is not looking correctly

Hi I am using Lilliput to resize image. I have imported it like this

"github.com/discordapp/lilliput"
but I keep getting following error when building code.

#7 49.65 /root/go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libopencv_imgcodecs.a(grfmt_png.cpp.o): In function cv::PngDecoder::readData(cv::Mat&)': #7 49.65 grfmt_png.cpp:(.text._ZN2cv10PngDecoder8readDataERNS_3MatE+0x266): undefined reference to png_set_longjmp_fn'
#7 49.65 /root/go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libopencv_imgcodecs.a(grfmt_png.cpp.o): In function cv::PngDecoder::readHeader()': #7 49.65 grfmt_png.cpp:(.text._ZN2cv10PngDecoder10readHeaderEv+0x1e5): undefined reference to png_set_longjmp_fn'
#7 49.65 /root/go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libopencv_imgcodecs.a(grfmt_png.cpp.o): In function cv::PngEncoder::write(cv::Mat const&, cv::_InputArray const&)': #7 49.65 grfmt_png.cpp:(.text._ZN2cv10PngEncoder5writeERKNS_3MatERKNS_11_InputArrayE+0x385): undefined reference to png_set_longjmp_fn'

Hi, first off thanks a lot for making & open-sourcing this library! Extremely helpful, very much looking forward to seeing it perform in production!

Question: I am assuming it's not (easily) possible, but is there a way to build a static go binary from code that uses the lilliput library as a dependency?

Background of the question is that I'd love to have a nice and slim Docker image for running my image resizing server in production, and producing a statically linked binary would be an easy path there. Happy to hear about any other ways of accomplishing that!

Thanks!

can you share some bbs discus group or telegram channel for this repo ?

Is there an example of a Dockerfile or docker-compose file that can be used?

Since recent updates to lilliput, I started seeing some weird corrupted gifs (includes attachments and animated emojis). Here is an example link to a gif that glitches in Discord (note it was recorded using ShareX tool, and there was no problem with it before):

https://i.alexflipnote.xyz/eed252.gif

I've compiled the examples/main.go on both a MacBook Pro (Early 2015 i5) and CentOS (Amazon t2.large) and the OSX build is much faster

Mac

time ./main-osx --input=tlou2ps4pro1.jpg --output=test123.webp                                                                        10:40:01
file type: JPEG
3840px x 2160px
image written to test123.webp
        1.20 real         1.10 user         0.07 sys

t2.large

[ec2-user@ip-172-31-35-112 examples]$ time ./main --input=tlou2ps4pro1.jpg --output=test113.webp
file type: JPEG
3840px x 2160px
image written to test113.webp

real	0m8.039s
user	0m8.001s
sys	0m0.040s

Is there anything obvious I'm missing? I've tried the build-deps-linux.sh out of the box, as well as adding the following flags to the OpenCV cmake:

-DCMAKE_BUILD_TYPE=RELEASE -DWITH_TBB=ON -DWITH_IPP=ON -DWITH_V4L=ON -DENABLE_AVX=ON -DENABLE_SSSE3=ON -DENABLE_SSE41=ON -DENABLE_SSE42=ON -DENABLE_POPCNT=ON -DENABLE_FAST_MATH=ON -DBUILD_EXAMPLES=OFF -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF

But nothing is bringing it close to the Mac speed.

Cheers.

If a gif has a custom transparency color, it will not be resized correctly. I have only tested this using Discord, so I suppose it's possible it's been fixed here already. I don't know how often you update the version you're using.

Example created using ImageMagick with -transparent-color #0A0295.

Original

Converted

`CGO_ENABLED=1 GOOS=linux \

go build -o ./bin/gouploader main.go

# github.com/discordapp/lilliput

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libavformat.a(mov.o): in function `snprintf':

/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libavformat.a(mov.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: more undefined references to `__snprintf_chk' follow

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libavformat.a(mov.o): in function `memcpy':

/usr/include/x86_64-linux-gnu/bits/string_fortified.h:34: undefined reference to `__memcpy_chk'

/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libavformat.a(mov.o): in function `snprintf':

/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'`
...

`collect2: error: ld returned 1 exit status

# github.com/discordapp/lilliput

In file included from ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavutil/common.h:464,

                 from ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavutil/avutil.h:296,

                 from ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavutil/samplefmt.h:24,

                 from ../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavcodec/avcodec.h:31,

                 from avcodec.cpp:7:

../go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/include/libavutil/mem.h:341:79: warning: 'alloc_size' attribute ignored on a function returning 'int' [-Wattributes]
  341 | av_alloc_size(2, 3) int av_reallocp_array(void *ptr, size_t nmemb, size_t size);                                                                ^

make: *** [Makefile:13: build] Error 2`

I am using docker with alpine:3.12

RUN apk add --no-cache --virtual .fetch-deps \ ca-certificates \ make \ git \ wget \ which \ openssh-client \ libgcc \ musl-dev \ gcc \ g++

could someone point me to the right direction ?!

Using a golang docker image (which I believe is based on Debian), I get the errors described in #6. Presumably they are appearing again because of #45.

16:12 $ docker run -it golang:1.11 /bin/bash
root@46ba7a1e3c11:/go# ls
bin  src
root@46ba7a1e3c11:/go# go get github.com/discordapp/lilliput
# github.com/discordapp/lilliput
/usr/bin/ld: src/github.com/discordapp/lilliput/deps/linux/lib/libswscale.a(swscale.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: src/github.com/discordapp/lilliput/deps/linux/lib/libswscale.a(utils.o): relocation R_X86_64_32S against `.bss' can not be used when making a shared object; recompile with -fPIC

Here's my thoughts:
Things work fine in the latest centos docker image. After some research, it looks like recent versions of gcc (6+) provide a "-pie" option by default that requires shared libraries be compiled with "fPIC". I found that if I added -no-pie to all the #cgo linux LDFLAGS: lines (and added -no-pie to CGO_LDFLAGS_ALLOW) it would build just fine in the golang:1.11 image.

I suppose we could also downgrade the version of gcc too.

My question: What do you guys do? Do you have build recommendations? It would be nice to put some stuff in the README so that future users don't stumble like I did. Or am I missing something obvious?

Numerous errors arise when building the example main.go, such as:

/usr/bin/ld: src/github.com/discordapp/lilliput/deps/linux/lib/libavcodec.a(bsf.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: src/github.com/discordapp/lilliput/deps/linux/lib/libavcodec.a(vp8dsp_init.o): relocation R_X86_64_32S against hidden symbol `ff_put_vp8_pixels8_mmx' can not be used when making a shared object

Tested environments:

• go1.7.4 linux/amd64 + gcc version 6.3.0 20170516 (Debian 6.3.0-18)
• go1.9.2 linux/amd64 + gcc version 6.4.0 (Gentoo Hardened 6.4.0 p1.1)

Thanks.

Could someone kindly outline the challenges with supporting windows? Whether it works on windows, but unsupported, or doesnt work and for what reasons? This would be a great help in assessing whether we can use this.

https://github.com/discordapp/lilliput/blob/c2b38847094c248c1dad51293c60bdbb9397a2a1/deps/build-deps-linux.sh#L79 requires file 0001-export-exif-orientation.patch which does not exist any more.

$BASEDIR should be $SRCDIR. There are other stuff in this directory that could be improved.

resizeMethod := lilliput.ImageOpsFit
	if stretch {
		resizeMethod = lilliput.ImageOpsResize
	}

In your example you have this bit of code.
When I used the example to understand how Lilliput works, I couldn't really see a difference in the output whether I used ImageOpsFit or ImageOpsResize

What is the difference supposed to be?

I keep getting issues pertaining to the C libraries when trying to build this as part of my project.

I have used both go mod and glide as dependency management tools.

# github.com/discordapp/lilliput
--
208 | /usr/bin/ld: cannot find -lpng
209 | /usr/bin/ld: cannot find -lpng
210 | collect2: error: ld returned 1 exit status

Build command:

CGO_ENABLED=1 GOOS=linux go build -a -tags netgo -ldflags '-w' .

Linux version 3.10.0-1062.1.2.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Mon Sep 30 14:19:46 UTC 2019

when go get github.com/discordapp/lilliput

/root/go/pkg/mod/github.com/discordapp/[email protected]/deps/linux/lib/libopencv_core.a(out.cpp.o):out.cpp:(.text._ZN2cv6detail12PtrOwnerImplINS_15MatlabFormatterENS_14DefaultDeleterIS2_EEED0Ev[_ZN2cv6detail12PtrOwnerImplINS_15MatlabFormatterENS_14DefaultDeleterIS2_EEED5Ev]+0x6): more undefined references to `operator delete(void*, unsigned long)' follow collect2: error: ld returned 1 exit status

Hello,
https://gist.github.com/lookapanda/39577ec2880f3f8765fb6e20e793fb08

when I try to transform this image with Lilliput, I get the error message in the title.
I don't really know what's wrong with that image. My program does work with other base64 images.

buf, err := b64.StdEncoding.DecodeString(res.Image) // res.Image being the base64 string
// ...
img, err := ProcessImage(buf, 256, 256, ".png", false)
// ...
func ProcessImage(buf []byte, outputWidth int, outputHeight int, outputType string, stretch bool) ([]byte, error) {
	decoder, err := lilliput.NewDecoder(buf)
	if err != nil {
		fmt.Printf("error decoding image, %s\n", err)
		return nil, errors.New("error decoding image")
	}
	defer decoder.Close()

	header, err := decoder.Header()
	if err != nil {
		fmt.Printf("error reading image header, %s\n", err)
		return nil, errors.New("error reading image header")
	}

	if EncodeOptions[outputType] == nil {
		return nil, errors.New("invalid outputType")
	}

	ops := lilliput.NewImageOps(512)
	defer ops.Close()

	// create a buffer to store the output image, 50MB in this case
	outputImg := make([]byte, 0.5*1024*1024)

	if outputWidth == 0 {
		outputWidth = header.Width()
	}

	if outputHeight == 0 {
		outputHeight = header.Height()
	}

	resizeMethod := lilliput.ImageOpsFit
	if stretch {
		resizeMethod = lilliput.ImageOpsResize
	}

	opts := &lilliput.ImageOptions{
		FileType:             outputType,
		Width:                outputWidth,
		Height:               outputHeight,
		ResizeMethod:         resizeMethod,
		NormalizeOrientation: true,
		EncodeOptions:        EncodeOptions[outputType],
	}

	// resize and transcode image
	return ops.Transform(decoder, opts, outputImg)
}

It definitely happens inside the Transform method, the logs above are not the same error message.

edit: okay, I just checked it. It fails in the opencv.gofile:

newMat := C.opencv_mat_create_from_data(C.int(width), C.int(height), C.int(pixelType), unsafe.Pointer(&f.buf[0]), C.size_t(len(f.buf)))
	if newMat == nil {
		return ErrBufTooSmall
	}

I guess you can help me more since you've built this library 📦

It seems lilliput's algorithm for compressing PNGs is causing them to become noticeably desaturated compared to the original image. I also tested Google's Hangout's compression algorithm which didn't result in desaturation (incidentally lilliput's file size is also ~25% larger than Google's)

Examples

Original image (105 kb):

lilliput compression (via Discord) (95 kb) (notice how the red and green are desaturated vs the original):

Google compression (via Google Hangouts) (76 kb):

Here is a few examples:
https://cdn.discordapp.com/attachments/155491653858230273/423000223615287316/21312.gif
https://cdn.discordapp.com/attachments/420995378582913030/431850348647415818/trelloderpy.gif

They look fine in the browser, but for an example. Try to paste this into Discord and they will start flickering.

Hi,
Is such global variables will be secure, if you will run e.g. tests in parallel, for code that using lilliput with -race flag?
https://github.com/discordapp/lilliput/blob/master/opencv.go#L25
Cuz I'm afraid here is race condition
thx.

Hello there.
I am stack for a couple of days

can not build the lilliput under the alpine linux in docker

here is the full error : https://pastebin.com/ZZrK12Xh

this is what I fetching in container

RUN apk add --no-cache --virtual .fetch-deps \ libtirpc-dev \ libtirpc \ procps \ imlib2 \ gcc \ ca-certificates \ curl \ nano \ file \ ffmpeg \ gawk \ build-base \ linux-headers \ autoconf \ nasm \ cmake \ libtool \ automake

but still had no luck :(

is there anything else I need to do ?

this is an error. when I use go mod and try to compile.

# github.com/discordapp/lilliput
/usr/bin/ld: cannot find -lpng
/usr/bin/ld: cannot find -lpng
collect2: error: ld returned 1 exit status

So I am working on an image resizer that runs on serverless on a lambda function.
When I run my go server on local machine using go run *.go the code works fine. But when I build a binary from the code I get errors related to see that "usr/bin/ld -lpng" not found

I am sure that lilliput uses some c bindings inorder to work with the image transformations but I am not able to get the way to properly build a single binary which I could deploy to my lambda function.

If you are getting this message, it's because you're on a version of Go that forbids certain Cgo flags due to a recent security issue discovered in Go. Upgrading to Go 1.10 or newer will fix this issue, as the Go team relaxed the set of flags allowed after fixing the security issue

See golang/go#23749 for more details.

Im having issues with cross compiling. Compiling normally on OSX works fine. However when i try the following it produces errors. Any ideas?

$ GOOS=linux go build
# github.com/discordapp/lilliput
../../discordapp/lilliput/lilliput.go:26: undefined: ImageHeader
../../discordapp/lilliput/lilliput.go:41: undefined: Framebuffer
../../discordapp/lilliput/lilliput.go:48: undefined: Framebuffer
../../discordapp/lilliput/ops.go:43: undefined: Framebuffer

Edit:
I get this error when building the deps for osx or linux.

checking for object file format of host system... Mach-O64
checking for object file format specifier (NAFLAGS) ... -fmacho64 -DMACHO -D__x86_64__
checking whether the assembler (nasm -fmacho64 -DMACHO -D__x86_64__) works... no
configure: error: installation or configuration problem: assembler cannot create object files.

Hi team,

I've been encountering this issue when using the library: buffer too small to hold image. Is there a way to know how big of a buffer is needed or a way to bypass passing in a buffer?

Thanks!

As a user of this project, I'd like to be able to align my resized image to a specified direction relative to the original image.

This is handled in #35.

Build is failing on macOS Mojave 10.14

# github.com/discordapp/lilliput
ld: file too small (length=10) file '/Users/sherifabdlnaby/go/pkg/mod/github.com/discordapp/[email protected]/deps/osx/lib/libpng.a' for architecture x86_64
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)

Compilation finished with exit code 2

I tried uninstalling Command Line Tools 10.14 and Install High Sierra's but still failing to build.
It build on High Sierra successfully.

We're having an issue with certain types of PNG images. For example, this 320x320 image of a tomato has a white background, but has transparent borders on the top and bottom of the image.

After resizing to 250x250 with lilliput we get this image

We get a visible dark line where the transparent borders begin on the original image. We've messed about with different PNG compression levels to no avail. Any ideas on how we can fix this?

Should at least check if input file exists and print a useful message

Input	Output

To reproduce:

1. go get github.com/discord/lilliput
2. cd ~/go/src/github.com/discord/lilliput/examples
3. go build
4. ./examples -input input.gif -output output.gif

Alternatively, upload the input gif to Discord to reproduce.

Using GIMP to inspect the output gif, it seems like Lilliput is incorrectly outputting transparent pixels when it shouldn't be.

(The gif provided here is a minimal reproducible example, which is spliced from a longer gif - that's why there are transparent bars in the gif. I can upload the full gif if needed.)

Since Discord is increasingly relying on rust for performance critical use-cases, the question arises whether or not there are plans for lilliput to be oxidised as well.

Lilliput is also a very performance sensitive application.
Plus lilliput is a library, and calling go libs from other languages have a non-trivial overhead due to the runtime and the gc and all that.

https://blog.discord.com/why-discord-is-switching-from-go-to-rust-a190bbca2b1f

when an image is resized it loses some saturation, see below image (left is resized, right is original)

Hi,

We are using lilliput as custom backend in picfit, it's a better way to handle gif than golang image standard library.

Since Lilliput handles memory using buffer, the allocated memory of the image server is increasing over time with the following config:

// DefaultMaxBufferSize is the maximum size of buffer for lilliput
const DefaultMaxBufferSize = 8192

// DefaultImageBufferSize is the default image buffer size for lilliput
const DefaultImageBufferSize = 50 * 1024 * 1024

What approach do you recommend in this case?

Thank you

Hi, I got this error when import the library from vendor folder.
`
make go-build-http
make[1]: Entering directory '/home/nakama/Documents/go_projects/src/github.com/tokopedia/uploadpedia'
github.com/tokopedia/uploadpedia/vendor/github.com/discordapp/lilliput

github.com/tokopedia/uploadpedia/vendor/github.com/discordapp/lilliput

In file included from ./avcodec.hpp:4:0,
from vendor/github.com/discordapp/lilliput/avcodec.go:12:
./opencv.hpp:7:10: fatal error: opencv2/core/fast_math.hpp: No such file or directory
#include <opencv2/core/fast_math.hpp>
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:12: recipe for target 'go-build-http' failed
make[1]: *** [go-build-http] Error 2
make[1]: Leaving directory '/home/nakama/Documents/go_projects/src/github.com/tokopedia/uploadpedia'
Makefile:22: recipe for target 'go-run-http' failed
make: *** [go-run-http] Error 2
`

Any Idea?

Thank you.

Could this library ever support compilation onto these platforms?

Firstly many thanks for open sourcing this library!

I do have an issue however. I'm wanting to read the file only once and pass the decoder and header into a struct for use later on.

The problem that I am having. Is that once an image gets processed the decoder inside the struct gets flushed and I get the error: "error transforming image, Framebuffer contains no pixels".

I have created a simple gist to show what I am doing:

https://gist.github.com/paulm17/4bd17334660101139a973921924a00b3

I really don't want to read the file for every resize at scale. What can I do here?

Thanks

#29
Steps to reproduce is using a image with a color profile.

While playing with lilliput, I ran into the "strangely corrupted images" issue mentioned in How Discord Resizes 150 Million Images Every Day with Go and C++ on my MacBook Pro running macOS (not sure if things will be different under Linux).

My demo code (slightly modified version of examples/main.go@bab63a0) is here: https://github.com/discordapp/lilliput/compare/master...arthow4n:demo-excessive-memory-usage-and-corruption?expand=1

The corrupted images issue can be reproduced with following steps:

1. Build the modified demo code with go build on a Mac
2. ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output demo.jpg -width 300 -height 200 -runs 3
   • the Sample-jpg-image-10mb.jpg can be downloaded from http://www.sample-videos.com/img/Sample-jpg-image-10mb.jpg, I'm hosting it with a local file server with python -m SimpleHTTPServer solely for reducing transfer time.
   • Output (unexpected Corrupt JPEG data: premature end of data segment from OpenCV?):

     $ ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output demo.jpg -width 300 -height 200 -runs 3
     file type: JPEG
     7724px x 5148px
     file type: JPEG
     7724px x 5148px
     file type: JPEG
     7724px x 5148px
     Corrupt JPEG data: premature end of data segment
     Corrupt JPEG data: premature end of data segment
     image written to 0-demo.jpg
     image written to 2-demo.jpg
     image written to 1-demo.jpg
     

3. When Corrupt JPEG data: premature end of data segment happens, at least one of the output images would be a corrupted one.
   • I cannot reproduce the issue with -runs 1 => ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output demo.jpg -width 300 -height 200 -runs 1

This can also be reproduced with -input with a tons of image bomb which trys to eat all memory in the system:
./examples -input Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 100 (or higher run)
But I don't know why it is easier to reproduce it (can happen with significatly lower -runs) with -remoteInput.

A normal output looks like this:

While corrupted outputs look like this:

Also, sometimes the modified example program uses excessive memory with ops.Transform when pairing with -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -runs 1.

$ rm -f *-gg.jpg && /usr/bin/time -l ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 1
file type: JPEG
7724px x 5148px
image written to 0-gg.jpg
        0.68 real         0.50 user         0.10 sys
 175280128  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
     42836  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         1  messages sent
       538  messages received
         0  signals received
       976  voluntary context switches
      1406  involuntary context switches

After several runs of the exact same command, an unexpected high peak memory usage can be observed.

$ rm -f *-gg.jpg && /usr/bin/time -l ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 1
file type: JPEG
7724px x 5148px
image written to 0-gg.jpg
        0.97 real         0.65 user         0.29 sys
 651296768  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    159051  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         1  messages sent
       772  messages received
         0  signals received
       397  voluntary context switches
      1562  involuntary context switches

This can also be observed with -run 3 or more

$ rm -f *-gg.jpg && /usr/bin/time -l ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 3
file type: JPEG
7724px x 5148px
file type: JPEG
7724px x 5148px
file type: JPEG
7724px x 5148px
Corrupt JPEG data: premature end of data segment
Corrupt JPEG data: premature end of data segment
image written to 0-gg.jpg
image written to 2-gg.jpg
image written to 1-gg.jpg
        0.98 real         1.85 user         0.30 sys
 460050432  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    112338  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         3  messages sent
      2409  messages received
         0  signals received
      3507  voluntary context switches
      7663  involuntary context switches

$ rm -f *-gg.jpg && /usr/bin/time -l ./examples -remoteInput http://127.0.0.1:8000/Sample-jpg-image-10mb.jpg -output gg.jpg -width 300 -height 200 -runs 3
file type: JPEG
7724px x 5148px
file type: JPEG
7724px x 5148px
file type: JPEG
7724px x 5148px
Corrupt JPEG data: premature end of data segment
image written to 0-gg.jpg
image written to 2-gg.jpg
image written to 1-gg.jpg
        2.21 real         2.16 user         0.99 sys
1902272512  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    464427  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         1  block output operations
         3  messages sent
      2005  messages received
         0  signals received
       940  voluntary context switches
     23701  involuntary context switches

Anyway, thanks for open sourcing this insanely fast library!

Hi lilliput team!

I was wondering when the next release version will be released? Currently, if I use go get..., my project works, but if I use dep ensure, it fails due to a C++ issue. I believe this was solved in one of the commits that got merged after v1.1. Could you release an update release to solve this?

Thanks!

There should be an option to enable progressive jpeg encoding as encoding options for jpeg.

The following gist includes the code that runs the concurrent processes that resizes images that ranges from 8mb to 14mb all in jpg format:
https://gist.github.com/MohsenElgendy/5188013d269ccf9ffacf6902805efd8f

Steps the program do:

1. Reads all files in a directory named _ ./bulk-in_.
2. Sends goroutines for each file found.
3. the method that the goroutine runs is basically the example provided by lilliput with the addition of execution time calculation.

and the images that outputs the error mentioned above are glitched or gray, the exact same source code is used with gocv and it works without producing this issue.

Information:

• Processor: 3.2 GHz Intel Core i5
• Memory: 8 GB 1600 MHz DDR3
• OpenCV version is: 3.4.2
• Go version is: 1.10.3

Any ideas would be appreciated, thanks!

Is there a way to to make lilliput work on centos 6, with libc 2.12 installed? Maybe build it in special way or something like that?

When I Transform a jpeg (resize to a thumbnail) sometimes (5% or so of the time) the resulting image is a mostly grey output image ... often entirely grey. 95% of the time the same image succeeds in being resized.

There's usually an error code spit out to the logs when this happens: Corrupt JPEG data: 31283 extraneous bytes before marker 0xc4 however no error is returned from Transform

Here's the relevant code. Am I doing anything that I shouldn't be?

    inputBuf, err := ioutil.ReadAll(file)
    if err != nil {
        log.Printf("failed to read input file, %s\n", err)
        return err
    }

    decoder, err := lilliput.NewDecoder(inputBuf)
    if err != nil {
        log.Printf("error decoding image, %s\n", err)
        return err
    }
    defer decoder.Close()

    _, err = decoder.Header()
    if err != nil {
        log.Printf("error reading image header, %s\n", err)
        return err
    }

    ops := lilliput.NewImageOps(func(x, y int) int {
        if x > y {
            return x
        }
        return y
    }(image.Width, image.Height))
    defer ops.Close()

    width := int(float64(args.Height) * (float64(image.Width)/float64(image.Height)))

    opts := &lilliput.ImageOptions{
        FileType:             "." + image.Format,
        Width:                width,
        Height:               int(args.Height),
        ResizeMethod:         lilliput.ImageOpsResize,
        NormalizeOrientation: false,
        EncodeOptions:        map[string]map[int]int{
            "jpeg": map[int]int{lilliput.JpegQuality: 75},
            "png":  map[int]int{lilliput.PngCompression: 7},
            "webp": map[int]int{lilliput.WebpQuality: 75},
        }[image.Format],
    }

    // create a buffer to store the output image ... for some reason this
    // sometimes bigger than the input ...
    outputImg := make([]byte, len(inputBuf)*2)

    outputImg, err = ops.Transform(decoder, opts, outputImg)
    if err != nil {
        log.Printf("error transforming image, %s\n", err)
        return err
    }

I occasionally get this error message libpng error: IDAT: unknown compression method in my logs, followed by this error returned by Transform failed to decode image ... This is after I've successfully called NewDecoder on the image bytes and after calling decoder.Header without error.

Strangely, when the background job is attempted again after a few seconds with the same image, Transform succeeds without error.

Could anyone offer any help about what might be happening? Relevant code:

    inputBuf, err := ioutil.ReadAll(file)
    if err != nil {
        log.Printf("failed to read input file, %s\n", err)
        return err
    }

    decoder, err := lilliput.NewDecoder(inputBuf)
    if err != nil {
        log.Printf("error decoding image, %s\n", err)
        return err
    }
    defer decoder.Close()

    _, err = decoder.Header()
    if err != nil {
        log.Printf("error reading image header, %s\n", err)
        return err
    }

    ops := lilliput.NewImageOps(func(x, y int) int {
        if x > y {
            return x
        }
        return y
    }(image.Width, image.Height))
    defer ops.Close()

    width := int(float64(args.Height) * (float64(image.Width)/float64(image.Height)))

    opts := &lilliput.ImageOptions{
        FileType:             "." + image.Format,
        Width:                width,
        Height:               int(args.Height),
        ResizeMethod:         lilliput.ImageOpsResize,
        NormalizeOrientation: false,
        EncodeOptions:        map[string]map[int]int{
            "jpeg": map[int]int{lilliput.JpegQuality: 75},
            "png":  map[int]int{lilliput.PngCompression: 7},
            "webp": map[int]int{lilliput.WebpQuality: 75},
        }[image.Format],
    }

    // create a buffer to store the output image ... for some reason this
    // sometimes bigger than the input ...
    outputImg := make([]byte, len(inputBuf)*2)

    outputImg, err = ops.Transform(decoder, opts, outputImg)
    if err != nil {
        log.Printf("error transforming image, %s\n", err)
        return err
    }

Sometimes we have very large images which take a long time to transform; we'd like to be able to cancel the transformation if a time limit is reached.

I've tried implementing this using time.After and running the image transformation in a separate goroutine. Then cancelling the goroutine if the timeout is reached; however this causes a runtime error:

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30d pc=0x7fff639a9d82]

runtime stack:
runtime.throw(0x4fec013, 0x2a)
        /usr/local/Cellar/go/1.13.3/libexec/src/runtime/panic.go:774 +0x72
runtime.sigpanic()
        /usr/local/Cellar/go/1.13.3/libexec/src/runtime/signal_unix.go:378 +0x47c

goroutine 83 [syscall]:
runtime.cgocall(0x44eed90, 0xc00027cdb8, 0xc00011d340)
        /usr/local/Cellar/go/1.13.3/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc00027cd88 sp=0xc00027cd50 pc=0x4004bdb
github.com/discordapp/lilliput._Cfunc_giflib_decoder_decode_frame_header(0x8b003e0, 0x0)
        _cgo_gotypes.go:303 +0x49 fp=0xc00027cdb8 sp=0xc00027cd88 pc=0x43e6229

Is there a graceful way to cancel a call to ImageOps.Transform?