Support for WebP image format about dynmapcore

[rautamiekka]

^ Not everyone is gonna change. Like me, I'm not changing from Firefox for WebP, and such thinking is just wrong.

[nejento]

I'm definitely not forcing anyone to switch to Chrome nor Opera nor Vivaldi nor to anything else. I'm just pointing out that since there is a majority of users using Chromium based browser that there can be the option to use WebP as smaller sized high quality image format.

Not everyone is using dynmap as a public thing. Just as some kind of supervision or just for great renders of the world. For that and many more reasons is implementing WebP a good step. :)

[sachk]

I made a quick set of comparisons.

Flat Renders
Isometric Renders

To my eye, WEBP looks like much more than a 34% improvement and that's even with MozJPEG, which I think is quite a bit better than Java's built-in JPEG encoder.

I didn't bother comparing encoding speed since there's too many extra variables considering how small the results will be (PNG decompression, the analysis output cwebp prints, etc). I don't think that cwebp is slow enough to significantly decrease render times, but I could be wrong.

[bmarwell]

@mikeprimm do you accept pull requests? i saw the project has not been updated for a while.

[rhylos]

Yes he does.  On Friday, December 1, 2017, 10:27, Ben <[email protected]> wrote: @mikeprimm do you accept pull requests? i saw the project has not been updated for a while. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[bmarwell]

I thought that would be an easy one. Sadly, the library https://bitbucket.org/luciad/webp-imageio does require native libraries. I'm not sure shipping native libraries and need to configure them is wanted. :-(

[bmarwell]

I found out that there are Java classes provided by Google. I'll take a look into this issue soon.

[mikeprimm]

Support could be added - I'd just be concerned about the span of browser support: see https://caniuse.com/#feat=webp

[bmarwell]

@mikeprimm
Don't have time for tests right now, but can you please take a quick look?

https://github.com/bmhm/DynmapCore/commit/c94057707312ea198c871ba0dd9ddc289b0030e2

[bmarwell]

This is the actual commit for webp: https://github.com/bmhm/DynmapCore/commit/a0c256300ab988ae70e58c04f083bfea3c171be0

[bmarwell]

@nejento @rhylos @mikeprimm

while my commits will add native webp support, one might opt to use mod_pagespeed and nginx instead for performance. This way there will be both jpg and webp files generated (webp in nginx process) which are only served to capable clients.

Anyway, webp would still be interesting as soon as you configure lossless or another lossy compression factor and more browsers will support it.

[nejento]

Regarding the browser support, it's actually not that hot with the browser support for the WebP format when you keep in mind that Chrome (and Chromium based browsers) are widely used by most of the web users. And when you implement the dynmap into your website you can always create a warning about unsupported browser or you can have secondary map in lower quality in jpg-80 so that it doesn't take up that much space and users of other browsers still can see the map.

The main thing about WebP is especially saving the space when keeping the high resolution and thus saving the space on servers hostings which offer for example only 10 GB space for you map.

[bmarwell]

So, would you test and accept a pull request with the commits from above?? ^^

[mikeprimm]

I'd accept it as a format option (so long as the library support for encoding WebP is pure Java - not getting in to building and distributing JNI libraries for every platform in creation....). Definitely wouldn't be default, nor a suggested format option: I really don't think the browser support is broad enough, and while the gains exist they aren't compelling enough to impact user's browser selection (latest data puts Chrome at 55% overall, 60% of desktop - that's majority, but the unsupported browser users represent at least 1/3 of the likely user community) - and, if you wind up using something that just-in-time converts the webp images to jpeg or png for non-supporting browsers, you're adding a bunch of runtime compute load.
Most folks with size sensitivity just opt for the JPEG format with some dialing in on the quality (since this will save a lot - more than lossless WebP vs PNG- even at relatively high quality (the default for our jpeg format (85% quality) tends to result in 60+% file size reduction versus PNG, with almost no perceivable difference in appearance)). I don't doubt that, lossy vs lossy, WebP will outperform classic JPEG (on size, cannot say for encoding performance), but I'd take the relatively slow adoption by non-Google browsers as supporting the notion that it isn't exactly a level of advantage that is being found to be particularly compelling.

[bmarwell]

@mikeprimm
Pure Java is not possible yet. No one has published this with acceptable licence and/or performance. Good thing is: There is no need to compile C libraries. For those interested, just drop the .dll file or .so file into the java native lib path. That's it.

So, even with these restrictions -- why not give the user a choice? Is it our choice to make that the user may not use WebP, because it is not adopted widely yet? If that were true, we have a hen-and-egg-problem. Someone just has to start using it. If someone wants to use it and it works, why stop them from using it?

Just my 2 cents. I'd still be happy if someone could test it. I'd also volunteer to write a wiki page on how to use it (java native lib path plus configuring).

[sachk]

So what's happening with this - is this ever going to be properly implemented? It'd make Dynmap far more usable.

@mikeprimm @bmhm It took quite a few years to get here, but we've finally reached the point where virtually all browsers (including Firefox and Safari) support WEBP: https://caniuse.com/#search=webp

Any chance for official WEBP support in Dynmap to make use of these advancements?

[mikeprimm]

The biggest issue is we need either the JVM to support the encoding to the format, or a pure java implementation of such an encoder. Dynmap is used on LOTS of platforms - Windows, Linux (ARM and Intel), MacOS, and more - so incorporating platform and OS version sensitive native shared libraries is a problem. The browser support isn't really a concern, as support for the format would always be optional to the server user - but I'm not willing to field support requests every other week because the native library in question doesn't support 32 bit vs 64 bit linux, or glibc vs uclibc, or Debian vs Redhat versus Alpine, etc, etc. If there is a good ENCODING library that is pure java - no JNI or other native dependencies (e.g. running tools like GIMP on the command line), I'd be happy to add support.

[mikeprimm]

The biggest thing, honestly, is that the improvement in size is OK, but not something Earth shaking, so I'm not willing to add support load that I already don't have time to handle for the sake of '25-34% smaller size' (Google's claim - but versus the reference obsolete JPEG codec) or 'about 11% vs modern JPEG codecs' (per https://siipo.la/blog/is-webp-really-better-than-jpeg ).

[sachk]

The biggest thing, honestly, is that the improvement in size is OK, but not something Earth shaking, so I'm not willing to add support load that I already don't have time to handle for the sake of '25-34% smaller size' (Google's claim - but versus the reference obsolete JPEG codec) or 'about 11% vs modern JPEG codecs' (per https://siipo.la/blog/is-webp-really-better-than-jpeg ).

This doesn't match up to my experience with screenshots from Minecraft -from what I've observed the difference appears to be much greater for images with large areas of the same colour and non-contiguous repeating patterns. The compression performance difference and lack of Java support makes it seem a lot less practical though.

[bmarwell]

I agree, it is not worth it as long as there is no easy native java support. Also, https://bitbucket.org/luciad/webp-imageio/src/default/ hasn't been updated in years. Same with other libraries, like https://github.com/lonnyj/webp-imageio.

There is also a library from spreadshirt, which also requires building a native library. All do.

---

about

to handle for the sake of '25-34% smaller size' […] or 'about 11% vs modern JPEG codecs'

and

This doesn't match up to my experience with screenshots from Minecraft

Why don't you just convert all generated images and see for yourself if it is worth it?
https://developers.google.com/speed/webp/docs/img2webp

Maybe it is more than 30% better for tiles, maybe it isn't. We will not know if we dont try it.

[bmarwell]

// addendum
Maybe you want to open a JDK bug. Similar to this one?
https://bugs.openjdk.java.net/browse/JDK-8249136

Maybe it will be inlcuded in JDK17 if you hurry (which is the next LTS) =)

[mikeprimm]

Cool - the main point is still this: I'm not adding something that will create a problem with cross-platform support: we need to find a non-JNI/non-CLI-tool solution to encoding WEBP, because of the previously stated reasons (besides the additional ones of many hosting folks not allowing the addition of native libraries and the like). This position isn't going to change, as support for WEBP is a 'nice to have' but introducing something that makes folks who are successfully using Dynmap already not be able to continue functioning properly - whether that is them running on a Raspberry Pi, or running Alpine Linux in a docker container, or running on old Sparc hardware (all of these are currently among the cases across the 18000 servers currently running just the Spigot/Paper version).

On the timeline side, we do have some time - https://caniuse.com/#search=webp shows it being in the 'technology preview' for Safari14, but not in the latest release version (which is 13.1.2). Breaking Safari a non-starter for the feature to even be offered in Dynmap, and the lack of IE 11 support is also a problem.

[bmarwell]

Well, there's a solution for both problems.

1. Browsers

Just put a warning in and let the server admins decide. That's an easy one. They can't blame dynmap, because the documentation will have started this.

2. libraries

Well, you can ship the libs in a jar file, unpack them to a temporary directory at startup and load them in the runtime. In fact, I started a library project for this. I haven't completed it, but it's useable already. I can post the link later.
That will make the jni stuff easy to use. No setup required by server admins. No dependencies on a particular JVM or OS.

Webp library for jni would only needed to be compiled for each target system, but that's not too complicated.

[sachk]

There's also the option of using something browser side to expand WebP support, like this polyfill for instance https://github.com/chase-moskal/webp-hero. As for OS support, Alpine will run a static binary any other Linux distro of the same architecture will - problems arise if you try to run dynamically linked binaries that use glibc. I haven't used imageio before, but I'll take a look at this a bit later, and see if I might be able to integrate it in a way that doesn't require including static binaries for x86, x86-64, ARMv8, ARMhf etc directly in the JAR. https://github.com/qwong/j-webp

…

On Mon, 27 Jul 2020, 12:34 am mikeprimm, ***@***.***> wrote: Cool - the main point is still this: I'm not adding something that will create a problem with cross-platform support: we need to find a non-JNI/non-CLI-tool solution to encoding WEBP, because of the previously stated reasons (besides the additional ones of many hosting folks not allowing the addition of native libraries and the like). This position isn't going to change, as support for WEBP is a 'nice to have' but introducing something that makes folks who are successfully using Dynmap already not be able to continue functioning properly - whether that is them running on a Raspberry Pi, or running Alpine Linux in a docker container, or running on old Sparc hardware (all of these are currently among the cases across the 18000 servers currently running just the Spigot/Paper version). On the timeline side, we do have some time - https://caniuse.com/#search=webp shows it being in the 'technology preview' for Safari14, but not in the latest release version (which is 13.1.2). Breaking Safari a non-starter for the feature to even be offered in Dynmap, and the lack of IE 11 support is also a problem. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#82 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACN6UI4KOWDHPC65KGKALB3R5Q5FVANCNFSM4DN6CIVA> .

[bmarwell]

Here's the library I mentioned

https://github.com/bmhm/libloader