jayxon / leanify Goto Github PK
View Code? Open in Web Editor NEWlightweight lossless file minifier/optimizer
License: MIT License
lightweight lossless file minifier/optimizer
License: MIT License
The first is: error: ‘perror’ was not declared in this scope
I fixed it by adding "#include <stdio.h>" in fileio.h
The second is: formats/pe.cpp:376:25: error: ‘to_string’ is not a member of ‘std’
new_name += std::to_string(entry[i].Name);
Successfully compiled current master on Linux (8c1cc53).
It seems that the --keep-exif
option does not work for me? Or does color profile info not count? Or am I using it wrong.
$ cp -a source.jpg original.jpg
$ cp -a source.jpg reencoded.jpg
$ leanify --keep-exif reencoded.jpg
Processing: reencoded.jpg
3.14 MB -> 2.94 MB Leanified: 206.19 KB (6.40%)
$ file original.jpg reencoded.jpg
original.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 10000x7514, frames 3
reencoded.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, progressive, precision 8, 10000x7514, frames 3
$ exiftool -a -u -g1 original.jpg reencoded.jpg
======== original.jpg
---- ExifTool ----
ExifTool Version Number : 9.74
---- System ----
File Name : original.jpg
Directory : .
File Size : 3.1 MB
File Modification Date/Time : 2016:03:30 19:04:07+00:00
File Access Date/Time : 2016:04:25 16:55:04+00:00
File Inode Change Date/Time : 2016:04:25 16:53:47+00:00
File Permissions : rw-r--r--
---- File ----
File Type : JPEG
MIME Type : image/jpeg
Image Width : 10000
Image Height : 7514
Encoding Process : Baseline DCT, Huffman coding
Bits Per Sample : 8
Color Components : 3
Y Cb Cr Sub Sampling : YCbCr4:4:4 (1 1)
---- JFIF ----
JFIF Version : 1.01
Resolution Unit : None
X Resolution : 1
Y Resolution : 1
---- ICC-header ----
Profile CMM Type : ADBE
Profile Version : 2.1.0
Profile Class : Display Device Profile
Color Space Data : RGB
Profile Connection Space : XYZ
Profile Date Time : 2000:08:11 19:51:59
Profile File Signature : acsp
Primary Platform : Apple Computer Inc.
CMM Flags : Not Embedded, Independent
Device Manufacturer : none
Device Model :
Device Attributes : Reflective, Glossy, Positive, Color
Rendering Intent : Perceptual
Connection Space Illuminant : 0.9642 1 0.82491
Profile Creator : ADBE
Profile ID : 0
---- ICC_Profile ----
Profile Copyright : Copyright 2000 Adobe Systems Incorporated
Profile Description : Adobe RGB (1998)
Media White Point : 0.95045 1 1.08905
Media Black Point : 0 0 0
Red Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
Green Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
Blue Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
Red Matrix Column : 0.60974 0.31111 0.01947
Green Matrix Column : 0.20528 0.62567 0.06087
Blue Matrix Column : 0.14919 0.06322 0.74457
---- Composite ----
Image Size : 10000x7514
======== reencoded.jpg
---- ExifTool ----
ExifTool Version Number : 9.74
---- System ----
File Name : reencoded.jpg
Directory : .
File Size : 2.9 MB
File Modification Date/Time : 2016:04:25 16:54:49+00:00
File Access Date/Time : 2016:04:25 16:55:01+00:00
File Inode Change Date/Time : 2016:04:25 16:54:49+00:00
File Permissions : rw-r--r--
---- File ----
File Type : JPEG
MIME Type : image/jpeg
Image Width : 10000
Image Height : 7514
Encoding Process : Progressive DCT, Huffman coding
Bits Per Sample : 8
Color Components : 3
Y Cb Cr Sub Sampling : YCbCr4:4:4 (1 1)
---- JFIF ----
JFIF Version : 1.01
Resolution Unit : None
X Resolution : 1
Y Resolution : 1
---- Composite ----
Image Size : 10000x7514
2 image files read
$ leanify -h
Unknown option: h
Leanify 0.4.3
Usage: leanify [options] paths
-i, --iteration <iteration> More iterations produce better result, but
use more time, default is 15.
-d, --max_depth <max depth> Maximum recursive depth, unlimited by default.
Set to 1 will disable recursive minifying.
-f, --fastmode Fast mode, no recompression.
-q, --quiet No output to stdout.
-v, --verbose Verbose output.
--keep-exif Do not remove Exif.
It would be a great improvement if Leanify could utilise the swix console program http://appnee.com/swix/ to convert swf to xml and then optimize embedded base64 streams then restore it back to swf with same program then optimize compression .. you need to include mp3packer support to optimize audio streams
Issue summarized in another issue. Closing this one.
Leanify supports various kinds of image format except this. I think this format is common.
Microsoft Visual C++ Runtime Library
Runtime Error!
Program:
R6002
after hitting Details (Подробности) button
when i try to leanify xmplay.exe it crash.
subj
CBZ and CBT are used far less than CBR these days, it seems to be the primary format. Is support for RARs a possibility?
same goes to Nikkho`s zrecompress LZMA mode, I posted issue on encode.su
OpenDocument (.odt, .ods, .odp, .odb, .odg, .odf) contain images (.png, .jpg, .svg), are this optimized, or only ZIP container?
I obtain a:
not a valid Win32 application
in xp sp3
...by launching with any argument. I downloaded binary release, 0.4.3 and 0.4.2.
Manjaro 17.1.12 Hakoila
(gdb) run cryfs.png
Starting program: /home/user/Downloads/leanify cryfs.png
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
core.leanify.1000.65596381a2c44c2a84fe4ab6a6122586.32726.1538230669000000.zip
Process the test file with Leanify.exe -d 1 app-debug.apk
, and get the below msg and an broken .apk file.
Central directory header magic mismatch!
EOCD not found!
app-debug.zip(rename to .apk or just process it)
Tested with nightly build 3a29d22 win64/32
v0.4.3 win stable also failed, but only output EOCD not found!
Leanify removes spaces it shouldn't in the attached .pptx file. The title looks like this in the original:
And gets transformed to this with leanify on default settings:
This may be the result of poor formatting in the original document, but it is still a valid .pptx that was created by someone who was not specifically trying to find bugs in leanify.
The filenames can not contain ñ, á, é, í, ó, ú, ü, Etc.
The filenames can not start with -
Recompress FontTables
It's compressed with zlib.
Support WOFF in DataURI search
data:application/font-woff;charset=utf-8;base64,d09GRgABAAAAAU...
Remove metadata?
There's a comment in zip.cpp:218 regarding files using the store method: "don't try to change it to deflate, it might break some file"
Can you elaborate on this? I note that the advzip program does compress stored files.
As suggested by Jaff at http://encode.ru/threads/2036-Leanify?p=43423&viewfull=1#post43423
Would you like to add a small hack to optimise htm/html/css etc web based files wich contains base64 encoded images?
Detect base64 image, convert to binary file, optimise it, encode it back to base64 and replace original image in document.
It work with .png, .gif, .jpeg images. Thank you in advance!
some usefull info: http://en.wikipedia.org/wiki/Data_URI_scheme
How can I make use of all the CPU cores?
Is there any simple way?
Leanify.exe crash with RegistryChangesView.exe
Things to do:
it gives me:
g++ -Wall -O3 -msse2 -mfpmath=sse -flto --std=c++11 fileio.cpp utils.cpp leanify.cpp main.cpp formats/base64.cpp formats/ico.cpp formats/lua.cpp formats/dwf.cpp formats/swf.cpp formats/rdb.cpp formats/gft.cpp formats/xml.cpp formats/png.cpp formats/data_uri.cpp formats/pe.cpp formats/bmp.cpp formats/tar.cpp formats/jpeg.cpp formats/zip.cpp formats/gz.cpp lzma.a miniz.o mozjpeg.a tinyxml2.o zopfli.a zopflipng.a -s -o leanify
/tmp/cctGuFIz.ltrans0.ltrans.o: In function `Jpeg::Leanify(unsigned long)':
<artificial>:(.text+0x6d5e): undefined reference to `jpeg_std_error'
<artificial>:(.text+0x6d90): undefined reference to `jpeg_destroy_compress'
<artificial>:(.text+0x6da0): undefined reference to `jpeg_destroy_decompress'
<artificial>:(.text+0x6de3): undefined reference to `jpeg_CreateDecompress'
<artificial>:(.text+0x6ded): undefined reference to `jpeg_std_error'
<artificial>:(.text+0x6e15): undefined reference to `jpeg_CreateCompress'
<artificial>:(.text+0x6e4a): undefined reference to `jpeg_c_set_int_param'
<artificial>:(.text+0x6e63): undefined reference to `jpeg_mem_src'
<artificial>:(.text+0x6e83): undefined reference to `jpeg_save_markers'
<artificial>:(.text+0x6e95): undefined reference to `jpeg_read_header'
<artificial>:(.text+0x6ea2): undefined reference to `jpeg_read_coefficients'
<artificial>:(.text+0x6eba): undefined reference to `jpeg_copy_critical_parameters'
<artificial>:(.text+0x6f0a): undefined reference to `jpeg_mem_dest'
<artificial>:(.text+0x6f15): undefined reference to `jpeg_write_coefficients'
<artificial>:(.text+0x6f5e): undefined reference to `jpeg_write_marker'
<artificial>:(.text+0x6f79): undefined reference to `jpeg_finish_compress'
<artificial>:(.text+0x6f86): undefined reference to `jpeg_finish_decompress'
<artificial>:(.text+0x6f93): undefined reference to `jpeg_destroy_decompress'
<artificial>:(.text+0x6fca): undefined reference to `jpeg_destroy_compress'
/tmp/cctGuFIz.ltrans0.ltrans.o: In function `Swf::Leanify(unsigned long)':
<artificial>:(.text+0xd89e): undefined reference to `LzmaCompress'
<artificial>:(.text+0xda6e): undefined reference to `LzmaUncompress'
/tmp/cctGuFIz.ltrans0.ltrans.o: In function `Png::Leanify(unsigned long)':
<artificial>:(.text+0x10585): undefined reference to `ZopfliPNGOptions::ZopfliPNGOptions()'
<artificial>:(.text+0x1075c): undefined reference to `ZopfliPNGOptimize(std::vector<unsigned char, std::allocator<unsigned char> > const&, ZopfliPNGOptions const&, bool, std::vector<unsigned char, std::allocator<unsigned char> >*)'
collect2: error: ld returned 1 exit status
make: *** [Makefile:23: leanify] Error 1
on fedora x86-64 29. The git master builds fine. Please mint a new release.
Google Perceptual JPEG encoder
https://github.com/google/guetzli
Some docx embeds zip files inside OLEobjects that are not optimized by Leanify.
If you add an OLEobject codec, you should also support DOC and XLS old format.
The reason is TinyXML2
does not support xml:space="preserve"
.
<w:t xml:space="preserve"> </w:t>
will become
<w:t xml:space="preserve"/>
But the author of TinyXML2
is not willing to fix it.
Should I switch to another xml library?
It has been reported on FileOptimizer forums, that Leanify is corrupting APK files in some cases.
I have been able to reproduce it with daily binaries on both x86, and x64. Problem seems to be Leanify aborts with message "EOCD not found!" but does not generate an error errorlevel (it exits with code 0).
You have more details as well as sample files here: https://sourceforge.net/p/nikkhokkho/discussion/fileoptimizer/thread/74f4f5df/
Anyway, let me know if further details are needed.
subj
Here's one of my wedding photos. What happens when you try to Leanify it, please?
When I try, the orientation changes from portrait to landscape. I think this might be a bug, and the orientation should stay the same?
http://googledevelopers.blogspot.com/2013/02/compress-data-more-densely-with-zopfli.html
Typically used to re-encode ZIP & GZip, but Google uses it for their fonts for their Web Font API.
It is very slow (should be a flag to turn on), but does consistently make smaller archives.
Would you provide Win64 binaries for Leanify?
As reported on FileOptimizer discussion (http://sourceforge.net/p/nikkhokkho/discussion/fileoptimizer/thread/666f945a/#2f64)
Setup: Win7x86 with FileOptimizer v7.60.1252
Steps:
Optimize a large LibreOffice ODT (Open Text Document) file (tested on 898k and 130k files)
Open file
Defect:
http://i.imgur.com/wqV9QHP.png
OpenOffice 4.1.1 had the same response. LibreOffice (v4.3.5.2 and v4.3.7.2) prompted me to repair it and it came back ok.
Tested both with 32 and 64-bit versions and operating systems.
File compression was excellent: 898k -> 751k (or 84%). This is remarkable because it's even better than a 7-zip "ultra"-level compression of the same file, which cut less than 1% off the file size.
Leanify lib
mozjpeg 3.2
tinyxml2 4.0.1
LZMA 16.04
Controlled Length Out of Bounds Write
Using the corpus of test files from the previous crash I also included an XML file. The crash is relative to the XML library implementation of Leanify resulting in another issue with an OOB write independent of the fixed OOB Read/Write in ICO. I discovered this via fuzzing and did some light root cause analysis and another party heavily expanded on the exploitability of this issue. Details below:
A moderately controllable memory corruption vulnerability affects the implementation of xml_memory_writer in formats/xml.cpp:179 as shown below:
++
struct xml_memory_writer : pugi::xml_writer {
uint8_t* p_write;
void write(const void* data, size_t size) override {
memcpy(p_write, data, size); # Line 179
p_write += size;
}
};
} // namespace
pugi::xml_writer implementations should additionally track the capacity of the target buffer as shown in the pugixml documentation here: https://pugixml.org/docs/samples/save_custom_writer.cpp
It is possible to control the amount of data written past the end of the buffer by appending characters that require escaping as shown in the text_output_escaped function in pugixml.cpp:3906 as shown below:
++
PUGI__FN void text_output_escaped(xml_buffered_writer& writer, const char_t* s, chartypex_t type, unsigned int flags)
{
/* TRUNCATED */
switch (*s)
{
case 0: break;
case '&':
writer.write('&', 'a', 'm', 'p', ';');
++s;
break;
case '<':
writer.write('&', 'l', 't', ';');
++s;
break;
case '>':
writer.write('&', 'g', 't', ';');
++s;
break;
case '"':
writer.write('&', 'q', 'u', 'o', 't', ';');
++s;
break;
default: // s is not a usual symbol
{
unsigned int ch = static_cast(*s++);
assert(ch < 32);
if (!(flags & format_skip_control_chars))
writer.write('&', '#', static_cast<char_t>((ch / 10) + '0'), static_cast<char_t>((ch % 10) + '0'), ';');
}
/* TRUNCATED */
The following base64 test case will crash the target binary by expanding a number of & to the escaped & (a gain of 4 bytes in extension).
PHBlcnNvbiBpZD0iPgombmFtYW5nciBSb2FDL2RpdBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECYm
JiYmJiYmJiYmJiZ5JnkmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JyYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJkhFQVBfT1ZFUkZMT1cmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
JiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYm
EBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBBpaWlpaWlpaWlpaWlpaWlpaWlpaWlpaWlp
aWlpaWlpaWlpaWlpaWlpaWlpaWlpaWlsaWlpaWlpaWlpaWlpaWlpaWlpaWlpaWlpaWlpaXdpaWlp
aWlpEBgQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEGdhIFJvYUMvZGl0eT4KEBAQEBAQEBAQ
EBBpaWlpaWlpaWlpaWlpaWlDaXR5PkghbmdhYXN0ZXIgSXNhaWxzLiAsIj4KPC9wZXJzb24+Cg==
For full stack trace on the crash, see the ASAN output below:
=================================================================
==461==ERROR: AddressSanitizer: unknown-crash on address 0xb7fce000 at pc 0xb7af7a42 bp 0xbfffc8f8 sp 0xbfffc4cc
WRITE of size 518 at 0xb7fce000 thread T0
#0 0xb7af7a41 in __asan_memcpy (/usr/lib/i386-linux-gnu/libasan.so.2+0x8aa41)
#1 0xb7af7c2f in memcpy (/usr/lib/i386-linux-gnu/libasan.so.2+0x8ac2f)
#2 0x819a38c in (anonymous namespace)::xml_memory_writer::write(void const*, unsigned int) [clone .lto_priv.243] formats/xml.cpp:179
#3 0x80d927e in flush lib/pugixml/pugixml.cpp:3716
#4 0x80d9106 in pugi::impl::(anonymous namespace)::xml_buffered_writer::flush() [clone .lto_priv.561] lib/pugixml/pugixml.cpp:3705
#5 0x80bfed4 in pugi::xml_document::save(pugi::xml_writer&, char const*, unsigned int, pugi::xml_encoding) const lib/pugixml/pugixml.cpp:7168
#6 0x819b5fe in Xml::Leanify(unsigned int) formats/xml.cpp:307
#7 0x81af203 in LeanifyFile(void*, unsigned int, unsigned int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const
&) /root/Leanify/leanify.cpp:140
#8 0x81b08c2 in ProcessFile /root/Leanify/main.cpp:53
#9 0x81b1cde in main /root/Leanify/main.cpp:230
#10 0xb76e6636 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x18636)
#11 0x804b650 (/root/Leanify/leanify+0x804b650)
The testfile was generated using the AFL fuzzer by lcamtuf.
https://gist.github.com/nuomi1/1de7077fa1ca76125f1b
https://gist.github.com/anonymous/469034ba6df1be56dfc109d1871c6cc9
clang: error: invalid linker name in argument '-fuse-ld=gold'
make: *** [leanify] Error 1
It is documented that most metadata is removed from most formats, with the exception of keep-exif swith.
Would be nice having a global flag to not remove any metadata at all, or to selectively keep it as keep-iptc, ...
知乎上看到的 很不錯的小工具 给个赞!
批量处理几百张图片有点慢。。。。
i7 8700的性能发挥不出来呀
有些压缩成1MB多点 而有些6MB多 差异有点大?
这张压缩成6MB的图片直接扔到windows自带的画图工具另存为jpg 可以压缩到2MB
继续将这张2MB的图片扔到tinypng处理变成400K 我不指望能直接上400K 但也不至于8MB只压缩到了6MB
例子:https://imgchr.com/i/PG5dld
链接: https://pan.baidu.com/s/16lpqdHM41Ku8qIzg_Xn9Xw 密码: 4kza
这是被Leanify压缩后的 默认没有备份功能 所以没原图了
比如某个文件优化失败了可以跳过继续优化别的
Processing: D:\Data\Desktop\DCIM\100D5500\Leanify.exe
Open file error!
????????????????????????????????????????
Any chance you can add PDF to the file types? There are often multiple embedded images in pdfs which, I suspect, make up the bulk of the file size.
A issue reported on FileOptimizer have come after some investigation on my side to be releated to Leanify 0.4.2 x86 and x64 precompiled binaries.
The problem happens on at least GZ and PNG files.
Launching it from pure command-line (cmd.exe) works OK, and properly returns errorlevel, but when launched using CreateProcess, GetExitCodeProcess code always returns 259 (STILL_ACTIVE), and what is more, asking it for completion with WaitForSingleObject INFINITE it gets stuck waiting.
Details are here: https://sourceforge.net/p/nikkhokkho/discussion/general/thread/d04bc925/
subj
1.gz
It seems that Leanify does not support UNC path.
I found Leanify while doing some research and decided to use Radamsa to test the robustness of the application. It appears pretty solid. I grabbed a ton of test files and then ran this simple loop to generate some more files; for i in $(ls); do cat $i | radamsa; done.
I did note a few things that were interesting, the most of which was a crash noted by ASAN from mutated files that proved hard to debug. I've done the test case reduction already down from several hundred bytes to the smallest byte string I could find that still induced the crash.
I did note that by modifying the length of the input file, the resulting address of the crash could be changed. Note that in the reproduction steps below, amongst the output shows the READ attempt at crash time. Another interesting crash I noted was a WRITE of 1024 bytes in another crash file.
//Cleaning up stack traces.
I see it in source, but how it works with default settings? Is it applied for all files or not? How do I force it to be always used?
Feature request: Add an option to keep file modification date&time.
Reason: For some files (e.g. .png) no other way to store create/modification time than in file attributes (when change the file name is undesirably).
Effect is: Make lossless optimization real lossless: keep important file information on the place.
Minifying html, js and css would be a nice feature to add to Leanify.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.