kjn / lbzip2 Goto Github PK
View Code? Open in Web Editor NEWParallel bzip2 utility
License: GNU General Public License v3.0
Parallel bzip2 utility
License: GNU General Public License v3.0
Formally present gnulib is removed, which triggers build errors. Is this intended?
lbzip2-2.3/src/common.h:33:48: fatal error: xalloc.h: No such file or directory
#include "xalloc.h" /* XMALLOC() */
^
We can specify the thread number to be applied with -n
. However there is no upper bound limit for it. If the value is huge, lbzip2 will report unable to create a POSIX thread: Resource temporarily unavailable
and hang there.
We think it's reasonable to have some upper bound check for the thread number.
Hello there!
Upcoming changes in glibc 2.28 broke the package.
https://koji.fedoraproject.org/koji/taskinfo?taskID=28542910
https://kojipkgs.fedoraproject.org//work/tasks/2910/28542910/build.log
gcc -DHAVE_CONFIG_H -I. -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -c -o printf-args.o printf-args.c
fseterr.c: In function 'fseterr':
fseterr.c:77:3: error: #error "Please port gnulib fseterr.c to your platform! Look at the definitions of ferror and clearerr on your system, then report this to bug-gnulib."
#error "Please port gnulib fseterr.c to your platform! Look at the definitions of ferror and clearerr on your system, then report this to bug-gnulib."
^~~~~
make[3]: *** [Makefile:1474: fseterr.o] Error 1
make[3]: *** Waiting for unfinished jobs....
Thanks for help.
When I'm not the owner of a file and when using lbzip2 as a compression tool, the file permissions get changed! Can this be fixed?
Dear,
Using lbzip2 from the command line fails when adding either -v
or --verbose
to the command. In that case, it makes a bz2 file, but in there, there is only a .cpgz file to be found. The command line throws out the following error: Illegal instruction: 4
.
This is on High Sierra with an iMac 21.5-inch, Mid 2011 with an 2,8 GHz Intel Core i7.
I tried to compile it myself and to install it using homebrew with no difference whatsoever.
The same iMac running on Yosemite does the job as it should, so I suppose this problem is due to an instruction that High Sierra doesn't understand.
Any way to fix this?
Thank you in advance,
Erwin
Some bz2 decoders (at least 7-Zip) may enforce Kraft equality. Lets make sure that dummy trees (generated in case only one tree is used in a block) are complete.
On my gentoo system with lbzip2-2.0 I get an segf for
lbzip2 -d -c -- automake-1.11.1.tar.bz2 > automake-1.11.1.tar
(gdb) set args -d -c -- automake-1.11.1.tar.bz2 > asd
(gdb) run
Starting program: /usr/bin/lbzip2 -d -c -- automake-1.11.1.tar.bz2 > asd
[Thread debugging using libthread_db enabled]
[New Thread 0x2aaaab698700 (LWP 32031)]
[New Thread 0x2aaaab899700 (LWP 32032)]
[New Thread 0x2aaaaba9a700 (LWP 32033)]
[Thread 0x2aaaab899700 (LWP 32032) exited]
[New Thread 0x2aaaabd9c700 (LWP 32034)]
[New Thread 0x2aaab0200700 (LWP 32035)]
[New Thread 0x2aaab05cf700 (LWP 32036)]
[New Thread 0x2aaab07d0700 (LWP 32037)]
[New Thread 0x2aaab09d1700 (LWP 32038)]
[New Thread 0x2aaab0bd2700 (LWP 32039)]
[New Thread 0x2aaab0dd3700 (LWP 32040)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x2aaaaba9a700 (LWP 32033)]
work_retrieve (ispec=0x7fffffffc5a0, w2m_q=, sw2w_q=0x2aaaab697c40, ibits_left=8, ibitbuf=134217728, ipos=4528, s2w_blk=0x2aaaabd9d010) at lbunzip2.c:1118
1118 lbunzip2.c: No such file or directory.
in lbunzip2.c
(gdb) bt
#0 work_retrieve (ispec=0x7fffffffc5a0, w2m_q=, sw2w_q=0x2aaaab697c40, ibits_left=8, ibitbuf=134217728, ipos=4528, s2w_blk=0x2aaaabd9d010) at lbunzip2.c:1118
#1 work_scan (ispec=0x7fffffffc5a0, w2m_q=, sw2w_q=0x2aaaab697c40, s2w_blk=) at lbunzip2.c:1316
#2 work (ispec=0x7fffffffc5a0, w2m_q=, sw2w_q=0x2aaaab697c40) at lbunzip2.c:1330
#3 work_wrap (v_work_arg=) at lbunzip2.c:1359
#4 0x00002aaaaaedff1c in start_thread () from /lib64/libpthread.so.0
#5 0x00002aaaab1d921d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
version 0.23 is fine.
System uname: Linux-3.1.1-lh-x86_64-Intel-R-_Core-TM-i7_CPU_860@_2.80GHz-with-gentoo-2.1
Timestamp of tree: Wed, 23 Nov 2011 07:00:01 +0000
ccache version 3.1.6 [enabled]
app-shells/bash: 4.2_p10
dev-java/java-config: 2.1.11-r3
dev-lang/python: 2.5.4-r4, 2.6.7-r2, 2.7.2-r3, 3.1.4-r3, 3.2.2
dev-util/ccache: 3.1.6
dev-util/cmake: 2.8.6-r3
dev-util/pkgconfig: 0.26
sys-apps/baselayout: 2.1
sys-apps/openrc: 0.9.4
sys-apps/sandbox: 2.5
sys-devel/autoconf: 2.13, 2.68
sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1-r1
sys-devel/binutils: 2.22
sys-devel/gcc: 4.2.4-r1, 4.3.6-r1, 4.4.6-r1, 4.5.3-r1, 4.6.2
sys-devel/gcc-config: 1.5-r2
sys-devel/libtool: 2.4.2
sys-devel/make: 3.82-r3
sys-kernel/linux-headers: 2.6.39 (virtual/os-headers)
sys-libs/glibc: 2.14.1
I was wondering, if it would be possible to use the library under the terms of the LGPL instead of the GPL for open source projects.
Trouble is, that I don't want to use GPL, but more like BSD as a license for my stuff and I'm worried, that I had to relicense the code according to GPL when using this library. Thus, I'm trying to find out, if dynamic linking and LGPL would work.
host lbzip2.org
Host lbzip2.org not found: 3(NXDOMAIN)
Has been this way for a few days at least. Perhaps host the bootstrapped tarball elsewhere?
Using the parameter -v
causes lbzip2 to crash for me. Weirdly after that happens, any call to lbzip2 crashes for a while. After a few minutes (and doing something else) it works again.
$ lbzip2 -v README.md
[1] 80520 illegal hardware instruction lbzip2 -v README.md
$ lbzip2 README.md
[1] 80755 illegal hardware instruction lbzip2 README.md
This is on lbzip2 2.5 installed from Homebrew. It's built with ./configure --disable-debug --disable-dependency-tracking
in case that matters.
Here's the crash log:
Process: lbzip2 [80520]
Path: /usr/local/Cellar/lbzip2/2.5/bin/lbzip2
Identifier: lbzip2
Version: 0
Code Type: X86-64 (Native)
Parent Process: zsh [70006]
Responsible: lbzip2 [80520]
User ID: 501
Date/Time: 2018-04-25 12:44:04.527 +0300
OS Version: Mac OS X 10.13.2 (17C88)
Report Version: 12
Bridge OS Version: 3.0 (14Y661)
Anonymous UUID: E44E2E84-CDBC-B3B7-16B1-EB8100EA5BA1
Sleep/Wake UUID: 06491AE0-82A8-407A-97AF-E1E3867DB275
Time Awake Since Boot: 1900000 seconds
Time Since Wake: 82000 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes: 0x0000000000000001, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Illegal instruction: 4
Termination Reason: Namespace SIGNAL, Code 0x4
Terminating Process: exc handler [0]
Application Specific Information:
%n used in a non-immutable format string
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_c.dylib 0x00007fff6e1f3a0b __vfprintf + 16437
1 libsystem_c.dylib 0x00007fff6e218091 __v2printf + 473
2 libsystem_c.dylib 0x00007fff6e1fd4af _vsnprintf + 415
3 libsystem_c.dylib 0x00007fff6e1fd50c vsnprintf_l + 41
4 libsystem_c.dylib 0x00007fff6e1ee4d4 snprintf + 180
5 lbzip2 0x000000010dc5a966 vasnprintf + 3094
6 lbzip2 0x000000010dc58d55 rpl_fprintf + 185
7 lbzip2 0x000000010dc551f4 log_generic + 61
8 lbzip2 0x000000010dc5518c info + 170
9 lbzip2 0x000000010dc576a8 work + 97
10 lbzip2 0x000000010dc56642 main + 2767
11 libdyld.dylib 0x00007fff6e15c115 start + 1
Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x0000000000000000 rcx: 0x0000050000000000 rdx: 0x0000040000000503
rdi: 0x0000000000012068 rsi: 0x0000000000000500 rbp: 0x00007ffee1fb59f0 rsp: 0x00007ffee1fb5670
r8: 0x00007fffa7245da0 r9: 0x0000000000000040 r10: 0x00007fffa7245d98 r11: 0xffffffffffffffff
r12: 0x00007ffee1fb5dc4 r13: 0x0000000000000000 r14: 0x0000000000000006 r15: 0x00007ffee1fb61f8
rip: 0x00007fff6e1f3a0b rfl: 0x0000000000010202 cr2: 0x000000010dc7c000
Logical CPU: 4
Error Code: 0x00000000
Trap Number: 6
Binary Images:
0x10dc48000 - 0x10dc61fff +lbzip2 (0) <A2637AA1-9432-3466-B680-E2313C060313> /usr/local/bin/lbzip2
0x1120ed000 - 0x11213798f dyld (519.2.2) <6695F30B-4E88-3C0B-9867-7D738C44A3E6> /usr/lib/dyld
0x7fff6ba45000 - 0x7fff6ba78fff libclosured.dylib (519.2.2) <48051216-5647-3643-B979-B77D0FD20011> /usr/lib/closure/libclosured.dylib
0x7fff6bf57000 - 0x7fff6bf58ff3 libSystem.B.dylib (1252) <5EDEEA85-0387-31EE-9D9B-DA1B75E7AEF3> /usr/lib/libSystem.B.dylib
0x7fff6c18b000 - 0x7fff6c1e1fff libc++.1.dylib (400.9) <FCF5E1F6-2B04-3545-8004-F3AB32FED172> /usr/lib/libc++.1.dylib
0x7fff6c1e2000 - 0x7fff6c206ff7 libc++abi.dylib (400.7) <217656D5-BC40-37FF-B322-91CB2AAD4F34> /usr/lib/libc++abi.dylib
0x7fff6d558000 - 0x7fff6d9467e7 libobjc.A.dylib (723) <93A92316-DE1E-378C-8891-99720B50D075> /usr/lib/libobjc.A.dylib
0x7fff6dff3000 - 0x7fff6dff7ff7 libcache.dylib (80) <354F3B7D-404E-3398-9EBF-65CA2CE65211> /usr/lib/system/libcache.dylib
0x7fff6dff8000 - 0x7fff6e002ff3 libcommonCrypto.dylib (60118.30.2) <674286D3-7744-36A3-9AAA-49DFCD97A986> /usr/lib/system/libcommonCrypto.dylib
0x7fff6e003000 - 0x7fff6e00afff libcompiler_rt.dylib (62) <4487CFBA-A5D7-3282-9E6B-94CAD7BE507E> /usr/lib/system/libcompiler_rt.dylib
0x7fff6e00b000 - 0x7fff6e013ffb libcopyfile.dylib (146.30.2) <2C7C67D7-562B-3FFA-973D-BACF4C10E1EC> /usr/lib/system/libcopyfile.dylib
0x7fff6e014000 - 0x7fff6e099fff libcorecrypto.dylib (562.30.10) <8A53EFE1-AFCA-3676-BEE1-FA5ED9F0E222> /usr/lib/system/libcorecrypto.dylib
0x7fff6e121000 - 0x7fff6e15aff7 libdispatch.dylib (913.30.4) <7D0E3183-282B-3FEE-A734-2C0ADC092084> /usr/lib/system/libdispatch.dylib
0x7fff6e15b000 - 0x7fff6e178ff7 libdyld.dylib (519.2.2) <C50D02BC-A333-3313-B787-02F255A6783F> /usr/lib/system/libdyld.dylib
0x7fff6e179000 - 0x7fff6e179ffb libkeymgr.dylib (28) <6D84A96F-C65B-38EC-BDB5-21FD2C97E7B2> /usr/lib/system/libkeymgr.dylib
0x7fff6e187000 - 0x7fff6e187ff7 liblaunch.dylib (1205.30.29) <E66F58ED-C15E-3DFB-BC22-A861E13918C6> /usr/lib/system/liblaunch.dylib
0x7fff6e188000 - 0x7fff6e18cffb libmacho.dylib (900.0.1) <756F2553-07B6-3B42-ACEA-2F0F1A5E8D0F> /usr/lib/system/libmacho.dylib
0x7fff6e18d000 - 0x7fff6e18fff3 libquarantine.dylib (86) <6AC8773F-3817-3D82-99C2-01BABB9C3CBB> /usr/lib/system/libquarantine.dylib
0x7fff6e190000 - 0x7fff6e191ff3 libremovefile.dylib (45) <912FA211-DD8C-3C92-8424-21B89F8B10FD> /usr/lib/system/libremovefile.dylib
0x7fff6e192000 - 0x7fff6e1a9fff libsystem_asl.dylib (356.1.1) <94972913-9DF0-3C78-847C-43E58919E3DA> /usr/lib/system/libsystem_asl.dylib
0x7fff6e1aa000 - 0x7fff6e1aafff libsystem_blocks.dylib (67) <F2493BB5-B1C6-3C4D-9F1F-1B402E0F1DB7> /usr/lib/system/libsystem_blocks.dylib
0x7fff6e1ab000 - 0x7fff6e234ff7 libsystem_c.dylib (1244.30.3) <E0136C71-0648-36F0-9F84-82EA2748A8D7> /usr/lib/system/libsystem_c.dylib
0x7fff6e235000 - 0x7fff6e238ffb libsystem_configuration.dylib (963.30.1) <0F8D0B76-4F7D-34EC-AB6C-50F9465809DA> /usr/lib/system/libsystem_configuration.dylib
0x7fff6e239000 - 0x7fff6e23cffb libsystem_coreservices.dylib (51) <21A488D0-2D07-344E-8631-CC8B2A246F35> /usr/lib/system/libsystem_coreservices.dylib
0x7fff6e23d000 - 0x7fff6e23efff libsystem_darwin.dylib (1244.30.3) <2F750CB1-BC26-3FA3-AE59-553EE30D451B> /usr/lib/system/libsystem_darwin.dylib
0x7fff6e23f000 - 0x7fff6e245ff7 libsystem_dnssd.dylib (878.30.4) <EB9BB165-45A4-367C-B33A-688D4F383A95> /usr/lib/system/libsystem_dnssd.dylib
0x7fff6e246000 - 0x7fff6e28fff7 libsystem_info.dylib (517.30.1) <7D79E167-4B5C-3833-81EE-3AF3FB53616D> /usr/lib/system/libsystem_info.dylib
0x7fff6e290000 - 0x7fff6e2b5ff7 libsystem_kernel.dylib (4570.31.3) <D2E842AA-3B8D-31BF-8234-8C1BE11CFE32> /usr/lib/system/libsystem_kernel.dylib
0x7fff6e2b6000 - 0x7fff6e301fcb libsystem_m.dylib (3146) <ABB1B85F-9FFE-31B8-AD4F-E39A30794A93> /usr/lib/system/libsystem_m.dylib
0x7fff6e302000 - 0x7fff6e321fff libsystem_malloc.dylib (140.1.1) <9F0745FF-B92F-330D-8812-BB74001D1D33> /usr/lib/system/libsystem_malloc.dylib
0x7fff6e322000 - 0x7fff6e3c6ff3 libsystem_network.dylib (1229.30.11) <40BAD301-8744-3AD8-A688-E7925C587B00> /usr/lib/system/libsystem_network.dylib
0x7fff6e3c7000 - 0x7fff6e3d1ffb libsystem_networkextension.dylib (767.30.7) <C5ED4BAB-2480-369D-8AB6-DFB37C15A1EE> /usr/lib/system/libsystem_networkextension.dylib
0x7fff6e3d2000 - 0x7fff6e3dbff3 libsystem_notify.dylib (172) <98EA3D62-7C86-30DE-8261-D020D2F1EFF3> /usr/lib/system/libsystem_notify.dylib
0x7fff6e3dc000 - 0x7fff6e3e3ff7 libsystem_platform.dylib (161.20.1) <C049250F-8C35-314D-810F-4E28AEAED983> /usr/lib/system/libsystem_platform.dylib
0x7fff6e3e4000 - 0x7fff6e3effff libsystem_pthread.dylib (301.30.1) <ABA848E1-6978-3B42-A3A7-608B2C36FA93> /usr/lib/system/libsystem_pthread.dylib
0x7fff6e3f0000 - 0x7fff6e3f3ff3 libsystem_sandbox.dylib (765.30.4) <A51828BA-26D8-3205-AF3A-DF5C37269468> /usr/lib/system/libsystem_sandbox.dylib
0x7fff6e3f4000 - 0x7fff6e3f5ff3 libsystem_secinit.dylib (30) <F06ADB8F-9E94-34A7-B3C9-2C22FDD14BAD> /usr/lib/system/libsystem_secinit.dylib
0x7fff6e3f6000 - 0x7fff6e3fdff7 libsystem_symptoms.dylib (820.30.7) <DC3586C2-AA56-3419-88D3-FC0DBF08E3C0> /usr/lib/system/libsystem_symptoms.dylib
0x7fff6e3fe000 - 0x7fff6e411ff7 libsystem_trace.dylib (829.30.14) <69EBF017-D40F-30D7-9B0B-BFC862D761A5> /usr/lib/system/libsystem_trace.dylib
0x7fff6e413000 - 0x7fff6e418ff7 libunwind.dylib (35.3) <6D4FCD49-D2A9-3233-95C7-A7635CE265F2> /usr/lib/system/libunwind.dylib
0x7fff6e419000 - 0x7fff6e445ff7 libxpc.dylib (1205.30.29) <F7E5F1BC-614B-39CB-B6CE-92A9C7B7EC0B> /usr/lib/system/libxpc.dylib
External Modification Summary:
Calls made by other processes targeting this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by all processes on this machine:
task_for_pid: 19972451
thread_create: 0
thread_set_state: 0
VM Region Summary:
ReadOnly portion of Libraries: Total=197.0M resident=0K(0%) swapped_out_or_unallocated=197.0M(100%)
Writable regions: Total=17.3M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=17.3M(100%)
VIRTUAL REGION
REGION TYPE SIZE COUNT (non-coalesced)
=========== ======= =======
Kernel Alloc Once 8K 2
MALLOC 9276K 8
MALLOC guard page 16K 5
STACK GUARD 56.0M 2
Stack 8192K 2
__DATA 2324K 43
__LINKEDIT 188.3M 4
__TEXT 8976K 42
shared memory 8K 3
=========== ======= =======
TOTAL 272.4M 102
Can someone explain what are the arguments to divbwt in encode.c?
//-- encode.c --
uint8_t *block = (void *)(s->SA + s->max_block_size + GROUP_SIZE);
(...)
s->bwt_idx = divbwt(block, s->SA, s->u.bucket, s->nblock);
I tried to test it with sais or libsais and it didn't work according to a plan. First i read a bit about them, found en example, and prepared tenporary buffer (Atmp
) of 4 blocks size then tried to call it as shown below but it didn't work.
//-- encode.c-sais --
int* Atmp = malloc(bs100k*100000u*sizeof(int));
(...)
/* s->bwt_idx = divbwt(block, s->SA, s->u.bucket, s->nblock); */
/*// sais_u8_bwt(T, T, SA, (sa_int32_t)m, 256)) //*/
s->bwt_idx = sais_u8_bwt(block, block, s->SA, s->nblock, 256); /// somewhat corrupted
s->bwt_idx = sais_u8_bwt(block, s->SA, s->u.bucket, s->nblock, 256); /// Segmentation fault
s->bwt_idx = sais_u8_bwt(block, s->SA, Atmp, s->nblock, 256); /// Segmentation fault
Here, the first example (somewhat corrupted
) is the only that didn't segfault but still it's corrupted (repeats short piece of encoded message; I guess it's too small buffer but why the others with Atmp
fail so miserably?
-- encode.c-libsais
int* Atmp = malloc(bs100k*100000u*sizeof(int));
(...)
/* s->bwt_idx = divbwt(block, s->SA, s->u.bucket, s->nblock); */
s->bwt_idx = libsais_bwt(block, s->SA, s->u.bucket, s->nblock); ///const int idx=libsais_bwt(buf, buf, ptr, n);
s->bwt_idx = libsais_bwt(block, s->SA, Atmp, s->nblock); ///const int idx=libsais_bwt(buf, buf, ptr, n);
Ed: I realised that there is something like discussion on github and this shall be moved there. The only thing is it's not enabled in this repo. If some maintainer could enable out and move the thread to discussion, that would be better, I think.
I found an interesting thing,
if a file a
does already exist,
then
lbzcat a.bz2 > a
is much much slower than
lbzcat a.bz2 >! a
and
rm -f a; lbzcat a.bz2 > a
Why?
I had tried to compile it on MinGW, but it didn't work out because of signals
MinGW has only a small amount of signals
those are missing:
I had defined them at the bottom of common.h , not sure what is the effect of this.
ideas for signals alternative:
http://www.perlmonks.org/?node_id=1022364
https://social.msdn.microsoft.com/Forums/vstudio/en-US/8d128695-1727-434b-9fd1-7b74b2bf25ea/need-a-better-way-for-inter-thread-communication-?forum=vcgeneral
also had an error:
signals.c:78:20: error: storage size of 'act' isn't known
struct sigaction act;
migw's signal.h
#ifndef _INC_SIGNAL
#define _INC_SIGNAL
#include <crtdefs.h>
#include <pthread_signal.h>
#ifdef __cplusplus
extern "C" {
#endif
#ifndef _SIG_ATOMIC_T_DEFINED
#define _SIG_ATOMIC_T_DEFINED
typedef int sig_atomic_t;
#endif
#define NSIG 23
#define SIGINT 2
#define SIGILL 4
#define SIGABRT_COMPAT 6
#define SIGFPE 8
#define SIGSEGV 11
#define SIGTERM 15
#define SIGBREAK 21
#define SIGABRT 22 /* used by abort, replace SIGIOT in the future */
#define SIGABRT2 22
#ifdef _POSIX
#define SIGHUP 1 /* hangup */
#define SIGQUIT 3 /* quit */
#define SIGTRAP 5 /* trace trap (not reset when caught) */
#define SIGIOT 6 /* IOT instruction */
#define SIGEMT 7 /* EMT instruction */
#define SIGKILL 9 /* kill (cannot be caught or ignored) */
#define SIGBUS 10 /* bus error */
#define SIGSYS 12 /* bad argument to system call */
#define SIGPIPE 13 /* write on a pipe with no one to read it */
#ifdef __USE_MINGW_ALARM
#define SIGALRM 14 /* alarm clock */
#endif
#endif
typedef void (*__p_sig_fn_t)(int);
#define SIG_DFL (__p_sig_fn_t)0
#define SIG_IGN (__p_sig_fn_t)1
#define SIG_GET (__p_sig_fn_t)2
#define SIG_SGE (__p_sig_fn_t)3
#define SIG_ACK (__p_sig_fn_t)4
#define SIG_ERR (__p_sig_fn_t)-1
extern void **__cdecl __pxcptinfoptrs(void);
#define _pxcptinfoptrs (*__pxcptinfoptrs())
__p_sig_fn_t __cdecl signal(int _SigNum,__p_sig_fn_t _Func);
int __cdecl raise(int _SigNum);
#ifdef __cplusplus
}
#endif
#endif
pthread_signal.h
#ifndef WIN_PTHREADS_SIGNAL_H
#define WIN_PTHREADS_SIGNAL_H
/* Windows has rudimentary signals support. */
#define pthread_sigmask(H, S1, S2) 0
#endif /* WIN_PTHREADS_SIGNAL_H */
just to give some sense about the errors:
the last one i did not knew how to solve
.
$ make
Making all in lib
make[1]: Entering directory '/c/Users/user/Downloads/lbzip2-2.5/lbzip2-2.5/lib'
GEN alloca.h
GEN errno.h
GEN c++defs.h
GEN arg-nonnull.h
GEN warn-on-use.h
GEN fcntl.h
GEN inttypes.h
GEN math.h
GEN stdio.h
GEN stdlib.h
GEN string.h
GEN sys/stat.h
GEN sys/time.h
GEN sys/types.h
GEN time.h
GEN unistd.h
GEN wchar.h
make all-recursive
make[2]: Entering directory '/c/Users/user/Downloads/lbzip2-2.5/lbzip2-2.5/lib'
make[3]: Entering directory '/c/Users/user/Downloads/lbzip2-2.5/lbzip2-2.5/lib'
CC dtotimespec.o
CC gettime.o
CC math.o
CC printf-frexp.o
CC printf-frexpl.o
CC stat-time.o
CC timespec.o
CC timespec-add.o
CC timespec-sub.o
CC unistd.o
CC utimens.o
CC xmalloc.o
CC xsize.o
CC asnprintf.o
CC fprintf.o
CC frexp.o
CC frexpl.o
CC fseterr.o
CC fstat.o
CC gettimeofday.o
CC isnanl.o
CC malloc.o
CC msvc-inval.o
CC printf-args.o
CC printf-parse.o
CC stat.o
CC vasnprintf.o
AR libgnu.a
make[3]: Leaving directory '/c/Users/user/Downloads/lbzip2-2.5/lbzip2-2.5/lib'
make[2]: Leaving directory '/c/Users/user/Downloads/lbzip2-2.5/lbzip2-2.5/lib'
make[1]: Leaving directory '/c/Users/user/Downloads/lbzip2-2.5/lbzip2-2.5/lib'
Making all in src
make[1]: Entering directory '/c/Users/user/Downloads/lbzip2-2.5/lbzip2-2.5/src'
CC compress.o
CC crctab.o
CC decode.o
CC divbwt.o
CC encode.o
encode.c: In function 'encoder_init':
encode.c:121:3: warning: implicit declaration of function 'bzero' [-Wimplicit-function-declaration]
bzero(s->cmap, 256u * sizeof(bool));
^~~~~
encode.c:121:3: warning: incompatible implicit declaration of built-in function 'bzero'
encode.c: In function 'generate_initial_trees':
encode.c:829:5: warning: incompatible implicit declaration of built-in function 'bzero'
bzero(&s->length[t][a], b - a);
^~~~~
encode.c: In function 'assign_codes':
encode.c:901:3: warning: incompatible implicit declaration of built-in function 'bzero'
bzero(tree, sizeof(tree));
^~~~~
encode.c: In function 'generate_prefix_code':
encode.c:1060:5: warning: incompatible implicit declaration of built-in function 'bzero'
bzero(s->frequency, nt * sizeof(*s->frequency));
^~~~~
CC expand.o
expand.c: In function 'on_input_avail':
expand.c:842:3: warning: implicit declaration of function 'bzero' [-Wimplicit-function-declaration]
bzero((char *)buffer + size, missing);
^~~~~
expand.c:842:3: warning: incompatible implicit declaration of built-in function 'bzero'
CC main.o
main.c: In function 'info':
main.c:107:5: warning: implicit declaration of function 'flockfile' [-Wimplicit-function-declaration]
flockfile(stderr); \
^
main.c:123:1: note: in expansion of macro 'DEF'
DEF(info ( const char *fmt, ...), 0,0,0,0,1)
^~~
main.c:117:7: warning: implicit declaration of function 'funlockfile' [-Wimplicit-function-declaration]
funlockfile(stderr); \
^
main.c:123:1: note: in expansion of macro 'DEF'
DEF(info ( const char *fmt, ...), 0,0,0,0,1)
^~~
main.c: In function 'xstrtol':
main.c:173:12: warning: implicit declaration of function 'index' [-Wimplicit-function-declaration]
endptr = index(suffix, *endptr);
^~~~~
main.c:173:12: warning: incompatible implicit declaration of built-in function 'index'
main.c: In function 'output_regf_uninit':
main.c:867:13: warning: implicit declaration of function 'fchown' [-Wimplicit-function-declaration]
if (-1 == fchown(outfd, sbuf->st_uid, sbuf->st_gid)) {
^~~~~~
main.c:877:15: warning: implicit declaration of function 'fchmod' [-Wimplicit-function-declaration]
if (-1 == fchmod(outfd, sbuf->st_mode & (S_IRWXU | S_IRWXG | S_IRWXO))) {
^~~~~~
CC parse.o
CC process.o
CC signals.o
signals.c: In function 'xempty':
signals.c:37:7: warning: implicit declaration of function 'sigemptyset' [-Wimplicit-function-declaration]
if (sigemptyset(set) != 0)
^~~~~~~~~~~
signals.c: In function 'xadd':
signals.c:44:7: warning: implicit declaration of function 'sigaddset' [-Wimplicit-function-declaration]
if (sigaddset(set, sig) != 0)
^~~~~~~~~
signals.c: In function 'xpending':
signals.c:58:7: warning: implicit declaration of function 'sigpending' [-Wimplicit-function-declaration]
if (sigpending(set) != 0)
^~~~~~~~~~
signals.c: In function 'xmember':
signals.c:68:8: warning: implicit declaration of function 'sigismember' [-Wimplicit-function-declaration]
rv = sigismember(set, sig);
^~~~~~~~~~~
signals.c: In function 'xaction':
signals.c:78:20: error: storage size of 'act' isn't known
struct sigaction act;
^~~
signals.c:84:7: warning: implicit declaration of function 'sigaction' [-Wimplicit-function-declaration]
if (sigaction(sig, &act, NULL) != 0)
^~~~~~~~~
signals.c: In function 'halt':
signals.c:232:9: warning: implicit declaration of function 'sigsuspend' [-Wimplicit-function-declaration]
ret = sigsuspend(&saved);
^~~~~~~~~~
signals.c: In function 'xraise':
signals.c:257:7: warning: implicit declaration of function 'kill' [-Wimplicit-function-declaration]
if (kill(pid, sig) != 0)
^~~~
signals.c: In function 'bailout':
signals.c:308:1: warning: 'noreturn' function does return
}
^
make[1]: *** [Makefile:1342: signals.o] Error 1
make[1]: Leaving directory '/c/Users/user/Downloads/lbzip2-2.5/lbzip2-2.5/src'
make: *** [Makefile:1308: all-recursive] Error 1
Using 2.3, when #14 happens, the bz2 file is not deleted:
$ lbzip2 beic-010.10.tar
lbzip2: skipping "beic-010.10.tar": open("beic-010.10.tar.bz2"): File exists
$ ls -lh beic-010.10*
-rw-rw-r-- 1 fondazione fondazione 4,1G giu 20 11:22 beic-010.10.tar
-rw------- 1 fondazione fondazione 25M giu 20 11:38 beic-010.10.tar.bz2
Bug reported under Debian BTS:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=582476
The lib
directory that contains Gnulib C files seems missing from this repository, despite that it's there inside release versions:http://lbzip2.org/download.
The top post on the lbzip2.org website is The four most popular Colombian women on Onlyfans: one of them charges over $100,000 for racy photos and videos. All the other posts are technical content about lbzip2 and this one is out of place.
And again, here the downstream report
https://bugs.gentoo.org/show_bug.cgi?id=393155
Simply download the latest version from here
http://pytz.sourceforge.net/
and extract the archive. It fails to extract pytz/init.py
@bonzini forwarded me a request to contribute this here. Thanks Paolo, and thank you Mikolaj for lbzip2.
There's an opportunity to significantly speed up the common case in the inverse MTF transform where the reordered index is in the first sliding window (i.e. 0..15). Instead of taking a poorly predictable branch, we can load the entire window, shuffle it on registers, and write it back. With SSSE3 this boils down to a load-pshufb-store sequence, but could be done as well by shifting/masking on general registers (obviously more instructions, but still should be an improvement over the current branchy code).
I coded the following as a personal exercise. Probably not upstreamable, would like to hear what the requirements for a proper patch might be. If you try this patch, keep in mind that CFLAGS need to enable SSSE3 (-mssse3
, or something like -march=native
).
--- ../lbzip2/src/decode.c 2022-07-06 00:57:22.365988349 +0300
+++ src/decode.c 2022-07-06 09:54:45.420303970 +0300
@@ -452,6 +452,34 @@ mtf_one(uint8_t **imtf_row, uint8_t *imt
version is given in #else clause.
*/
#if ROW_WIDTH == 16
+#if defined(__SSSE3__)
+ {
+ typedef char c8v16 __attribute__((vector_size(16)));
+ static const c8v16 tab[] = {
+ { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 1, 0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 2, 0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 3, 0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 4, 0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 5, 0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 6, 0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 7, 0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15 },
+ { 8, 0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15 },
+ { 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15 },
+ { 10, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15 },
+ { 11, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15 },
+ { 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15 },
+ { 13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15 },
+ { 14, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15 },
+ { 15, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 },
+ };
+ c8v16 t;
+ memcpy(&t, pp, sizeof t);
+ t = __builtin_ia32_pshufb128(t, tab[nn]);
+ memcpy(pp, &t, sizeof t);
+ return c;
+ }
+#endif
switch (nn) {
default:
abort();
Currently, the only parallel bzip2 implemention in Homebrew is pbzip2. Having one for lbzip2 would be great. I could add one, but the author may want first crack at it.
More info:
On the website, in DOWNLOAD section, the SOURCE CODE subsection says:
For instructions how to build lbzip2 from source code unpack sources and read the INSTALL file included there. lbzip2 uses GNU build system, so compilation and installation usually boils down to the following sequence:
As well, directly above that, the site links to the source code are missing.
"You can download source tarball of the latest released version here: lbzip2-2.5.tar.gz, or in bz2 format: lbzip2-2.5.tar.bz2. Downloadable tarballs of older releases are listed at the botton of this page."
Thanks for the hard work.
CentOs 6.4 lbzip2 2.3
Getting crash:
make_code_lengths: Assertion `i == n' failed.
Hard to reproduce, we are using it as:
pv uncomressed | lbzip2 > compressed.
And after 1-2 TB - it crashing.
minbzcat fails to decompress some files with trailing garbage, in particular f82c7be77eec69287f7924618924b4c8.bz2 from PROTOS Genome.
$ ./minbzcat <bz2/f82c7be77eec69287f7924618924b4c8.bz2
****
minbzcat: data error in bz2 file
$ echo $?
1
$ bzcat <bz2/f82c7be77eec69287f7924618924b4c8.bz2
****
bzcat: (stdin): trailing garbage after EOF ignored
$ echo $?
0
Could you please add the release tarball?
First, I don't know if this is a lbzip2 problem, or a problem with tar. I posted this question on stack exchange (here). Here is the short description: tar allows you to specify the compression algorithm and pass flag/options to that algorithm. So, this should tar a directory and pass that on to lbzip2 for compression using two processors in verbose mode:
tar cf dst.tar.bz2 src --use-compress-program=lbzip2 -v -n 2
The verbose flag works, but lbzip always uses the max number of processors.
RHEL 7.6
lbzip2 2.5
I am just copying over from downstream report:
https://bugs.gentoo.org/show_bug.cgi?id=436382
Something in a certain QEMU raw image seems to upset lbzip2-2.2. Other images are okay but I could not test another NTFS file system image. The error happens with a small part of the image so I will attach it.
Earlier versions of lbzip2 seem to be unaffected.
Reproducible: Always
Steps to Reproduce:
lbzip2 fails to check stream CRC when decompressing bzip2 some files with blocks larger than 1 MiB (highly compressible blocks). As the result these files aren't decompressed correctly. Note than lbzip2 itself never creates such files, so this bug doesn't affect decompressing files created by any version of lbzip2.
Ways to reproduce it:
$ echo QlpoOTFBWSZTWTcImb8AAAAAAIAAoACAJpQKpCETF3JFOFCQNwiZvw== | base64 -d | lbzcat
lbzcat: stdin: stream CRC mismatch
Justin Lecher reported the following case to also trigger the bug:
$ lbzip2 -d patchutils-0.3.2.tar.bz2
lbzip2: "patchutils-0.3.2.tar.bz2": stream CRC mismatch
Work to fix this problem is in progress. Estimated time of arrival: January 2012. For impatient there is a workaround available:
diff --git a/src/lbunzip2.c b/src/lbunzip2.c
index 113be47..67579ea 100644
--- a/src/lbunzip2.c
+++ b/src/lbunzip2.c
@@ -1453,7 +1453,7 @@ mux(struct w2m_q *w2m_q, struct m2s_q *m2s_q, struct files
if (w2m_blk->bs100k) {
bs100k = w2m_blk->bs100k;
any |= (9u >= bs100k);
- if (crc != w2m_blk->crc)
+ if (0 && crc != w2m_blk->crc)
log_fatal("%s: %s%s%s: stream CRC mismatch\n", pname, ispec->sep,
ispec->fmt, ispec->sep);
crc = 0u;
Bug reported under Debian BTS.
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=645999
It would be better not to abuse bzip2 bug that makes it accept more than 18002 selectors. Extra padding bits (max 7) can be added eg. with suboptimal tree descriptors (even number of bits) and up to one dummy selector.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.