robmueller / cache-fastmmap Goto Github PK

View Code? Open in Web Editor NEW

18.0 18.0 15.0 219 KB

Uses an mmap'ed file to act as a shared memory interprocess cache

Home Page: http://search.cpan.org/~robm/Cache-FastMmap/

Perl 52.64% C 41.52% XS 5.84%

cache-fastmmap's People

Contributors

Stargazers

Watchers

Forkers

alepharchives dsteinbrunner hugmeir realdecisions azunite 1nickt caldrin toddr vividsnow oschwald manwar radiator-software rafl potatogim

cache-fastmmap's Issues

Fails with perls < 5.10

As per subject. Best illustrated with the matrix:

http://matrix.cpantesters.org/?dist=Cache-FastMmap%201.47

Sample fail report:

http://www.cpantesters.org/cpan/report/eb3e5e1a-3962-11e8-bef7-ef2f924579b0

Diagnostics:

# Global symbol "$serializer" requires explicit package name at /tmpfs/.cpan-build-cpansand/2018040600/Cache-FastMmap-1.47-18/blib/lib/Cache/FastMmap.pm line 598.

Offending operator seems to be C<//>

If this cannot easily be fixed, it would be cool if you could declare your requirement. That way the FAILS are converted to UNKNOWNS. Details about how to do it are on this page: http://wiki.cpantesters.org/wiki/CPANAuthorNotes

Thanks!

Sereal/Json not faster than storable

On some of my smoker systems t/19.t fails:

#   Failed test 'Sereal faster than storable'
#   at t/19.t line 36.
# Looks like you failed 1 test of 7.
t/19.t .. 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/7 subtests

#   Failed test 'Json faster than storable'
#   at t/19.t line 37.
# Looks like you failed 1 test of 7.
t/19.t .. 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/7 subtests

t/18.t often fails with "Less fd's in child than caches:" or "Less fd's in child than caches:"

Alternatively it can fails with "Less fd's in child than caches:"
See eg: https://pkgsubmit.mageia.org/uploads/failure/cauldron/core/release/20220329135017.tv.duvel.4059698/log/perl-Cache-FastMmap-1.570.0-2.mga9/build.x86_64.0.20220329135335.log

Failed test 'Less fd's in child than caches: 29 < 20'

build.x86_64.0.20220329135335.log

JSON required for tests but not declared

Could not load serialization package: JSON : Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /__w/cache-fastmmap/cache-fastmmap/blib/lib /__w/cache-fastmmap/cache-fastmmap/blib/arch /usr/local/lib/perl5/site_perl/5.30.3/x86_64-linux-gnu /usr/local/lib/perl5/site_perl/5.30.3 /usr/local/lib/perl5/vendor_perl/5.30.3/x86_64-linux-gnu /usr/local/lib/perl5/vendor_perl/5.30.3 /usr/local/lib/perl5/5.30.3/x86_64-linux-gnu /usr/local/lib/perl5/5.30.3 .) at (eval 13) line 1.
# Looks like your test exited with 2 just after 2.

See full output: https://github.com/szabgab/cache-fastmmap/runs/1374034225?check_suite_focus=true

One solution would be to skip that specific test if JSON is not available.

Random test failures (1.53, 1.54)

t/2.t fails randomly like this (but sometimes with differing values):

#   Failed test 'expire get 12'
#   at t/2.t line 52.
#          got: undef
#     expected: '123'

#   Failed test 'expire get 15'
#   at t/2.t line 56.
#          got: undef
#     expected: '123'

#   Failed test 'expire get 20'
#   at t/2.t line 67.
#          got: undef
#     expected: '123'
# Looks like you failed 3 tests of 51.
t/2.t ... 
Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/51 subtests

The failures seem to be independent from perl versions or operating systems.

Compilation fails on Windows

This is an error report I received from somebody installing my project Coocook. Cache::FastMmap is one dependency and fails to build.

Looking at http://matrix.cpantesters.org/?dist=Cache-FastMmap+1.48 I am not sure whether Cache::FastMmap supports Win32. Does it?

How to build and install it?

==> Found dependencies: Cache::FastMmap
Searching Cache::FastMmap (1.29) on cpanmetadb ...
--> Working on Cache::FastMmap
Fetching http://www.cpan.org/authors/id/R/RO/ROBM/Cache-FastMmap-1.48.tar.gz
-> OK
Unpacking Cache-FastMmap-1.48.tar.gz
Entering Cache-FastMmap-1.48
Checking configure dependencies from META.json
Checking if you have ExtUtils::MakeMaker 7.00 ... Yes (7.38)
Configuring Cache-FastMmap-1.48
Running Makefile.PL
Checking if your kit is complete...
Looks good
Generating a gmake-style Makefile
Writing Makefile for Cache::FastMmap
Writing MYMETA.yml and MYMETA.json
-> OK
Checking dependencies from MYMETA.json ...
Checking if you have Storable 0 ... Yes (3.15)
Checking if you have ExtUtils::MakeMaker 7.00 ... Yes (7.38)
Building and testing Cache-FastMmap-1.48
cp lib/Cache/FastMmap.pm blib\lib\Cache\FastMmap.pm
Running Mkbootstrap for FastMmap ()
"C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command -e chmod -- 644 "FastMmap.bs"
"C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command::MM -e cp_nonempty -- FastMmap.bs blib\arch\auto\Cache\FastMmap\FastMmap.bs 644
"C:\Strawberry\perl\bin\perl.exe" "C:\Strawberry\perl\lib\ExtUtils/xsubpp"  -typemap C:\STRAWB~1\perl\lib\ExtUtils\typemap  FastMmap.xs > FastMmap.xsc
"C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command -e mv -- FastMmap.xsc FastMmap.c
gcc -c  -I. -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE -D__USE_MINGW_ANSI_STDIO -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields -s -O2   -DVERSION=\"1.48\" -DXS_VERSION=\"1.48\"  "-IC:\STRAWB~1\perl\lib\CORE"   FastMmap.c
gcc -c  -I. -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE -D__USE_MINGW_ANSI_STDIO -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields -s -O2   -DVERSION=\"1.48\" -DXS_VERSION=\"1.48\"  "-IC:\STRAWB~1\perl\lib\CORE"   mmap_cache.c
gcc -c  -I. -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE -D__USE_MINGW_ANSI_STDIO -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields -s -O2   -DVERSION=\"1.48\" -DXS_VERSION=\"1.48\"  "-IC:\STRAWB~1\perl\lib\CORE"   win32.c
win32.c:161:5: error: conflicting types for 'mmc_lock_page'
 int mmc_lock_page(mmap_cache* cache, MU32 p_offset) {
     ^~~~~~~~~~~~~
In file included from win32.c:26:
mmap_cache_internals.h:125:5: note: previous declaration of 'mmc_lock_page' was here
 int mmc_lock_page(mmap_cache* cache, MU64 p_offset);
     ^~~~~~~~~~~~~
win32.c: In function '_mmc_set_error':
win32.c:216:5: warning: 'strncat' specified bound 1024 equals destination size [-Wstringop-overflow=]
     strncat(errbuf, ": ", 1024);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
gmake: *** [Makefile:346: win32.o] Error 1
-> FAIL Installing Cache::FastMmap failed. See C:\Users\XXX_USERNAME_XXX\.cpanm\work\1578136759.14308\build.log for details. Retry with --force to force install it.
-> FAIL Installing the dependencies failed: Module 'Cache::FastMmap' is not installed

Wide character problem

Hi,

we are using this package to cache translations in our web application. They often contain wide characters, which is working fine when no compression is enabled. Once we enable compression (tested with LZ4 and Snappy), storing something in the Cache crashes with "Wide character in subroutine entry at [..]/Cache/FastMmap.pm line 740".

Since wide characters are OK when not compressing, I would think handling wide characters would be the responsibility of Cache::FastMmap and not the caller.

What do you think of it?

A patch could be supplied if you think the same way :)

Thanks and best regards

Bug: Current unix timestamp is more than 1500000000

see here.

#date -d @1500000000
Fri Jul 14 10:40:00 CST 2017

Cache::FastMmap segfaults with files larger than 4GB or so

https://rt.cpan.org/Public/Bug/Display.html?id=92518

@autarch opened on Mon Jan 27 16:11:25 2014

I'm guessing this is because all the internals are based on 32-bit integers.?

Missing dependency Test::Deep in 1.53

t/2.t is using Test::Deep, but it is not mentioned as a dependency.
I saw this while maintaining the perl modules repo for openSUSE
https://build.opensuse.org/package/show/devel:languages:perl/perl-Cache-FastMmap

Lock failed: Bad file descriptor

Hi all. I have trouble. After one or two days work FastMmap says like this. And no more sessions after that.

2016-05-17 19:41:19: (mod_fastcgi.c.2695) FastCGI-stderr: [error] Caught exception in largeheart::Controller::Root->cart "Lock failed: Bad file descriptor at /usr/local/lib/perl5/site_perl/5.14.2/mach/Cache/FastMmap.pm line 1200."

Has any idea to how to fix it?
running on FreeBSD 8.4.

Data::MessagePack

Please also include Data::MessagePack as a seralization algorith, or a generic way to plug in any serializer ala Plack::Session::Store::File.

Typo in POD in file lib/Cache/FastMmap.pm

lib/Cache/FastMmap.pm

=item * B<raw_values>

Store values as raw binary data rather than using Storable to free/thaw
data structures (default: 0)

I guess, this should be "freeze/thaw".

Infrequently: segfault in FastMmap.so

Hello,

I love Cache::FastMmap, except...

Infrequently, my web application crashes and will not restart until the cache file is deleted. The crash sometimes happens once a week, sometimes multiple times on a single day with no apparent pattern.

Version: '1.56'
Perl: 5.32.1
Environment: Linux 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux

The workers initialize their cache with the following piece of code:

	$self->{cache} = Cache::FastMmap->new(
		share_file      => '/run/shm/webapp',
		expire_time     => 0,
		unlink_on_exit  => 0,
	);

Here is what /var/log/messages looks like. I have been unable to get other sort of diagnostics.

Nov 27 01:06:53 frontendsvr kernel: [1009530.623177] webapp.pl[2477416]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:53 frontendsvr kernel: [1009530.623503] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:53 frontendsvr kernel: [1009530.762896] webapp.pl[2470298]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:53 frontendsvr kernel: [1009530.762912] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:53 frontendsvr kernel: [1009530.882630] webapp.pl[2480764]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:53 frontendsvr kernel: [1009530.882649] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:53 frontendsvr kernel: [1009530.910137] webapp.pl[2457890]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:53 frontendsvr kernel: [1009530.910154] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:54 frontendsvr kernel: [1009531.204498] webapp.pl[2482236]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:54 frontendsvr kernel: [1009531.204515] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:54 frontendsvr kernel: [1009531.306465] webapp.pl[2468108]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:54 frontendsvr kernel: [1009531.306508] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:54 frontendsvr kernel: [1009531.718009] webapp.pl[2465247]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:54 frontendsvr kernel: [1009531.718027] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:54 frontendsvr kernel: [1009531.861339] webapp.pl[2468513]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:54 frontendsvr kernel: [1009531.861385] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:55 frontendsvr kernel: [1009531.976008] webapp.pl[2477412]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:55 frontendsvr kernel: [1009531.976141] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13
Nov 27 01:06:55 frontendsvr kernel: [1009532.036817] webapp.pl[2477599]: segfault at 8 ip 00007f4df256564e sp 00007ffcf7fcc660 error 4 in FastMmap.so[7f4df2561000+6000]
Nov 27 01:06:55 frontendsvr kernel: [1009532.036831] Code: 89 44 24 20 4d 39 ee 0f 83 1e 01 00 00 45 31 e4 89 e8 44 89 e5 4d 89 f4 4d 89 fe 41 89 c7 66 0f 1f 44 00 00 49 8b 34 24 31 d2 <8b> 46 08 41 f7 f7 0f 1f 40 00 89 d0 49 8d 1c 86 8b 03 85 c0 74 13

Any help would be appreciated.

Best regards,
Peter