perlffi / ffi-checklib

Check that a library is available for FFI

FFI::CheckLib

Check that a library is available for FFI

SYNOPSIS

use FFI::CheckLib;

check_lib_or_exit( lib => 'jpeg', symbol => 'jinit_memory_mgr' );
check_lib_or_exit( lib => [ 'iconv', 'jpeg' ] );

# or prompt for path to library and then:
print "where to find jpeg library: ";
my $path = <STDIN>;
check_lib_or_exit( lib => 'jpeg', libpath => $path );

DESCRIPTION

This module checks whether a particular dynamic library is available for FFI to use. It is modeled heavily on Devel::CheckLib, but will find dynamic libraries even when development packages are not installed. It also provides a find_lib function that will return the full path to the found dynamic library, which can be feed directly into FFI::Platypus or another FFI system.

Although intended mainly for FFI modules via FFI::Platypus and similar, this module does not actually use any FFI to do its detection and probing. This module does not have any non-core runtime dependencies. The test suite does depend on Test2::Suite.

FUNCTIONS

All of these take the same named parameters and are exported by default.

find_lib

my(@libs) = find_lib(%args);

This will return a list of dynamic libraries, or empty list if none were found.

[version 0.05]

If called in scalar context it will return the first library found.

Arguments are key value pairs with these keys:

• lib

  Must be either a string with the name of a single library or a reference to an array of strings of library names. Depending on your platform, CheckLib will prepend lib or append .dll or .so when searching.

  [version 0.11]

  As a special case, if * is specified then any libs found will match.

• libpath

  A string or array of additional paths to search for libraries.

• systempath

  [version 0.11]

  A string or array of system paths to search for instead of letting FFI::CheckLib determine the system path. You can set this to [] in order to not search any system paths.

• symbol

  A string or a list of symbol names that must be found.

• verify

  A code reference used to verify a library really is the one that you want. It should take two arguments, which is the name of the library and the full path to the library pathname. It should return true if it is acceptable, and false otherwise. You can use this in conjunction with FFI::Platypus to determine if it is going to meet your needs. Example:

  use FFI::CheckLib;
  use FFI::Platypus;
  
  my($lib) = find_lib(
    lib => 'foo',
    verify => sub {
      my($name, $libpath) = @_;
  
      my $ffi = FFI::Platypus->new;
      $ffi->lib($libpath);
  
      my $f = $ffi->function('foo_version', [] => 'int');
  
      return $f->call() >= 500; # we accept version 500 or better
    },
  );

• recursive

  [version 0.11]

  Recursively search for libraries in any non-system paths (those provided via libpath above).

• try_linker_script

  [version 0.24]

  Some vendors provide .so files that are linker scripts that point to the real binary shared library. These linker scripts can be used by gcc or clang, but are not directly usable by FFI::Platypus and friends. On select platforms, this options will use the linker command (ld) to attempt to resolve the real .so for non-binary files. Since there is extra overhead this is off by default.

  An example is libyaml on Red Hat based Linux distributions. On Debian these are handled with symlinks and no trickery is required.

• alien

  [version 0.25]

  If no libraries can be found, try the given aliens instead. The Alien classes specified must provide the Alien::Base interface for dynamic libraries, which is to say they should provide a method called dynamic_libs that returns a list of dynamic libraries.

  [version 0.28]

  In 0.28 and later, if the Alien is not installed then it will be ignored and this module will search in system or specified directories only. This module will still throw an exception, if the Alien doesn't look like a module name or if it does not provide a dynamic_libs method (which is implemented by all Alien::Base subclasses).

  [version 0.30] [breaking change]

  Starting with version 0.30, libraries provided by Aliens is preferred over the system libraries. The original thinking was that you want to prefer the system libraries because they are more likely to get patched with regular system updates. Unfortunately, the reason a module needs to install an Alien is likely because the system library is not new enough, so we now prefer the Aliens instead.

assert_lib

assert_lib(%args);

This behaves exactly the same as find_lib, except that instead of returning empty list of failure it throws an exception.

check_lib_or_exit

check_lib_or_exit(%args);

This behaves exactly the same as assert_lib, except that instead of dying, it warns (with exactly the same error message) and exists. This is intended for use in Makefile.PL or Build.PL

find_lib_or_exit

[version 0.05]

my(@libs) = find_lib_or_exit(%args);

This behaves exactly the same as find_lib, except that if the library is not found, it will call exit with an appropriate diagnostic.

find_lib_or_die

[version 0.06]

my(@libs) = find_lib_or_die(%args);

This behaves exactly the same as find_lib, except that if the library is not found, it will die with an appropriate diagnostic.

check_lib

my $bool = check_lib(%args);

This behaves exactly the same as find_lib, except that it returns true (1) on finding the appropriate libraries or false (0) otherwise.

which

[version 0.17]

my $path = which($name);

Return the path to the first library that matches the given name.

Not exported by default.

where

[version 0.17]

my @paths = where($name);

Return the paths to all the libraries that match the given name.

Not exported by default.

has_symbols

[version 0.17]

my $bool = has_symbols($path, @symbol_names);

Returns true if all of the symbols can be found in the dynamic library located at the given path. Can be useful in conjunction with verify with find_lib above.

Not exported by default.

system_path

[version 0.20]

my $path = FFI::CheckLib::system_path;

Returns the system path as a list reference. On some systems, this is PATH on others it might be LD_LIBRARY_PATH on still others it could be something completely different. So although you may add items to this list, you should probably do some careful consideration before you do so.

This function is not exportable, even on request.

ENVIRONMENT

FFI::CheckLib responds to these environment variables:

• FFI_CHECKLIB_PACKAGE

  On macOS platforms with Homebrew and/or MacPorts installed, their corresponding lib paths will be automatically appended to $system_path. In case of having both managers installed, Homebrew will appear before.

  This behaviour can be overridden using the environment variable FFI_CHECKLIB_PACKAGE.

  Allowed values are:

  - none: Won't use either Homebrew's path nor MacPorts - homebrew: Will append $(brew --prefix)/lib to the system paths - macports: Will append port's default lib path

  A comma separated list is also valid:

  export FFI_CHECKLIB_PACKAGE=macports,homebrew
  

  Order matters. So in this example, MacPorts' lib path appears before Homebrew's path.

• FFI_CHECKLIB_PATH

  List of directories that will be considered by FFI::CheckLib as additional "system directories". They will be searched before other system directories but after libpath. The variable is colon separated on Unix and semicolon separated on Windows. If you use this variable, FFI_CHECKLIB_PACKAGE will be ignored.

• PATH

  On Windows the PATH environment variable will be used as a search path for libraries.

On some operating systems LD_LIBRARY_PATH, DYLD_LIBRARY_PATH, DYLD_FALLBACK_LIBRARY_PATH or others may be used as part of the search for dynamic libraries and may be used (indirectly) by FFI::CheckLib as well.

FAQ

• Why not just use dlopen?

  Calling dlopen on a library name and then dlclose immediately can tell you if you have the exact name of a library available on a system. It does have a number of drawbacks as well.

  • No absolute or relative path

    It only tells you that the library is somewhere on the system, not having the absolute or relative path makes it harder to generate useful diagnostics.

  • POSIX only

    This doesn't work on non-POSIX systems like Microsoft Windows. If you are using a POSIX emulation layer on Windows that provides dlopen, like Cygwin, there are a number of gotchas there as well. Having a layer written in Perl handles this means that developers on Unix can develop FFI that will more likely work on these platforms without special casing them.

  • inconsistent implementations

    Even on POSIX systems you have inconsistent implementations. OpenBSD for example don't usually include symlinks for .so files meaning you need to know the exact .so version.

  • non-system directories

    By default dlopen only works for libraries in the system paths. Most platforms have a way of configuring the search for different non-system paths, but none of them are portable, and are usually discouraged anyway. Alien and friends need to do searches for dynamic libraries in non-system directories for share installs.

• My 64-bit Perl is misconfigured and has 32-bit libraries in its search path. Is that a bug in FFI::CheckLib?

  Nope.

• The way FFI::CheckLib is implemented it won't work on AIX, HP-UX, OpenVMS or Plan 9.

  I know for a fact that it doesn't work on AIX as currently implemented because I used to develop on AIX in the early 2000s, and I am aware of some of the technical challenges. There are probably other systems that it won't work on. I would love to add support for these platforms. Realistically these platforms have a tiny market share, and absent patches from users or the companies that own these operating systems (patches welcome), or hardware / CPU time donations, these platforms are unsupportable anyway.

SEE ALSO

• FFI::Platypus

  Call library functions dynamically without a compiler.

• Dist::Zilla::Plugin::FFI::CheckLib

  Dist::Zilla plugin for this module.

AUTHOR

Author: Graham Ollis [email protected]

Contributors:

Bakkiaraj Murugesan (bakkiaraj)

Dan Book (grinnz, DBOOK)

Ilya Pavlov (Ilya, ILUX)

Shawn Laffan (SLAFFAN)

Petr Písař (ppisar)

Michael R. Davis (MRDVT)

Shawn Laffan (SLAFFAN)

Carlos D. Álvaro (cdalvaro)

COPYRIGHT AND LICENSE

This software is copyright (c) 2014-2022 by Graham Ollis.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

ffi-checklib's Issues

error message for check_lib_or_exit not as helpful as it could be

example:

library not found at /home/ollisg/.perlbrew/libs/perl-5.20.1@dev/lib/perl5/FFI/CheckLib.pm line 153.

would be nice to include the name of the library :P

MSYS2

It passes the test suite, but almost certainly doesn't work correctly.

Add support for FFI_CHECKLIB_PATH

This is a path that will be searched IN ADDITION to the normal path. As mentioned here

#46 (comment)

we can use platform specific environment variables in some cases, but

• LD_LIBRARY_PATH is considered harmful imo in all but "one off" testing
• There is no portable environment variable to use

This path should be searched FIRST. The PR were I proposed this does not implement this feature (I called it _EXTRA there but on reflection it should be called _PATH), but does not implement it.

Windows DLLs using underscores instead of dashes

Since it shifted to a cmake build system libproj on windows has started using a naming scheme that separates components with underscores. This means the DLL is now libproj_9_1.dll instead of libproj-9-1.dll.

Assuming this is valid, the regexes at these points need to be updated.

FFI-CheckLib/lib/FFI/CheckLib.pm

Lines 93 to 98 in abd5339

	elsif($os eq 'MSWin32')
	{
	# handle cases like libgeos-3-7-0___.dll and libgtk-2.0-0.dll
	$pattern = [ qr{^(?:lib)?(\w+?)(?:-([0-9-\.]+))?_*\.dll$}i ];
	$version_split = qr/\-/;
	}

I think this will do the trick but have not tested it yet.

  $pattern = [ qr{^(?:lib)?(\w+?)(?:[_-]([0-9-\._]+))?_*\.dll$}i ];
  $version_split = qr/[_\-]/;

Let me know if you want me to work up a PR.

Shawn.

Invalid logic for symbol search

Hello,
You delete library from search before check that it has requested symbols. So if environment has 2 or more versions of needed library it's possible that library detection fail on 1 match that does not has symbols. Right now this is why Im failing my travis builds :/

The problem come from here https://github.com/plicease/FFI-CheckLib/blob/master/lib/FFI/CheckLib.pm#L251

Better document which environment variables are used

#46 (comment)

often depends on platform!

• PATH (windows)
• LD_LIBRARY_PATH on most unixen
• whatever the equivalent to LD_LIBRARY_PATH is on macOS
• also could be something entirely different!

Some of these variables are "used" indirectly, and we should make sure that is clear as well.

add find_lib_or_exit function

missing argument in call to catpath in find_lib()

: [ether@curacao ~]$; perl -d:Confess -MFFI::CheckLib -wle'print FFI::CheckLib::find_lib("lib", "newrelic", "alien", "Alien::libnewrelic")'
Use of uninitialized value $file in string ne at /Users/ether/perl5/perlbrew/perls/32.1/lib/5.32.1/darwin-2level/File/Spec/Unix.pm line 352.
	File::Spec::Unix::catpath("File::Spec", "", "/Users/ether/.perlbrew/libs/32.1\@std/lib/perl5/darwin-2level/"...) called at /Users/ether/.perlbrew/libs/32.1@std/lib/perl5/FFI/CheckLib.pm line 197
	FFI::CheckLib::find_lib("lib", "newrelic", "alien", "Alien::libnewrelic") called at -e line 1
Use of uninitialized value $file in concatenation (.) or string at /Users/ether/perl5/perlbrew/perls/32.1/lib/5.32.1/darwin-2level/File/Spec/Unix.pm line 360.
	File::Spec::Unix::catpath("File::Spec", "", "/Users/ether/.perlbrew/libs/32.1\@std/lib/perl5/darwin-2level/"...) called at /Users/ether/.perlbrew/libs/32.1@std/lib/perl5/FFI/CheckLib.pm line 197
	FFI::CheckLib::find_lib("lib", "newrelic", "alien", "Alien::libnewrelic") called at -e line 1
/Users/ether/.perlbrew/libs/32.1@std/lib/perl5/darwin-2level/auto/share/dist/Alien-libnewrelic/dynamic/libnewrelic.dylib

File::Spec::catpath expects three arguments, but only two are being passed on line 197, leading to an uninitialized warning when $file is checked.

(Found while doing perl -MNewFangle::FFI -we1.)

Undeclared dependency Env.pm

Env.pm is a core module, but some distributions (e.g. Fedora) ship a stripped perl installation without it. Probably it's better to add Env to the prereq list to avoid unnecessary test failures:

Can't locate Env.pm in @INC (@INC contains: ... .) at /home/cpansand/.cpan/build/2018111920/FFI-CheckLib-0.23-0iHlUt/blib/lib/FFI/CheckLib.pm line 34.
Compilation failed in require at t/ffi_checklib__os_mswin32.t line 4.
BEGIN failed--compilation aborted at t/ffi_checklib__os_mswin32.t line 4.
t/ffi_checklib__os_mswin32.t .. 
Dubious, test returned 2 (wstat 512, 0x200)
No subtests run 

Library extension .a does not get found

A library with the extension .a is not picked up on Mac

The reported fix is to add .a to the list of extensions on Mac:

   push @$pattern, qr{^lib(.*?)(?:\.([0-9]+(?:\.[0-9]+)*))?\.(?:dylib|bundle|a)$}; # add "a"

Maybe a different fix would also respect $Config{_a} from Config.pm.

See also [https://perlmonks.org/?node=11108106]

FFI::CheckLib should prefer alien over system libs

Releated
uperl/Archive-Libarchive#40

Please cleanup install

When I install FFI::CheckLib with cpanm --no-test, it still builds and installs all testing modules.

Test2::Tools::Process should be unconditional requirement

Problem for now is that T2:T:Process requires 5.10, and currently we support 5.8. That may and should change in the future.

counfounded by text file pointing to real .so file.

reported on Alien::LibYAML by @eserte via rt:

https://rt.cpan.org/Ticket/Display.html?id=129323

On my CentOS7 and Fedora28 smokers the ffi test fails if FFI::Platypus is installed. On these systems libyaml is already installed as a rpm package.

...
# Failed test 'ffi'
# at t/alien_libyaml__ffi.t line 14.
#   yaml_get_version_string not found
t/alien_libyaml__ffi.t .. 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/3 subtests 
...

Don't know if it matters, but /lib64/libyaml.so on these systems is just a text file with the contents:

INPUT(libyaml-0.so.2)

The referenced shared object looks like a real .so, though.

regex $pattern does not match name part of geos-3-7-0.dll on windows

The GEOS library on windows generates a DLL called geos-3-7-0.dll. This does not work with the regex used in FFI::CheckLib::_matches, as the name part appears invalid. FFI::CheckLib::find_lib returns undef, while the assert method dies with library not found.

my $pat  = qr{^(?:lib)?(.*?)(?:-([0-9])+)?\.dll$}i;  #  current regexp
my $pat2 = qr{^(?:lib)?(.*?)(?:-([0-9]+))?(?:-[0-9]+)*\.dll$}i;  #  revised regexp

my $string = 'geos-3-7-0.dll';

say 'gives: ' . join ', ', ($string =~ $pat);
#  gives: geos-3-7, 0
say 'gives: ' . join ', ', ($string =~ $pat2);
# gives: geos, 3

I'm not sure what you want to do with the remaining version components. They could be used in the _sort sub, but hopefully they are not needed as that would make it more complex. If you have a preference then I can work up a PR and test.

There are other complications in that some of the other DLLs I have under strawberry perl look like s1sgdk-win32-2.0-0.dll and libgd-3__.dll. I have nothing that uses them, but they are valid DLL names.

Feature Request: Allow regex as a lib argument in find_lib function

As of now, FFI::CheckLib provides find_lib function with 'lib' argument takes a 'string.

$libPath = find_lib(lib=>'gio-2.0',libpath=>'/usr/lib64');

Consider, If I write a module based on FFI::CheckLib and the module will run on may machines, The lib version on the target machine may vary. So please allow regex as a argument to lib, So I can preciously set my lib requirements (Ex: what version etc..)

Ex:
$libPath = find_lib(lib=>'gio-2./d+',libpath=>'/usr/lib64');

Can't find Windows libraries with .DLL extension in upper case

Hello.

The find_lib function can't find Windows libraries with .DLL extension in upper case.
For example I have Windows7 with COLORCNV.DLL and IPHLPAPI.DLL in C:\Windows\System32.

Code example:

my ($lib) = find_lib(
    lib => 'IPHLPAPI',
    # lib => 'IPHLPAPI.DLL'
)
# $lib is undef

if replace line 91 in FFI/CheckLib.pm

# from
$pattern = [ qr{^(?:lib)?(.*?)(?:-([0-9])+)?\.dll$} ];
# to
$pattern = [ qr{^(?:lib)?(.*?)(?:-([0-9])+)?\.(?i)dll$} ];

it resolve the problem.

Maybe exists a better solution

ffi_checklib__os_unix.t fails with perl 5.34.1 and 5.36.0

After upgrade from perl 5.34.0 to 5.34.1 the ffi_checklib__os_unix.t test started to fail as follows:

# perl                   5.034001 solaris i86pc-solaris-thread-multi-64
# DynaLoader             1.50
# ExtUtils::MakeMaker    7.62
# FFI::Platypus          -
# List::Util             1.55
# Test2::API             1.302183
# Test2::Require::EnvVar 0.000145
# Test2::Require::Module 0.000145
# Test2::Tools::Process  -
# Test2::V0              0.000145
# 
# 
# 
t/00_diag.t ................... ok
t/ci.t ........................ skipped: This test only runs if the $CIPSOMETHING environment variable is set
t/ffi_checklib.t .............. ok
t/ffi_checklib__exit.t ........ skipped: Module 'Test2::Tools::Process' is not installed
t/ffi_checklib__os_cygwin.t ... ok
t/ffi_checklib__os_darwin.t ... ok
t/ffi_checklib__os_mswin32.t .. ok
t/ffi_checklib__os_msys.t ..... ok
        # Failed test at t/ffi_checklib__os_unix.t line 165.
        # +-----------+----+-----------------+
        # | GOT       | OP | CHECK           |
        # +-----------+----+-----------------+
        # | libxor.so | eq | libxor.so.1.2.4 |
        # +-----------+----+-----------------+
    # Failed test 'multiple versions of the same lib'
    # at t/ffi_checklib__os_unix.t line 167.

# Failed test 'symlink'
# at t/ffi_checklib__os_unix.t line 189.
# Seeded srand with seed '20220803' from local date.
t/ffi_checklib__os_unix.t ..... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/16 subtests 

Test Summary Report
-------------------
t/ffi_checklib__os_unix.t   (Wstat: 256 Tests: 16 Failed: 1)
  Failed test:  14
  Non-zero exit status: 1
Files=9, Tests=64,  2 wallclock secs ( 0.08 usr  0.05 sys +  1.44 cusr  0.55 csys =  2.12 CPU)
Result: FAIL
Failed 1/9 test programs. 1/64 subtests failed.

While with perl 5.34.0 all tests passed:

# perl                   5.034000 solaris i86pc-solaris-thread-multi-64
# DynaLoader             1.50
# ExtUtils::MakeMaker    7.62
# FFI::Platypus          -
# List::Util             1.55
# Test2::API             1.302183
# Test2::Require::EnvVar 0.000145
# Test2::Require::Module 0.000145
# Test2::Tools::Process  -
# Test2::V0              0.000145
# 
# 
# 
t/00_diag.t ................... ok
t/ci.t ........................ skipped: This test only runs if the $CIPSOMETHING environment variable is set
t/ffi_checklib.t .............. ok
t/ffi_checklib__exit.t ........ skipped: Module 'Test2::Tools::Process' is not installed
t/ffi_checklib__os_cygwin.t ... ok
t/ffi_checklib__os_darwin.t ... ok
t/ffi_checklib__os_mswin32.t .. ok
t/ffi_checklib__os_msys.t ..... ok
t/ffi_checklib__os_unix.t ..... ok
All tests successful.
Files=9, Tests=64
Result: PASS

Please note that with perl 5.36.0 the ffi_checklib__os_unix.t test fails too with the same output as with perl 5.34.1.

alien option should operate in fallback mode

If the caller specifies alien and the alien isn't installed it should ignore that option instead of throwing an exception or dying on the require.

That is:

my $lib = find_lib lib => 'foo', alien => 'Alien::libfoo'

Should work either if the system libfoo.so is available, OR if Alien::libfoo is installed.