shared library build issue about cpymad

[coldfix]

In the latest master I get instead with pip install -e . I get [..]

Do I understand correctly that a separate installation of libmadx is mandatory now?

• If you build MAD-X as shared library: yes.
• If you build MAD-X as static library: only for building the C extension, not at runtime.
• I removed the ability of setup.py to automatically build madx again because it added a lot of complexity and was sometimes annoying in unintendedly downloading and building MAD-X when I forgot to pass the directory.

If you just want to use cpymad, just do pip install cpymad, it should do the job normally; if you really do need a development version, chances are that you may also want more control over the MAD-X build process anyway because the build will fail for some reason or the other (e.g. missing blas/lapack detection).

Personally, I use the following settings:

cmake \
    -DMADX_X11=OFF \
    -DMADX_STATIC=ON \
    -DMADX_ONLINE=OFF \
    -DBUILD_SHARED_LIBS=OFF \
    -DCMAKE_INSTALL_PREFIX=$MADXDIR \
    -DCMAKE_C_FLAGS="-fvisibility=hidden" \
    -DCMAKE_BUILD_TYPE=Release \
    -DMADX_INSTALL_DOC=OFF \
    ..
make install

In particular:

• build MAD-X as static library -DBUILD_SHARED_LIBS=OFF
• disable X11 with -DMADX_X11=OFF to decrease runtime dependencies (less problem with missing shared object after system updates)
• try using static dependencies via -DMADX_STATIC=ON if possible to further decrease runtime dependencies, but may not work depending on your system, so -DMADX_STATIC=OFF is sometimes required.

When building cpymad, one has to inform the setup script how MAD-X was built. Either by environment variables or on the command line, e.g.:

python setupy.py build_ext --no-X11 --blas --lapack --madxdir=$MADXDIR
pip install -e .

I was considering to encapsulate a madx build into a separate package that could be used as a build dependency and therefore eliminate the need to build madx, even when setting up a development version. What do you think?

Installing libmadx from sources and setting up MADXDIR works compiles, load, but seg_fault on twiss.

Under which exact circumstances does the segfault occur?

I have installed libmadx with:

cmake .. \
    -DMADX_ONLINE=OFF \
    -DMADX_INSTALL_DOC=OFF \
    -DCMAKE_INSTALL_PREFIX=~/.local \
    -DCMAKE_C_FLAGS="-fvisibility=hidden"

~/.local is definitely a good choice for a more permanent install location.

[rdemaria]

I was considering to encapsulate a madx build into a separate package that could be used as a build dependency and therefore eliminate the need to build madx, even when setting up a development version. What do you think?

I think it is a good idea, I am having issues now for instance, but it is not urgent I guess. I compile madx often anyway.

Installing libmadx from sources and setting up MADXDIR works compiles, load, but seg_fault on twiss.

Under which exact circumstances does the segfault occur?

I attach the result of pytest:

test_log.txt

cmake output is

cmake \
     -DMADX_X11=OFF \
     -DMADX_STATIC=ON \
     -DMADX_ONLINE=OFF \
     -DBUILD_SHARED_LIBS=OFF \
     -DCMAKE_INSTALL_PREFIX=$MADXDIR \
     -DCMAKE_C_FLAGS="-fvisibility=hidden" \
     -DCMAKE_BUILD_TYPE=Release \
     -DMADX_INSTALL_DOC=OFF \
     ..
-- Mad-X version: 5.04.02
-- Version num: 50402
-- Version date: 2018.10.03
-- Looking for SDDS libraries
-- Looking for SDDS libraries - found.
HOST = x86_64--linux-4.15.0-47-generic
-- 64 bit build
-- NTPSA turned on
-- A library with BLAS API not found. Please specify library location.
-- LAPACK requires BLAS
-- A library with LAPACK API not found. Please specify library location.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rdemaria/work/madx/MAD-X/build

[aoeftiger]

I have a similar issue not finding the already installed dynamic library for LAPACK.
I first followed the MAD-X compilation instructions from the cpymad manual, which gave the same error message as with your above extended cmake compiler flags.

On the present CentOS system, the two packages openblas-devel-0.3.3-2.el7.x86_64 and lapack-devel-3.4.2-8.el7.x86_64 are installed, the former is found while the latter is not:

$ cmake \
  -DMADX_X11=OFF \
  -DMADX_STATIC=ON \
  -DMADX_ONLINE=OFF \
  -DBUILD_SHARED_LIBS=OFF \
  -DCMAKE_INSTALL_PREFIX=$MADXDIR \
  -DCMAKE_C_FLAGS="-fvisibility=hidden" \
  -DCMAKE_BUILD_TYPE=Release \
  -DMADX_INSTALL_DOC=OFF \
  ..

(...)
-- Found BLAS: /usr/lib64/libopenblas.a  
-- A library with LAPACK API not found. Please specify library location.
(...)

This happens despite the file /usr/lib64/liblapack.so being present.

A first try with modifying CMakeLists.txt to include the shared library did not work (I did this as message("suffix: ${CMAKE_FIND_LIBRARY_SUFFIXES}") initially provided no suffixes at all): set(CMAKE_FIND_LIBRARY_SUFFIXES ".a;.so").

The system-wide cmake installation on this machine is at version 3.13.4, so what finally made cmake find LAPACK was to checkout into the MAD-X/cmake/ directory from the Kitware CMake github repo the files

• FindLAPACK.cmake
• CheckFortranFunctionExists.cmake
• CMakePushCheckState.cmake

Upon cmake .. with the above compiler flags it now finds LAPACK:

-- A library with LAPACK API found.
-- LAPACK uses /usr/lib64/libopenblas.a;/usr/lib64/libopenblas.a

[aoeftiger]

Unfortunately, this step didn't help me getting the github cpymad developer version to run, though..

from cpymad.mad import Madx
madx = Madx()

gives

ImportError                               Traceback (most recent call last)
<ipython-input-5-07c50d8cd6ed> in <module>
----> 1 madx = Madx()
      2 madx.options.echo = False
      3 # madx.options.warn = False
      4 # madx.options.info = False

~/anaconda3/lib/python3.7/site-packages/cpymad/madx.py in __init__(self, libmadx, command_log, stdout, history, **Popen_args)
    169             if callable(stdout):
    170                 self.reader = AsyncReader(self._process.stdout, stdout)
--> 171         if not libmadx.is_started():
    172             with self.reader:
    173                 libmadx.start()

~/anaconda3/lib/python3.7/site-packages/minrpc/client.py in DeferredMethod(*args, **kwargs)
    158         def DeferredMethod(*args, **kwargs):
    159             return self.__client._request('function_call', self.__module,
--> 160                                           funcname, args, kwargs)
    161         return DeferredMethod

~/anaconda3/lib/python3.7/site-packages/minrpc/client.py in _request(self, kind, *args)
    110                 self._conn.close()
    111                 raise RemoteProcessCrashed()
--> 112         return self._dispatch(response)
    113 
    114     def _communicate(self, message):

~/anaconda3/lib/python3.7/site-packages/minrpc/client.py in _dispatch(self, response)
    121         kind, args = response
    122         handler = getattr(self, '_dispatch_%s' % (kind,))
--> 123         return handler(*args)
    124 
    125     def _dispatch_exception(self, exc_type, message):

~/anaconda3/lib/python3.7/site-packages/minrpc/client.py in _dispatch_exception(self, exc_type, message)
    129         raise type(exc_type.__name__, (exc_type,), {
    130             '__str__': lambda *args: message,
--> 131             '__init__': lambda *args: None})
    132 
    133     def _dispatch_data(self, data):

ImportError: Traceback (most recent call last):
  File "/home/HPC/oeftiger/anaconda3/lib/python3.7/site-packages/minrpc/service.py", line 85, in _dispatch
    response = handler(*args)
  File "/home/HPC/oeftiger/anaconda3/lib/python3.7/site-packages/minrpc/service.py", line 101, in _dispatch_function_call
    module = __import__(modname, None, None, '*')
ImportError: /home/HPC/oeftiger/anaconda3/lib/python3.7/site-packages/cpymad/libmadx.cpython-37m-x86_64-linux-gnu.so: undefined symbol: dgelsd_

[aoeftiger]

... I guess the flag -DBUILD_SHARED_LIBS=OFF is supposed to prevent finding the .so in the first place -- anyway, installing the .a version (lapack-static) fixed not finding LAPACK also with the usual MAD-X shipped FindLAPACK.cmake without downloading from the kitware repo.

[coldfix]

Using -DBUILD_SHARED_LIBS=OFF means that mad-x will be built as static library and not include symbols from its dependencies. In this case you have to tell cpymad every library that is needed to link, including blas/lapack by doing something like:

python setupy.py build_ext --no-X11 --blas --lapack --madxdir=$MADXDIR

otherwise the symbol won't be found.

[coldfix]

(Note that -DBUILD_SHARED_LIBS=OFF should have nothing to do with whether dependencies such as lapack are resolved as .so or .a, this is controlled by -DMADX_STATIC)

[aoeftiger]

Yes this didn't work as the static files for LAPACK were not even installed, my bad.

[aoeftiger]

And yay, having installed lapack-static, now also the cpymad compilation works properly! So nevermind..

I managed to pre-compile MAD-X with

$ cmake \
  -DMADX_X11=OFF \
  -DMADX_STATIC=ON \
  -DMADX_ONLINE=OFF \
  -DBUILD_SHARED_LIBS=OFF \
  -DCMAKE_INSTALL_PREFIX=$MADXDIR \
  -DCMAKE_C_FLAGS="-fvisibility=hidden" \
  -DCMAKE_BUILD_TYPE=Release \
  -DMADX_INSTALL_DOC=OFF \
  ..

which properly found LAPACK and BLAS now that the .a files were present in /usr/lib64/. I could then make and make install to $MADXDIR as foreseen.

Then, in the cpymad clone, building with

python setup.py build_ext --no-X11 --blas --lapack --madxdir=$MADXDIR

works and actually returns useful output (which it didn't before, so I didn't realise things went wrong this way):

running build_ext
building 'cpymad.libmadx' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
creating build/temp.linux-x86_64-3.7/src/cpymad
gcc -pthread -B /home/HPC/oeftiger/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/HPC/oeftiger/aoeftiger/MAD-X/install/include -I/home/HPC/oeftiger/anaconda3/include/python3.7m -c src/cpymad/libmadx.c -o build/temp.linux-x86_64-3.7/src/cpymad/libmadx.o -std=gnu99
gcc -pthread -shared -B /home/HPC/oeftiger/anaconda3/compiler_compat -L/home/HPC/oeftiger/anaconda3/lib -Wl,-rpath=/home/HPC/oeftiger/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/src/cpymad/libmadx.o -L/home/HPC/oeftiger/aoeftiger/MAD-X/install/lib -L/home/HPC/oeftiger/aoeftiger/MAD-X/install/lib64 -lmadx -lptc -lgc-lib -lstdc++ -lgfortran -lquadmath -llapack -lblas -o build/lib.linux-x86_64-3.7/cpymad/libmadx.cpython-37m-x86_64-linux-gnu.so

Importing cpymad.mad and running Madx() does provide a working instance now!