compiling SCIP and CSIP in macOS about scip.jl

[rschwarz]

Hm, the failure of BinDeps is quite unspecific here.

We had build problems on OS X before, see #21. Unfortunately, I have neither experience with, nor access to any machines with OS X, so I don't know how to make this work.

@chkwon: Could you tell me whether make test in the folder SCIP/deps/src/CSIP-0.3.6 works?

EDIT: sorry, if make does nothing, then the tests won't work either. I thought your output is from running Pkg.build("SCIP") for the second time.

[chkwon]

My apology. This is the real error:

julia> Pkg.add("SCIP")
INFO: Installing SCIP v0.2.0
INFO: Building SCIP
INFO: Attempting to Create directory /Users/chkwon/.julia/v0.5/SCIP/deps/downloads
INFO: Downloading file https://github.com/SCIP-Interfaces/CSIP/archive/v0.3.6.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   125    0   125    0     0    534      0 --:--:-- --:--:-- --:--:--   536
100 21068  100 21068    0     0  42713      0 --:--:-- --:--:-- --:--:-- 42713
INFO: Done downloading file https://github.com/SCIP-Interfaces/CSIP/archive/v0.3.6.zip
INFO: Attempting to Create directory /Users/chkwon/.julia/v0.5/SCIP/deps/src
INFO: Attempting to Create directory /Users/chkwon/.julia/v0.5/SCIP/deps
INFO: Directory /Users/chkwon/.julia/v0.5/SCIP/deps already created
Archive:  /Users/chkwon/.julia/v0.5/SCIP/deps/downloads/v0.3.6.zip
71ca62a4b29352cc5ade76911725f435829ae6c8
   creating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/
  inflating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/.gitignore  
  inflating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/LICENSE  
  inflating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/Makefile  
  inflating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/README.md  
   creating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/include/
  inflating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/include/csip.h  
   creating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/
  inflating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.c  
   creating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/
  inflating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/minunit.h  
  inflating: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/test.c  
INFO: Changing Directory to /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6
mkdir -p /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib
make links
Creating symbolic links to headers and library within SCIPOPTDIR (/Users/chkwon/Applications/scipoptsuite-3.2.1).
gcc -std=c99 -Wall -pedantic -I/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib/include -I/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/include -c /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.c -L/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -Wl,-rpath,/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -lscipopt -fPIC -o /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.o
clang: warning: -Wl,-rpath,/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib: 'linker' input unused
clang: warning: -lscipopt: 'linker' input unused
clang: warning: argument unused during compilation: '-L/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib'
gcc -std=c99 -Wall -pedantic /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.o -L/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -Wl,-rpath,/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -lscipopt -fPIC -shared -o /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib/libcsip.so
INFO: Changing Directory to /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6
INFO: Changing Directory to /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6
=======================================================================[ ERROR: SCIP ]=======================================================================

LoadError: Provider BinDeps.BuildProcess failed to satisfy dependency libcsip
while loading /Users/chkwon/.julia/v0.5/SCIP/deps/build.jl, in expression starting on line 34

=============================================================================================================================================================

======================================================================[ BUILD ERRORS ]=======================================================================

WARNING: SCIP had build errors.

 - packages with build errors remain installed in /Users/chkwon/.julia/v0.5
 - build the package(s) and all dependencies with `Pkg.build("SCIP")`
 - build a single package by running its `deps/build.jl` script

=============================================================================================================================================================
INFO: Package database updated

and make test in SCIP/deps/src/CSIP-0.3.6:

:~/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6$ make test
compiling test
gcc -std=c99 -Wall -pedantic -I/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/include -g /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/test.c -Wl,-rpath,/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -Wl,-rpath,/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -lm -lcsip -lscipopt -L/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -o /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/test
/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/test 
dyld: Library not loaded: lib/libscipopt-3.2.1.darwin.x86_64.gnu.opt.so
  Referenced from: /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/test
  Reason: image not found
make: *** [test] Abort trap: 6

[rschwarz]

What I don't know is whether

1. A different Makefile (or options) should be used to compile the libscipopt, or
2. The Makefile for CSIP needs to updated, or
3. Both :-\

In the failing make test when it complains that "Library not loaded", does this file exist? Is it maybe a naming issue (with darwin)?

[chkwon]

A few observations:

1. libscipopt-3.2.1.darwin.x86_64.gnu.opt.so does exist in $SCIPOPTDIR/lib.
2. libscipopt-3.2.1.darwin.x86_64.gnu.opt.so does NOT exist in CSIP-0.3.6/lib:

~/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib$ ls -al
total 104
drwxr-xr-x   5 chkwon  staff   170B Mar  6 14:29 ./
drwxr-xr-x  10 chkwon  staff   340B Mar  6 12:54 ../
lrwxr-xr-x   1 chkwon  staff    60B Mar  6 12:54 include@ -> /Users/chkwon/Applications/scipoptsuite-3.2.1/scip-3.2.1/src
-rwxr-xr-x   1 chkwon  staff    41K Mar  6 14:29 libcsip.so*
lrwxr-xr-x   1 chkwon  staff    63B Mar  6 12:54 libscipopt.so@ -> /Users/chkwon/Applications/scipoptsuite-3.2.1/lib/libscipopt.so

3. $SCIPOPTDIR/lib is included in LD_LIBRARY_PATH.

[chkwon]

I don't have good memories with .so, but had some success with .dylib. I guess both SCIP and CSIP need to be .dylib?

[rschwarz]

Sadly, I know nothing about .dylib.

About the missing files: I think it's OK that only libscipopt.so is there, because this one should be used by the CSIP Makefile. Anyway, it's a symlink to SCIP's libscipopt.so which is itself a symlink to the most recently compiled library, in your case libscipopt-3.2.1.darwin.x86_64.gnu.opt.so.

[rschwarz]

By accident I found this older StackOverflow question, where a user tried to use PySCIPOpt on OS X. There was the issue with the shared library not found, and it was suggested to set DYLD_LIBRARY_PATH to include the path where libscipopt-3.2.1.darwin.x86_64.gnu.opt.so is located. Apparently this helped the user to go one step further.

I'm not sure why this would be necessary, but may it helps us understand the issue better.

[chkwon]

Unfortunately, this does not help. It generates the same error.

I tried a couple of things:

1. I tried physically copying libscipopt-3.2.1.darwin.x86_64.gnu.opt.so to CSIP-0.3.6/lib. It was successful:

chkwon@MacBook:~/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6$ make test
make[1]: `/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/test' is up to date.
/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/test/test 
Running tests...
  test_lp
  test_mip
  test_mip2
  test_mip3
  test_socp
  test_nlp
  test_quadobj
  test_lazy
  test_lazy2
  test_lazy_interrupt
  test_objsense
  test_sos1
  test_sos2
  test_sos1_sos2
  test_manythings
  test_doublelazy
  test_changeprob
  test_changequadprob
  test_changevartype
  test_initialsol
  test_heurcb
  test_params
All tests passed!

2. After creating a symbolic link named libscipopt-3.2.1.darwin.x86_64.gnu.opt.so, instead of libscipopt.so, it was successful. (without a physical copy)

Two guesses:

1. It seems like CSIP is looking for lib\libscipopt-3.2.1.darwin.x86_64.gnu.opt.so, not just libscipopt-3.2.1.darwin.x86_64.gnu.opt.so. Specifically in the lib directory of CSIP.

2. It does not know if libscipopt-3.2.1.darwin.x86_64.gnu.opt.so is the same one as libscipopt.so. Perhaps it knows the other way around.

[chkwon]

To compile dylib files, a flag -dynamiclib needs to be added. It seems that dylib files are needed for both SCIP and CSIP.

Could you tell me where I should add the flag -dynamiclib for SCIP? I also need to change the file extension .so to .dylib. I tried to figure it out by myself, but the makefile in SCIP is very complicated. Any advice will be useful.

[chkwon]

@leethargo I didn't know how to edit makefiles for SCIP, but I had a success.

1. To compile libsciopt.dylib:

After generating all object files following the main instruction of this package, I used the following line:

 g++ -install_name @rpath/libscipopt.dylib -dynamiclib -undefined suppress -flat_namespace -m64 -shared -o lib/libscipopt.dylib obj/*.o /Users/chkwon/Applications/scipoptsuite-3.2.1/scip-3.2.1/obj/O.darwin.x86_64.gnu.opt/lib/objscip/*.o /Users/chkwon/Applications/scipoptsuite-3.2.1/soplex-2.2.1/obj/O.darwin.x86_64.gnu.opt/lib/*o

2. To compile libcsip.dylib:

gcc -std=c99 -Wall -pedantic -I/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib/include -I/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/include -c /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.c -L/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -Wl,-rpath,/Users/chkwon/Applications/scipoptsuite-3.2.1/lib/ /Users/chkwon/Applications/scipoptsuite-3.2.1/lib/libscipopt.dylib -fPIC -o /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.o

and

gcc -std=c99 -Wall -pedantic /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.o -L/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -Wl,-rpath,/Users/chkwon/Applications/scipoptsuite-3.2.1/lib/ /Users/chkwon/Applications/scipoptsuite-3.2.1/lib/libscipopt.dylib -fPIC -shared -dynamiclib -o /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib/libcsip.dylib

I also copied libcsip.dylib to deps/usr/lib.

3. deps.jl

@checked_lib libcsip "/Users/chkwon/.julia/v0.5/SCIP/deps/usr/lib/libcsip.dylib"

Then it worked.

[fserra]

btw you can compile with USRFLAGS=theflagyouwant

…

On Mar 10, 2017 19:48, "Changhyun Kwon" ***@***.***> wrote: @leethargo <https://github.com/leethargo> I didn't know how to edit makefiles for SCIP, but I had a success. 1. To compile libsciopt.dylib: After generating all object files following the main instruction of this package, I used the following line: g++ -install_name @rpath/libscipopt.dylib -dynamiclib -undefined suppress -flat_namespace -m64 -shared -o lib/libscipopt.dylib obj/*.o /Users/chkwon/Applications/scipoptsuite-3.2.1/scip-3.2.1/obj/O.darwin.x86_64.gnu.opt/lib/objscip/*.o /Users/chkwon/Applications/scipoptsuite-3.2.1/soplex-2.2.1/obj/O.darwin.x86_64.gnu.opt/lib/*o 1. To compile libcsip.dylib: gcc -std=c99 -Wall -pedantic -I/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib/include -I/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/include -c /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.c -L/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -Wl,-rpath,/Users/chkwon/Applications/scipoptsuite-3.2.1/lib/ /Users/chkwon/Applications/scipoptsuite-3.2.1/lib/libscipopt.dylib -fPIC -o /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.o and gcc -std=c99 -Wall -pedantic /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/src/csip.o -L/Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib -Wl,-rpath,/Users/chkwon/Applications/scipoptsuite-3.2.1/lib/ /Users/chkwon/Applications/scipoptsuite-3.2.1/lib/libscipopt.dylib -fPIC -shared -dynamiclib -o /Users/chkwon/.julia/v0.5/SCIP/deps/src/CSIP-0.3.6/lib/libcsip.dylib I also copied libcsip.dylib to deps/usr/lib. 1. deps.jl @checked_lib libcsip "/Users/chkwon/.julia/v0.5/SCIP/deps/usr/lib/libcsip.dylib" Then it worked. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFNG9mG5Bg8YSST97gHrnZOx7xHm5KZ6ks5rkZr1gaJpZM4MURV5> .

[chkwon]

@fserra Thanks. I also needed to change the output library file extension from .so to .dylib.