Segfault in JuMP tests on Gurobi 6.5 about gurobi.jl

[mlubin]

versioninfo()?

[JackDunnNZ]

julia> versioninfo()
Julia Version 0.4.1
Commit cbe1bee* (2015-11-08 10:33 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin13.4.0)
  CPU: Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

[mlubin]

It's related to Julia's exception handling and libunwind. If you comment out the @test_throws then everything works.

Here's code to reproduce:

using JuMP, MathProgBase
using Base.Test
using Gurobi
lazy_solvers = [GurobiSolver()]
for lazysolver in lazy_solvers
    entered = [false,false]

    mod = Model(solver=lazysolver)
    @defVar(mod, 0 <= x <= 2, Int)
    @defVar(mod, 0 <= y <= 2, Int)
    @setObjective(mod, Max, y + 0.5x)
    function corners(cb)
        x_val = getValue(x)
        y_val = getValue(y)
        @show x_val, y_val
        TOL = 1e-6
        # Check top right
        if y_val + x_val > 3 + TOL
            @addLazyConstraint(cb, y + 0.5x + 0.5x <= 3)
        end
        entered[1] = true
        @test_throws ErrorException @defVar(cb, z)
        #@test_throws ErrorException @addLazyConstraint(cb, x^2 <= 1)
        println("finished")
    end
    addLazyCallback(mod, corners)
    addLazyCallback(mod, cb -> (entered[2] = true))
    status = solve(mod)
    @show status    

end

Backtrace:

(lldb) bt
* thread #1: tid = 0x9754, 0x0000000100a0e760 libjulia.dylib`libunwind::LocalAddressSpace::get64(this=0x0000000100d4c2a0, addr=140738767689443) + 16 at AddressSpace.hpp:92, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x0000000100a0e760 libjulia.dylib`libunwind::LocalAddressSpace::get64(this=0x0000000100d4c2a0, addr=140738767689443) + 16 at AddressSpace.hpp:92
    frame #1: 0x0000000100a13c6c libjulia.dylib`libunwind::CompactUnwinder_x86_64<libunwind::LocalAddressSpace>::stepWithCompactEncodingFrameless(encoding=50984960, functionStart=13180444720, addressSpace=0x0000000100d4c2a0, registers=0x00007fff5fbfbbe8, indirectStackSize=true) + 2732 at CompactUnwinder.hpp:855
    frame #2: 0x0000000100a12cec libjulia.dylib`libunwind::CompactUnwinder_x86_64<libunwind::LocalAddressSpace>::stepWithCompactEncoding(compactEncoding=50984960, functionStart=13180444720, addressSpace=0x0000000100d4c2a0, registers=0x00007fff5fbfbbe8) + 188 at CompactUnwinder.hpp:549
    frame #3: 0x0000000100a12c29 libjulia.dylib`libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_x86_64>::stepWithCompactEncoding(this=0x00007fff5fbfbb90, (null)=0x00007fff5fbfb660) + 57 at UnwindCursor.hpp:339
    frame #4: 0x0000000100a12bdc libjulia.dylib`libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_x86_64>::stepWithCompactEncoding(this=0x00007fff5fbfbb90) + 60 at UnwindCursor.hpp:337
    frame #5: 0x0000000100a0f4da libjulia.dylib`libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_x86_64>::step(this=0x00007fff5fbfbb90) + 90 at UnwindCursor.hpp:876
    frame #6: 0x0000000100a0e33e libjulia.dylib`::unw_step(cursor=0x00007fff5fbfbb90) + 30 at libuwind.cxx:288
    frame #7: 0x0000000100081e54 libjulia.dylib`jl_backtrace_from_here [inlined] rec_backtrace_ctx(maxsize=80000, uc=0x00000001085625d0) + 80 at task.c:661 [opt]
    frame #8: 0x0000000100081e04 libjulia.dylib`jl_backtrace_from_here [inlined] rec_backtrace(maxsize=80000) + 15 at task.c:645 [opt]
    frame #9: 0x0000000100081df5 libjulia.dylib`jl_backtrace_from_here + 213 at task.c:703 [opt]
    frame #10: 0x0000000104be2129
    frame #11: 0x0000000100017b03 libjulia.dylib`jl_apply_generic [inlined] jl_apply(f=<unavailable>, args=0x00007fff5fbfc0d8, nargs=1) + 14 at julia.h:1325 [opt]
    frame #12: 0x0000000100017af5 libjulia.dylib`jl_apply_generic(F=<unavailable>, args=0x00007fff5fbfc0d8, nargs=1) + 293 at gf.c:1684 [opt]
    frame #13: 0x0000000104bdfe32
    frame #14: 0x0000000100017b03 libjulia.dylib`jl_apply_generic [inlined] jl_apply(f=<unavailable>, args=0x0000000108ae8670, nargs=3) + 14 at julia.h:1325 [opt]
    frame #15: 0x0000000100017af5 libjulia.dylib`jl_apply_generic(F=<unavailable>, args=0x0000000108ae8670, nargs=3) + 293 at gf.c:1684 [opt]
    frame #16: 0x0000000104bdfb2c
    frame #17: 0x0000000104bdf378
    frame #18: 0x0000000104bdee90
    frame #19: 0x0000000311ac60b7 libgurobi65.so`PRIVATE00000000004719d9 + 871
    frame #20: 0x0000000311a28394 libgurobi65.so`PRIVATE000000000038c311 + 1396
    frame #21: 0x0000000311a293bf libgurobi65.so`PRIVATE000000000038e788 + 447
    frame #22: 0x0000000311a95e7b libgurobi65.so`PRIVATE00000000004298d8 + 299
    frame #23: 0x0000000311a9540a libgurobi65.so`PRIVATE00000000004285ac + 698
    frame #24: 0x0000000311a8e778 libgurobi65.so`PRIVATE000000000041cbf4 + 5512
    frame #25: 0x0000000311a88ce7 libgurobi65.so`PRIVATE0000000000417d9c + 727
    frame #26: 0x00000003119fbbf0 libgurobi65.so`PRIVATE00000000003437c8 + 3248
    frame #27: 0x00000003119e6ab0 libgurobi65.so`PRIVATE000000000031a52c + 14848
    frame #28: 0x00000003119da1af libgurobi65.so`PRIVATE000000000030c6a4 + 383
    frame #29: 0x0000000311abdf0f libgurobi65.so`PRIVATE0000000000463b98 + 7279
    frame #30: 0x0000000311abc185 libgurobi65.so`PRIVATE00000000004639c6 + 229
    frame #31: 0x0000000311abbef5 libgurobi65.so`GRBoptimize + 165
    frame #32: 0x0000000104bddf6c
    frame #33: 0x0000000104bdde82
    frame #34: 0x0000000104bddc63
    frame #35: 0x0000000100017b76 libjulia.dylib`jl_apply_generic [inlined] jl_apply(f=<unavailable>, args=0x00000001000179d0, nargs=1) + 422 at julia.h:1325 [opt]
    frame #36: 0x0000000100017b68 libjulia.dylib`jl_apply_generic(F=<unavailable>, args=0x00000001000179d0, nargs=1) + 408 at gf.c:1708 [opt]
    frame #37: 0x0000000104bb6e6b
    frame #38: 0x0000000104bb6491
    frame #39: 0x0000000100017b76 libjulia.dylib`jl_apply_generic [inlined] jl_apply(f=<unavailable>, args=0x000000010936ac01, nargs=1) + 422 at julia.h:1325 [opt]
    frame #40: 0x0000000100017b68 libjulia.dylib`jl_apply_generic(F=<unavailable>, args=0x000000010936ac01, nargs=1) + 408 at gf.c:1708 [opt]
    frame #41: 0x0000000104bb1ad3
    frame #42: 0x000000010008ec7f libjulia.dylib`jl_toplevel_eval_flex [inlined] jl_apply(args=<unavailable>, nargs=0) + 5 at julia.h:1325 [opt]
    frame #43: 0x000000010008ec7a libjulia.dylib`jl_toplevel_eval_flex(e=<unavailable>, fast=<unavailable>) + 1194 at toplevel.c:539 [opt]
    frame #44: 0x000000010008f2ad libjulia.dylib`jl_parse_eval_all(fname=<unavailable>, len=<unavailable>) + 317 at toplevel.c:577 [opt]
    frame #45: 0x000000010008f4d0 libjulia.dylib`jl_load(fname="/Users/mlubin/callback.jl", len=25) + 128 at toplevel.c:617 [opt]
    frame #46: 0x00000001028286e0 sys.dylib`julia_include_678 + 64
    frame #47: 0x0000000100017b03 libjulia.dylib`jl_apply_generic [inlined] jl_apply(f=<unavailable>, args=0x00007fff5fbff330, nargs=1) + 14 at julia.h:1325 [opt]
    frame #48: 0x0000000100017af5 libjulia.dylib`jl_apply_generic(F=<unavailable>, args=0x00007fff5fbff330, nargs=1) + 293 at gf.c:1684 [opt]
    frame #49: 0x00000001029ff143 sys.dylib`julia_include_from_node1_18995 + 387
    frame #50: 0x0000000100017b03 libjulia.dylib`jl_apply_generic [inlined] jl_apply(f=<unavailable>, args=0x00007fff5fbff5b8, nargs=1) + 14 at julia.h:1325 [opt]
    frame #51: 0x0000000100017af5 libjulia.dylib`jl_apply_generic(F=<unavailable>, args=0x00007fff5fbff5b8, nargs=1) + 293 at gf.c:1684 [opt]
    frame #52: 0x00000001029f38a5 sys.dylib`julia_process_options_18615 + 3621
    frame #53: 0x00000001029f1dc8 sys.dylib`julia__start_18614 + 792
    frame #54: 0x00000001029f2a79 sys.dylib`jlcall__start_18614 + 9
    frame #55: 0x0000000100017b03 libjulia.dylib`jl_apply_generic [inlined] jl_apply(f=<unavailable>, args=0x0000000000000000, nargs=0) + 14 at julia.h:1325 [opt]
    frame #56: 0x0000000100017af5 libjulia.dylib`jl_apply_generic(F=<unavailable>, args=0x0000000000000000, nargs=0) + 293 at gf.c:1684 [opt]
    frame #57: 0x0000000100002684 julia`true_main + 276
    frame #58: 0x0000000100002556 julia`main + 86
    frame #59: 0x00000001000018a4 julia`start + 52

CC @Keno

[Keno]

The unwinder can touch invalid memory if it encounters incorrect unwind info. On some platforms we detect this and simply reset. Is this on OS X? I thought we did that there too.

[mlubin]

@Keno, yep OS X

[Keno]

Ok, taking a look at the code, we only do this while profiling, since that's when we're most likely to encounter incorrect unwind info. We may be able to extend the same to regular unwinding as well.

[mlubin]

@Keno, the most likely issue here is that we're giving a julia callback to a C/C++ library which might use C++ exceptions internally

[Keno]

That should generally be fine.

[mlubin]

Ok, taking a look at the code, we only do this while profiling, since that's when we're most likely to encounter incorrect unwind info. We may be able to extend the same to regular unwinding as well.

@Keno, should I open a julia issue for this?

[Keno]

yes

[mlubin]

CC @chriscoey

[mlubin]

Should be fixed by jump-dev/JuMP.jl#924