ledatelescope / bifrost_tcc_wrapper Goto Github PK

(note: this is 'solved', but I'm writing up here for prosperity and discussion)

@LDRyan0 and I have been working on getting the Bifrost TCC plugin working on the topaz supercomputer.

To get it to compile in py 3.6, with the autoconf branch required some makefile edits (we will PR these once we've tidied it up).

Once compiled, we ran into an 'interesting' error when running it:

ctypes.ArgumentError: argument 2: <class 'TypeError'>: 
expected LP_struct_BFarray_ instance instead of struct_BFarray_

This turns out to be due to the definition of class struct_BFarray_(Structure) in btcc_generated.py, which even though it looks identical the definition of class struct_BFarray_ in bifrost.libbifrost_generated, is being considered a different class by ctypes.

Our fix was straightforward in the end: remove this from btcc_generated.py:

# ../src/bifrost/array.h: 128
class struct_BFarray_(Structure):
    pass

struct_BFarray_.__slots__ = [
    'data',
    'space',
    'dtype',
    'ndim',
    'shape',
    'strides',
    'immutable',
    'big_endian',
    'conjugated',
]
struct_BFarray_._fields_ = [
    ('data', POINTER(None)),
    ('space', BFspace),
    ('dtype', BFdtype),
    ('ndim', c_int),
    ('shape', c_long * int(BF_MAX_DIMS)),
    ('strides', c_long * int(BF_MAX_DIMS)),
    ('immutable', BFbool),
    ('big_endian', BFbool),
    ('conjugated', BFbool),
]

BFarray = struct_BFarray_# ../src/bifrost/array.h: 128

and replace with:

from bifrost.libbifrost_generated import struct_BFarray_
BFarray = struct_BFarray_

@jaycedowell: I'm assuming you didn't have to do this?

Background debug info

For some more debug information, the error was being raised here in btcc_genereatd.py:

    BTccExecute = _lib.BTccExecute
    # print("HERE", type(BFarray), type(POINTER(BFarray)))
    BTccExecute.argtypes = [btcc, POINTER(BFarray), POINTER(BFarray), BFbool]
    BTccExecute.restype = BFstatus

The LP_ prefix in LP_struct_BFarray_ is ctypes telling us it is a pointer to the struct_BFarray,and we are indeed passing POINTER(BFarray) to BTccExecute.argtypes. For example, changing the code to POINTER(POINTER(BFarray)) will result in LP_LP_struct_BFarray_.

The expected LP_struct_BFarray_ instance instead of struct_BFarray_ error message is misleading here, removing the POINTER() and running the code will result in a segfault (we do want to pass a pointer after all). The cryptic hint that helped me solve it was when after some trial and error the traceback said:

expected struct_BFarray_ instance instead of struct_BFarray_

!