trolldbois / ctypeslib Goto Github PK

your mom gAY

test.h:

#define HI(x) x

HI(int) y;

Traceback (most recent call last):
  File "/bin/clang2py", line 10, in <module>
    sys.exit(main())
  File "/lib/python3.6/site-packages/ctypeslib/clang2py.py", line 306, in main
    flags=clang_opts)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/codegenerator.py", line 888, in generate_code
    parser.parse(srcfile)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/clangparser.py", line 120, in parse
    self.startElement(node)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/clangparser.py", line 159, in startElement
    stop_recurse = self.parse_cursor(node)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/clangparser.py", line 273, in parse_cursor
    return self.cursorkind_handler.parse_cursor(cursor)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
    return mth(cursor)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/util.py", line 119, in fn
    return func(*args, **kwargs)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/cursorhandler.py", line 1041, in MACRO_DEFINITION
    tokens = self._literal_handling(cursor)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/util.py", line 119, in fn
    return func(*args, **kwargs)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/cursorhandler.py", line 528, in _literal_handling
    value = self.get_registered(value).body
  File "/lib/python3.6/site-packages/ctypeslib/codegen/handler.py", line 47, in get_registered
    return self.parser.get_registered(name)
  File "/lib/python3.6/site-packages/ctypeslib/codegen/clangparser.py", line 188, in get_registered
    return self.all[name]
KeyError: 'x'

clang2py doesn't generate code for incomplete C structures.

Considering next header file:

#ifndef T_H
#define T_H

struct Foo;
void do_something(struct Foo* foo);

#endif

clang2py issues a few warnings and doesn't generate definitions for struct Foo and do_something

clang2py -o t.py t.h 
ERROR:cursorhandler:Structure struct_Foo is Incomplete t.h:5 align:-2 size:-2
ERROR:cursorhandler:Structure struct_Foo is Incomplete t.h:5 align:-2 size:-2

Generated t.py is empty:

# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes




__all__ = \
    []

Having declared, but not defined structures is most common way in C to hide library internals and it's widely used.

When using clang2py on Ubuntu (clang 4.0) it is necessary to add default includes directory to the search path of clang. It could be noted somehow in the README file. It would also be handy if the script forward clangparser warnings to stdout by default.

run:

clang2py FOO.h --preload /home/myuser/libMYLIB.so -k dfest -d -o pymylib.py

# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes

CDLL('/home/myuser/libMYLIB.so', RTLD_GLOBAL)
....

I suppose it should be

# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes

ctypes.CDLL('/home/myuser/libMYLIB.so', ctypes.RTLD_GLOBAL)
....

There is a bug which causes unnamed structs and unnamed enums not to be parsed. If you show interest in correcting this issue I will write up a test case which demonstrates the problem.

File "/Library/Python/2.7/site-packages/ctypeslib2-2.0-py2.7.egg/ctypeslib/codegen/clangparser.py", line 224, in get_unique_name
_id = cursor.get_usr()
AttributeError: 'Type' object has no attribute 'get_usr'

Not sure if I’m using a different version of the bindings than you, or what… Do the tests complete?

Create a better documentation, examples and list usage for all arguments on the command line

• --clang-args CLANG_ARGS clang options

  • The argument to this long option actually needs to be set with an equals:

  $ clang2py --clang-args -std=c99 test.h > /dev/null || echo failed
  usage: clang2py [-h] [-c] [-d] [--debug] [-e] [-k TYPEKIND] [-i] [-l DLLS]
                  [-m module] [-o OUTPUT] [-p DLL] [-q] [-r EXPRESSION]
                  [-s SYMBOL] [-t TARGET] [-v] [-V] [-w W] [-x]
                  [--show-ids SHOWIDS] [--max-depth N] [--clang-args CLANG_ARGS]
                  files [files ...]
  clang2py: error: argument --clang-args: expected one argument
  failed
  $ clang2py --clang-args=-std=c99 test.h > /dev/null && echo success
  success
  

  • Multiple clang options also need to be quoted:

  $ clang2py --clang-args=-std=c99 -Wall test.h > /dev/null || echo failed
  usage: clang2py [-h] [-c] [-d] [--debug] [-e] [-k TYPEKIND] [-i] [-l DLLS]
                  [-m module] [-o OUTPUT] [-p DLL] [-q] [-r EXPRESSION]
                  [-s SYMBOL] [-t TARGET] [-v] [-V] [-w W] [-x]
                  [--show-ids SHOWIDS] [--max-depth N] [--clang-args CLANG_ARGS]
                  files [files ...]
  clang2py: error: unrecognized arguments: -Wall
  failed
  $ clang2py --clang-args="-std=c99 -Wall" test.h > /dev/null && echo success
  success
  

• -l DLLS, --include-librarie DLLS libraries to search for exported functions

  • Singular of 'libraries' is 'library', with a Y.
  • This option doesn't actually appear to support multiple DLL arguments, just one:

  $ clang2py --include-librarie /lib/x86_64-linux-gnu/libc.so.6 test.h >/dev/null && echo success
  success
  $ clang2py --include-librarie /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libm.so.6 test.h > /dev/null || echo failed
  Traceback (most recent call last):
    File "/usr/local/bin/clang2py", line 6, in <module>
      sys.exit(main())
    File "/usr/local/lib/python2.7/dist-packages/ctypeslib/clang2py.py", line 310, in main 
      flags=clang_opts)
    File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/codegenerator.py", line 872, in generate_code
      parser.parse(srcfile)
    File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/clangparser.py", line 103, in parse
      self.tu = index.parse(filename, self.flags, options=self.tu_options)
    File "/usr/lib/python2.7/dist-packages/clang/cindex.py", line 2210, in parse
      self)
    File "/usr/lib/python2.7/dist-packages/clang/cindex.py", line 2322, in from_source
      raise TranslationUnitLoadError("Error parsing translation unit.")
  clang.cindex.TranslationUnitLoadError: Error parsing translation unit.
  failed
  

This causes an error:

    struct foo {
     int bar;
    };

    typedef struct foo foo_t[256];

    typedef struct {
        foo_t baz;
    } __somestruct;

File "ctypeslib/codegen/clangparser.py", line 1035, in FIELD_DECL
raise TypeError('Field can not be None %s'%(name ))
TypeError: Field can not be None baz

I'm sorry, but I don't understand why to set pack to True always.
This will end up with wrong offset & sizeof.

Parsing radare2 header and have this error:

2017-12-28 12:23:23,355 'r_userconf.h' file not found with <angled> include; use "quotes" instead
2017-12-28 12:23:23,355 Source code has some error. Please fix.
2017-12-28 12:23:23,355 'r_userconf.h' file not found with <angled> include; use "quotes" instead

When running the following through clang2py

#define PACK __attribute__((packed,aligned(2))) 
typedef struct {
    long a;
    short b;
    char c;
    double d;
} PACK test;

I get this as ouput

class struct_c__SA_test(ctypes.Structure):
    _pack_ = True # source:True
    _fields_ = [
    ('a', ctypes.c_int32),
    ('b', ctypes.c_int16),
    ('c', ctypes.c_ubyte),
    ('d', ctypes.c_double),
    ('PADDING_0', ctypes.c_ubyte),
     ]
test = struct_c__SA_test

In the ctypes docs is says that pack should be a number (of bytes). When run through the original h2xml/xml2py the pack attribute is set to 2, as expected. Shouldn't this be the behavior of ctypeslib2 as well?

setuptools automatically includes test/test*.py, but omits test/__init__.py , despite unittest needing a __init__.py to work on Python 2.

Very annoying, but anyways the __init__.py has some import bits needed, so needs to be included.

Consider the source and generated python below.

#define PRE "before"
#define POST " after"
#define PREPOST PRE POST

char a[] = "what";
char b[] = "why" " though";
char c[] = PRE POST;
char d[] = PREPOST;

PRE = "before" # macro
POST = "after" # macro
PREPOST = ['PRE', 'POST'] # macro
a = (ctypes.c_ubyte * 4).in_dll(_libraries['/home/debian/sonicsd/test.so'], 'a')
b = (ctypes.c_ubyte * 3).in_dll(_libraries['/home/debian/sonicsd/test.so'], 'b')
c = (ctypes.c_ubyte * 3).in_dll(_libraries['/home/debian/sonicsd/test.so'], 'c')
d = (ctypes.c_ubyte * 7).in_dll(_libraries['/home/debian/sonicsd/test.so'], 'd')

The size for the char a[] is correct, but if initialized with multiple strings (as with char b[] ) it only gets the length of the first string. If it is initialized with a macro it (char c[]) it gets the length of the macro name instead of what it expands too. I have tried this both with and without linking in the shared lib.

Also note that the code generated for the PREPOST macro is not the same as what would be generated in h2xml/xml2py, which would expand the macro so that PREPOST = "why though" in the generated python. I can see how this behavior may not always be desired, but it would be good to have as an option. This may deserve its own issue but it seemed related.

Could this have anything to do with the error messages clang2py outputs:

WARNING:handler:MACRO_INSTANTIATION is not handled
WARNING:handler:_do_nothing for MACRO_INSTANTIATION/T1
WARNING:handler:_do_nothing for MACRO_INSTANTIATION/PRE
WARNING:handler:_do_nothing for MACRO_INSTANTIATION/POST
WARNING:handler:_do_nothing for MACRO_INSTANTIATION/PREPOST

Hi. I am trying to create bindings for libjxr. I get some output that looks somehow usable, but the structures have no useful attributes, it all looks like:

PKFactory = struct_tagPKFactory
class struct_tagPKCodecFactory(ctypes.Structure):
    _pack_ = True # source:False
    _fields_ = [
    ('PADDING_0', ctypes.c_ubyte),
     ]

PKCodecFactory = struct_tagPKCodecFactory
class struct_tagPKImageDecode(ctypes.Structure):
    _pack_ = True # source:False
    _fields_ = [
    ('PADDING_0', ctypes.c_ubyte),
     ]

PKImageDecode = struct_tagPKImageDecode
class struct_tagPKImageEncode(ctypes.Structure):
    _pack_ = True # source:False
    _fields_ = [
    ('PADDING_0', ctypes.c_ubyte),
     ]

PKImageEncode = struct_tagPKImageEncode

Any hints?...

When adding "m" to the -k option, I found that clang2py outputs additional, system-level macros that are missing dependencies. These macros are actually not necessary and do not exist when I was using h2xml/xml2py.

Example:

WCHAR_MIN = __WCHAR_MIN
__ASSERT_FUNCTION = PRETTY_FUNCTION

But the right-hand side symbols are not defined.

If I remove "m" from "-k", these symbols do not exit, but neither do other useful macros.

Is there a way to exclude those useless macros?

Thanks.

To support commonly installed Clang 8.0, and Clang 9.0.
Sadly this is blocked by the ethanhs/clang#4

Could be a mix of a few different bugs in here.

[   57s] ======================================================================
[   57s] ERROR: test_char (test_types_values.ConstantsTest)
[   57s] ----------------------------------------------------------------------
[   57s] Traceback (most recent call last):
[   57s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 123, in test_char
[   57s]     """)
[   57s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 168, in convert
[   57s]     self.gen(hfile, flags, debug)
[   57s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 153, in gen
[   57s]     exec(output, namespace)
[   57s]   File "<string>", line 12, in <module>
[   57s] NameError: name 'x' is not defined
[   57s] 
[   57s] ======================================================================
[   57s] FAIL: test_includes (test_fast_clang.CompareSizes)
[   57s] Test sizes of pod with std include.
[   57s] ----------------------------------------------------------------------
[   57s] Traceback (most recent call last):
[   57s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_fast_clang.py", line 54, in test_includes
[   57s]     self.assertSizes(name)
[   57s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 189, in assertSizes
[   57s]     name)
[   57s] AssertionError: False is not true : int8_t was not found in source
[   57s] 
[   57s] ======================================================================
[   57s] FAIL: test_array (test_types_values.ConstantsTest)
[   57s] ----------------------------------------------------------------------
[   57s] Traceback (most recent call last):
[   57s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 163, in test_array
[   57s]     self.assertEqual(self.namespace.c1, [])
[   57s] AssertionError: <class 'c_ubyte_Array_1'> != []
[   57s] 
[   57s] ======================================================================
[   57s] FAIL: test_char_p (test_types_values.ConstantsTest)
[   57s] ----------------------------------------------------------------------
[   57s] Traceback (most recent call last):
[   57s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 136, in test_char_p
[   57s]     self.assertEqual(self.namespace.x, [])
[   57s] AssertionError: <class 'c_ubyte_Array_10'> != []
[   57s] 
[   57s] ======================================================================
[   57s] FAIL: test_extern_function_pointer (test_types_values.ConstantsTest)
[   57s] ----------------------------------------------------------------------
[   57s] Traceback (most recent call last):
[   57s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 271, in test_extern_function_pointer
[   57s]     'LP_c_char')
[   57s] AssertionError: 'LP_c_ubyte' != 'LP_c_char'
[   57s] - LP_c_ubyte
[   57s] + LP_c_char

clang2py doesn't generate constants/fields for enums.

enum myEnum {
ONE,
TWO,
FOUR = 4 
};

Generate with

clang2py t.h -o t.py -v --debug 2>&1
DEBUG:clangparser:ARCH sizes: long:c_int64 longdouble:c_long_double_t
DEBUG:clangparser:t.h:1: Found a ENUM_DECL|myEnum|myEnum
DEBUG:handler:get_unique_name: name "myEnum"
DEBUG:utils:ENUM_DECL: displayname:'myEnum'
DEBUG:handler:get_unique_name: name "myEnum"
DEBUG:clangparser:register: myEnum 
DEBUG:codegen:Input was parsed
DEBUG:codegen:generate Enumeration, myEnum
###########################
# Symbols defined:
#
# Variables:              0
# Struct/Unions:          0
# Functions:              0
# Enums:                  1
# Enum values:            0
# Typedefs:               0
# Pointertypes:           0
# Arraytypes:             0
# unknown functions:      0
# unknown variables:      0
#
# Total symbols:     1
###########################
needed 1 loop(s)

Output is:

# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes





# values for enumeration 'myEnum'
myEnum = ctypes.c_int # enum
__all__ = \
    ['myEnum']

Expected output something like this:

ONE = 0
TWO = 1
FOUR = 4

# values for enumeration 'myEnum'
myEnum = ctypes.c_int # enum
__all__ = \
    ['myEnum', 'ONE', 'TWO', 'FOUR']

Hello, I was trying to use the ctypeslib, installed from master branch and python-clang==3.5

When running clang2py NBioAPI_IndexSearch.h result in error

Do you have any idea what might be going wrong?

If help on something, the header that I was testing this is here: https://github.com/luzfcb/nbiosearch/blob/master/nbiosearch/eNBSP/include/NBioAPI_IndexSearch.h

sutransdev@sutransdev:~/nbiosearch/nbiosearch/eNBSP/include$ clang2py NBioAPI_IndexSearch.h 
WARNING:clangparser:Source code has some error. Please fix.
WARNING:clangparser:unknown type name 'wchar_t'
Traceback (most recent call last):
  File "/usr/local/bin/clang2py", line 6, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/clang2py.py", line 235, in main
    flags=clang_opts)
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/codegenerator.py", line 777, in generate_code
    parser.parse(srcfile)
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/clangparser.py", line 114, in parse
    self.startElement( node )
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/clangparser.py", line 125, in startElement
    stop_recurse = self.parse_cursor(node)
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/clangparser.py", line 232, in parse_cursor
    return self.cursorkind_handler.parse_cursor(cursor)
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
    return mth(cursor)
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/util.py", line 114, in fn
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/cursorhandler.py", line 498, in STRUCT_DECL
    return self._record_decl(cursor, typedesc.Structure)
  File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/cursorhandler.py", line 553, in _record_decl
    fields = list(cursor.type.get_fields())
AttributeError: 'Type' object has no attribute 'get_fields'

I am trying to package this for openSUSE Python 3.7, and almost have it passing all tests, only a few remaining issues, which could be my fault, or stuff fixed since 2.2.2 release, or needed enhancements.

[   23s] test_uint_minus_one (test_types_values.ConstantsTest) ... expected failure
[   23s] test_unicode (test_types_values.ConstantsTest)
[   23s] unicode conversion test from unittest in clang ... 'stddef.h' file not found
[   23s] Source code has some error. Please fix.
[   23s] 'stddef.h' file not found
[   23s] ERROR
[   23s] test_unicode_cpp11 (test_types_values.ConstantsTest)
[   23s] unicode conversion test from unittest in clang ... 'stddef.h' file not found
[   23s] Source code has some error. Please fix.
[   23s] 'stddef.h' file not found
[   23s] ERROR
[   23s] test_unicode_wchar (test_types_values.ConstantsTest)
[   23s] unicode conversion test from unittest in clang ... 'stddef.h' file not found
[   23s] Source code has some error. Please fix.
[   23s] 'stddef.h' file not found
[   23s] expected failure
[   23s] test_var (test_types_values.ConstantsTest)
[   23s] Basic POD test variable declaration' ... ok
[   23s] test_var_decl_and_scope (test_types_values.ConstantsTest) ... ok
[   23s] test_wchar (test_types_values.ConstantsTest) ... ok
[   23s] 
[   23s] ======================================================================
[   23s] ERROR: test_unicode (test_types_values.ConstantsTest)
[   23s] unicode conversion test from unittest in clang
[   23s] ----------------------------------------------------------------------
[   23s] Traceback (most recent call last):
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 83, in test_unicode
[   23s]     self.gen('test/data/test-strings.cpp', ['-x', 'c++'])
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 141, in gen
[   23s]     gen = self._gen(ofi, fname, flags)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 129, in _gen
[   23s]     self.parser.parse(fname)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 120, in parse
[   23s]     self.startElement(node)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 159, in startElement
[   23s]     stop_recurse = self.parse_cursor(node)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 273, in parse_cursor
[   23s]     return self.cursorkind_handler.parse_cursor(cursor)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
[   23s]     return mth(cursor)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/util.py", line 119, in fn
[   23s]     return func(*args, **kwargs)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 249, in VAR_DECL
[   23s]     init_value = self._VAR_DECL_value(cursor, _type)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 299, in _VAR_DECL_value
[   23s]     list(cursor.get_children()))
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 352, in _get_var_decl_init_value
[   23s]     _tmp = self._get_var_decl_init_value_single(_ctype, child)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 385, in _get_var_decl_init_value_single
[   23s]     child.get_children())
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 352, in _get_var_decl_init_value
[   23s]     _tmp = self._get_var_decl_init_value_single(_ctype, child)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 394, in _get_var_decl_init_value_single
[   23s]     _v = self.parse_cursor(child)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
[   23s]     return mth(cursor)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/util.py", line 119, in fn
[   23s]     return func(*args, **kwargs)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 463, in _literal_handling
[   23s]     str([str(t.spelling) for t in tokens]))
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 463, in <listcomp>
[   23s]     str([str(t.spelling) for t in tokens]))
[   23s]   File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 3294, in spelling
[   23s]     return conf.lib.clang_getTokenSpelling(self._tu, self)
[   23s]   File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 230, in from_result
[   23s]     return conf.lib.clang_getCString(res)
[   23s]   File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 105, in to_python_string
[   23s]     return x.value
[   23s]   File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 90, in value
[   23s]     return super(c_char_p, self).value.decode("utf8")
[   23s] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 1: invalid start byte
[   23s] 
[   23s] ======================================================================
[   23s] ERROR: test_unicode_cpp11 (test_types_values.ConstantsTest)
[   23s] unicode conversion test from unittest in clang
[   23s] ----------------------------------------------------------------------
[   23s] Traceback (most recent call last):
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 103, in test_unicode_cpp11
[   23s]     self.gen('test/data/test-strings.cpp', ['-x', 'c++', '--std=c++11'])
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 141, in gen
[   23s]     gen = self._gen(ofi, fname, flags)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 129, in _gen
[   23s]     self.parser.parse(fname)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 120, in parse
[   23s]     self.startElement(node)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 159, in startElement
[   23s]     stop_recurse = self.parse_cursor(node)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 273, in parse_cursor
[   23s]     return self.cursorkind_handler.parse_cursor(cursor)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
[   23s]     return mth(cursor)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/util.py", line 119, in fn
[   23s]     return func(*args, **kwargs)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 249, in VAR_DECL
[   23s]     init_value = self._VAR_DECL_value(cursor, _type)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 299, in _VAR_DECL_value
[   23s]     list(cursor.get_children()))
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 352, in _get_var_decl_init_value
[   23s]     _tmp = self._get_var_decl_init_value_single(_ctype, child)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 385, in _get_var_decl_init_value_single
[   23s]     child.get_children())
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 352, in _get_var_decl_init_value
[   23s]     _tmp = self._get_var_decl_init_value_single(_ctype, child)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 394, in _get_var_decl_init_value_single
[   23s]     _v = self.parse_cursor(child)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
[   23s]     return mth(cursor)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/util.py", line 119, in fn
[   23s]     return func(*args, **kwargs)
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 463, in _literal_handling
[   23s]     str([str(t.spelling) for t in tokens]))
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 463, in <listcomp>
[   23s]     str([str(t.spelling) for t in tokens]))
[   23s]   File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 3294, in spelling
[   23s]     return conf.lib.clang_getTokenSpelling(self._tu, self)
[   23s]   File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 230, in from_result
[   23s]     return conf.lib.clang_getCString(res)
[   23s]   File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 105, in to_python_string
[   23s]     return x.value
[   23s]   File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 90, in value
[   23s]     return super(c_char_p, self).value.decode("utf8")
[   23s] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 1: invalid start byte
[   23s] 
[   23s] ======================================================================
[   23s] FAIL: test_variable (test_clang2py.ArgumentTypeKind)
[   23s] run clang2py -k d test/data/test-strings.cpp
[   23s] ----------------------------------------------------------------------
[   23s] Traceback (most recent call last):
[   23s]   File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_clang2py.py", line 109, in test_variable
[   23s]     self.assertEqual(p.returncode, 0)
[   23s] AssertionError: 1 != 0
[   23s] 
[   23s] ----------------------------------------------------------------------
[   23s] Ran 79 tests in 8.790s
[   23s] 
[   23s] FAILED (failures=1, errors=2, skipped=4, expected failures=5)
[   23s] Test failed: <unittest.runner.TextTestResult run=79 errors=2 failures=1>
[   23s] error: Test failed: <unittest.runner.TextTestResult run=79 errors=2 failures=1>

records should be aware of the PACKED_ATTR.

see trolldbois/python-clang#1

Gives

LibclangError: [WinError 126] The specified module could not be found. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

both in console and notebook mode.

Problem:

$ pip install ctypeslib2
$ python3
>>> import ctypeslib
No module named 'clang'

Solution:

$ pip install clang
$ python3
>>> import ctypeslib
>>> # ok

Suggested fix:
Add the line from requirements.txt to setup.py install_requires. I did some preliminary investigation to use requirements.txt from setup.py and found this.

It is not at all clear what the dependencies of clang2py are.
For example, clang2py fails with key error "packed" with libclang-6.0-dev on debian and pypi clang

In short, ctypeslib2 implements a module named ctypeslib, which clobbers the legacy ctypeslib module without providing backwards compatibility. This breaks existing workflows and prevents their straightforward transition from gccxml to clang.

Example:

# h2xml --help
Usage: h2xml includefile ... [options]

Options:
  -h, --help         show this help message and exit
  -q, --quiet        
  -D NAME[=VALUE]    macros to define
  -U NAME            macros to undefine
  -I DIRECTORY       additional include directories
  -o XMLFILE         XML output filename
  -c, --cpp-symbols  try to find #define symbols - this may give compiler
                     errors, so it's off by default.
  -k                 don't delete the temporary files created (useful for
                     finding problems)
#
# pip install ctypeslib2
Downloading/unpacking ctypeslib2
  Downloading ctypeslib2-2.1.4.tar.gz (46kB): 46kB downloaded
  Running setup.py (path:/tmp/pip-build-dmhvZK/ctypeslib2/setup.py) egg_info for package ctypeslib2
    
Requirement already satisfied (use --upgrade to upgrade): clang>=3.7 in /usr/local/lib/python2.7/dist-packages (from ctypeslib2)
Installing collected packages: ctypeslib2
  Running setup.py install for ctypeslib2
    changing mode of build/scripts-2.7/clang2py from 644 to 755
    
    changing mode of /usr/local/bin/clang2py to 755
Successfully installed ctypeslib2
Cleaning up...
#
# h2xml --help
Traceback (most recent call last):
  File "/usr/bin/h2xml", line 3, in <module>
    from ctypeslib.h2xml import main
ImportError: No module named h2xml

I would suggest that the module provided by ctypeslib2 should be called ctypeslib2 if it is not a functional replacement for the existing ctypeslib module.

These tests currently cause (two different) exceptions in clang parser:

def test_extern_function_pointer(self):
    ns = self.convert('''
    extern int (*func_ptr)(const char *arg);
    ''')

def test_extern_function_pointer_multiarg(self):
    ns = self.convert('''
    extern int (*func_ptr)(const char *arg, int c);
    ''')

The header files I am trying to process with clang2py.py seem to fail because of a multi-line #define macro definition (where each line, excluding the last one, ends with a backslash \ character). If I comment out this macro and instead copy and paste the definition into each location the macro was used, the code is processed normally. Are multi-line #defines not supported? Would it be feasible to add this support?

Hi, I installed ctypeslib2 via pip: pip install ctypeslib2. It installed clang 6.0 and ctypeslib2, on my windows 10 64 bit, python 3.6 and anaconda.
I have clang 3.9.1 and clangdev installed on my anaconda.
I tried clang2py with an .h file, but receive the following:
function not found clang_CXXRecord_isAbstract. Please check compatibility of libclang.so version.
Now, I believe ctypeslib2 doesn't work on windows.
Any help?

Is it possible to avoid following exception?:
ctypeslib.codegen.handler.DuplicateDefinitionException: 'register: struct_mystruct_t already existed: struct_mystruct_t'

I am defining a name (myhandle) for an internal structure (mystruct_t), which is defined in an internal header file but hidden from API (opaque pointer):
typedef struct mystruct_t myhandle;

When this header file is included in multiple places and they are fed to clang2py, it stops with above exception. Is there a way to handle this without error?

if the source code is faulty, the codegeneration takes place without warning.
So sometimes, record size are "faulty" because the standard says that a faulty record is of size 1.

const char16_t const_utf16_char = u'あ';

produces

const_utf16_char = あ

Also would be nice to have option to ignore const scalar "variables", since they are almost always inlined.

https://github.com/trolldbois/ctypeslib/blob/master/ctypeslib/codegen/codegenerator.py#L736

archtypes should be generated, or generable

https://github.com/radare/radare2/issues/6350

If a record has an incompletearray, all fields have offset == -2. No padding will be done.
But the fields are ordered and code will be produces with typed info in the right offset.
so in most cases, it will work.

if there is a structure with incompletearray and padding or alignement issue, it will produce wrong results

Bug/corner case is probably in llvm python clang bindings/library libclang

typedef struct
{
  int x;
} foobar;

extern foobar something;

This errors in Clang_Parser.VAR_DECL because _ctype.KIND == TypeKind.RECORD is not handled.

Consider the following example:

struct example_detail {
	int first;
	int last;
};

struct example {
	int argsz;
	int flags;
	int count;
	struct example_detail details[];
};

From these structs, clang2py generates:

import ctypes

class struct_example_detail(ctypes.Structure):
    _pack_ = True # source:False
    _fields_ = [
    ('first', ctypes.c_int32),
    ('last', ctypes.c_int32),
     ]

class struct_example(ctypes.Structure):
    _pack_ = True # source:False
    _fields_ = [
    ('argsz', ctypes.c_int32),
    ('flags', ctypes.c_int32),
    ('count', ctypes.c_int32),
    ('details', struct_example_detail * 0),
    ('PADDING_0', ctypes.array_struct_example_detail, 98),
     ]

Which doesn't work:

Traceback (most recent call last):
  File "test.py", line 10, in 
    class struct_example(ctypes.Structure):
  File "test.py", line 17, in struct_example
    ('PADDING_0', ctypes.array_struct_example_detail, 98),
AttributeError: 'module' object has no attribute 'array_struct_example_detail'

By contrast, this worked in legacy ctypeslib, where h2xml/xml2py produced:

from ctypes import *

class example_detail(Structure):
    pass
example_detail._fields_ = [
    ('first', c_int),
    ('last', c_int),
]
class example(Structure):
    pass
example._fields_ = [
    ('argsz', c_int),
    ('flags', c_int),
    ('count', c_int),
    ('details', example_detail * 0),
]

uint8_t buf[2048];

produces

...
  File ".../ctypeslib/ctypeslib/codegen/cursorhandler.py", line 400, in _get_var_decl_init_value_single
    _v = _v[0]
IndexError: list index out of range

just want to know whether this works for python 2.7 or not? if not is there is any other ctypeslib
for python 2.7

This generates an error

def test_struct_with_pointer(self):
    ns = self.convert('''
    struct x {
        int y;
    };
    typedef struct x *x_n_t;

    typedef struct p {
        x_n_t g[1];
    } *p_t;
    ''')

File "/home/other/Compil/ctypeslib/ctypeslib/codegen/cursorhandler.py", line 517, in STRUCT_DECL
return self._record_decl(cursor, typedesc.Structure, num)
File "/home/other/Compil/ctypeslib/ctypeslib/codegen/cursorhandler.py", line 594, in _record_decl
members.append(self.FIELD_DECL(field))
File "/home/other/Compil/ctypeslib/ctypeslib/codegen/cursorhandler.py", line 868, in FIELD_DECL
raise ValueError("Field has no displayname")
ValueError: Field has no displayname

need to patch clang so that char[] type is exposed.

Can llvm-pdbutil be used to do a PDB-> yaml -> python ctypes ?
https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-pdbutil/YAMLOutputStyle.cpp