Coder Social home page Coder Social logo

p-p-h-d / mlib Goto Github PK

View Code? Open in Web Editor NEW
791.0 791.0 65.0 8.47 MB

Library of generic and type safe containers in pure C language (C99 or C11) for a wide collection of container (comparable to the C++ STL).

License: BSD 2-Clause "Simplified" License

Makefile 0.84% C 71.94% C++ 27.09% Batchfile 0.13%
algorithms array bitset c collections data-structures generic hashmap json list lock-free memory-pool priority-queue queue set stack string tree tuples variant

mlib's People

Contributors

derskythe avatar jonadem avatar kitterion avatar kkofler avatar p-p-h-d avatar taboege avatar timgates42 avatar vladipus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mlib's Issues

IN_STR method missing on array

As specified by documentation and confirmed by a quick look at the code in m-array.h, the container should provide the IN_STR method if the elements support it. This doesn't work in my tests.

ARRAY_DEF(my_list, string_t)
#define MY_LIST_OPLIST ARRAY_OPLIST(my_list, STRING_OPLIST)
#define M_OPL_my_list_t() MY_LIST_OPLIST

int main() {
    M_LET(list, list2, my_list_t)
    M_LET(string, string_t) {
        string_set_str(string, "hello");
        my_list_push_back(list, string);
        my_list_push_back(list, string);
        FILE *file = fopen("/tmp/test.data", "w+");
        assert(file);
        my_list_out_str(stdout, list);
        my_list_out_str(file, list);
        rewind(file);
        my_list_in_str(file, list2); /* <= not available :-( */
        my_list_out_str(stdout, list2);
        fclose(file);
    }
}

The same happens if I define an array of C native types like int. I didn't check with other types of container.

Sorry to bother you with these reports. Heavily using it into an extensive project such defects come up.

Constructor / Destructor Prologue / Epilogue for Stack Exception Handling

Constructor (and destructor) need to use user-defined prologue / epilogue.
This is in order to register the constructed object into a proper Stack Exception Handling so that exceptions throwing may work reliably.

Proposal:
M_CONSTRUCTOR_PROLOGUE(object, oplist);
M_CONSTRUCTOR_EPILOGUE(object, oplist);
M_DESTRUCTOR_PROLOGUE(object, oplist);
M_DESTRUCTOR_EPILOGUE(object, oplist);

Object creation will need to add all sub-objects into the stack, then unstack all to push instead the root object (which will recursively remove them).

See also http://freetype.sourceforge.net/david/reliable-c.html#cseh

Add Initialization value support for M_LET

Add Initialization value support for M_LET. Example:

M_LET((a,b),c, string_t) {
  // a has been init & set to b
  // c has been init
}

M_LET((a,1,2,3),list_t) {
  // a has been init & contains the items 1, 2 & 3

}

Plan:
Add new method INIT_WITH(variable, var_args)

  • In C99 mode , if an argument is given a value, if INIT_WITH exists in the oplist, uses it for the initialization, otherwise uses INIT_SET
  • In C11 mode , if an argument is given a value, if INIT_WITH exists in the oplist, tests if the type of the single argument is the same as the variable to create, and uses INIT_SET in this case, otherwise uses INIT_WITH for the initialization, otherwise uses INIT_SET (needs careful _Generic usage).

buffer inside another container?

I've defined a buffer (as queue) of elementary types but it looks I can't use it inside another container (a tuple). Maybe this is related to the fact that I was not able to find a macro like BUFFER_OPLIST to define the respective oplist.

I want to do something like:

BUFFER_DEF(int_queue, int, 10,
           BUFFER_QUEUE | BUFFER_BLOCKING | BUFFER_THREAD_SAFE,
           M_DEFAULT_OPLIST)
TUPLE_DEF2(my_tuple, (a_queue, int_queue_t), (a_number, int))

STRING_CTE incompatible with C+++

Find a way to make STRING_CTE compatible with C++.
The difficulty is generating a constant with its char[] array field aligned to an integer.

Memory allocation enhancement

This ticket is about enhancement of the memory allocation scheme to find way to deal properly with advance allocators:

  • non-default alignment requirements for types,
  • instance-based allocator (may need instance based variable access),
  • expected life of created type (temporary or permanent),
  • stack based allocator,
  • global variable access for allocator,
  • maximum allocation before failure.

I think most of theses are already more or less supported. Examples shall be created to show how to deal with this.

binary serialization

I was thinking if some kind of "binary serialization" on the model of get_str/parse_str could be possible. It would be a great feature from the application point-of-view: binary representation is more bandwidth-efficient if used on network communications.
Given that I'm not sure if this is out of the scopes of the library or if you are interested in this.

It could be a m*lib specific representation or you could even think to some standards like https://msgpack.org/ or https://developers.google.com/protocol-buffers/.

Consider this just a feature request and feel free to close it if it doesn't make sense for you.

Supplying references to equal and compare

I'm putting structs into a list like so:

typedef struct __MyData {
    int data;
} MyData;

inline bool MyData__equals(const MyData z1, const MyData z2) {
    return z1.data == z2.data;
}

LIST_DEF(MyList, MyData, M_OPEXTEND(M_POD_OPLIST, EQUAL(MyData__equals)))

When comparing two list elements, the MyData__equals function gets supplied two elements copied by value. Is it somehow possible to supply them as pointers, so they won't be copied through the stack? I have tried to use M_IPTR which gets me this:

inline bool MyData__equals(const MyData *z1, const MyData z2) {
    return z1->data == z2.data;
}

LIST_DEF(MyList, MyData, M_OPEXTEND(M_POD_OPLIST, EQUAL(MyData__equals M_IPTR)))

So I'm half-way there. Is there a way to also get the second value passed as pointer?

use of _ptr type inside a container

I want to insert into a container a pointer to another container but keeping typical syntax of A[1] structures.
Here what I would get but using explicit structures/typedef:

#include <m-tuple.h>
#include <stdio.h>

struct my_point_s {
    int x, y;
};
typedef struct my_point_s *my_point_ptr;
typedef struct my_point_s my_point_t[1];

struct my_points_s {
    my_point_ptr first_ref, second_ref;
    my_point_t third;
};
typedef struct my_points_s *my_points_ptr;
typedef struct my_points_s my_points_t[1];

int main() {
    my_point_t p1, p2;
    p1->x = p1->y = 4;
    p2->x = p2->y = 5;

    my_points_t pair;
    pair->first_ref = p1;
    pair->second_ref = p2;
    pair->third->x = pair->third->y = 6;

    assert(pair->first_ref->x == 4);
    p1->x = 10;
    assert(pair->first_ref->x == 10);

    assert(pair->third->x == 6);
}

Now I would do the same but with M*lib containers.
First attempt:

TUPLE_DEF2(mlib_point, (x, int), (y, int));
#define M_OPL_mlib_point \
    TUPLE_OPLIST(mlib_point, M_DEFAULT_OPLIST, M_DEFAULT_OPLIST)
TUPLE_DEF2(mlib_points, (first_ref, mlib_point_t *),
           (second_ref, mlib_point_t *), (third, mlib_point_t));

Second attempt:

TUPLE_DEF2(mlib_point, (x, int), (y, int));
#define M_OPL_mlib_point \
    TUPLE_OPLIST(mlib_point, M_DEFAULT_OPLIST, M_DEFAULT_OPLIST)
TUPLE_DEF2(mlib_points, (first_ref, mlib_point_ptr),
           (second_ref, mlib_point_ptr), (third, mlib_point_t));

Any hint? Thanks.

C++ compatibility with m-i-list.h

Compatibility with C++ of m-i-list.h is broken.

Due to how the interface of m-i-list.h was planned, C++ doesn't put the structure defined in INTERFACE in the global namespace but in a sub-namespace (contrary in C). This results in broken build when the structure struct M_C3(ilist_head_, name, _s) is used outside the sub namespace:

error: invalid use of incomplete type ‘struct ilist_head_tname_s’

Search character/str in string from given position.

It is not possible to scan a string for a given char, a given string from a given position: it always search from the start.
Such behavior seems desirable to handle parsing a string.
It is possible to extract the string and parse it again, but there is a huge performance impact.
The solution seems to add an offset argument to search functions.

Propagation of NEW & DEL & REALLOC operators to upper oplist

Currently if I defined a container C over the oplist O which contains the NEW, DEL or REALLOC operator, the oplist of the container C doesn't contain the NEW, DEL or REALLOC operators.
It will be better to automatically added theses operators in the container oplist to allow chained definition with overloaded NEW, DEL & REALLOC operators.
This has to be done for:

  • list
  • array
  • dict
  • RB Tree
  • shared
  • i-list
  • i-shared

For tuple / variant, what to do if only one element has overloaded theses operators? All elements?

tuple of tuple using global oplist

I'm trying to switch my code to use the possibility to use globally defined oplists without specification during definition of containers and related oplist declaration. I guessed this new feature from the commit names: is it missing in README.md?

During such switch I think I've found some kind of bug. I'm able to define an array of tuples like this:

TUPLE_DEF2(my_tuple, (name, string_t), (value, int))
#define M_OPL_my_tuple_t() TUPLE_OPLIST(my_tuple)

ARRAY_DEF(my_list_of_tuple, my_tuple_t)
#define M_OPL_my_list_of_tuple_t() ARRAY_OPLIST(my_list_of_tuple)

but at this point I can't define a tuple containing another tuple:

TUPLE_DEF2(my_tuple_of_tuple, (name, string_t), (inner_tuple, my_tuple_t))
#define M_OPL_my_tuple_of_tuple_t() TUPLE_OPLIST(my_tuple_of_tuple)

I get an error like: error: use of undeclared identifier 'm_no_default_function'

I've not tested other combinations of containers.

How embed an __uint128_t?

I would use as element in a container an "almost" native type __uint128_t (from inttypes.h). For lack of knowledge on the OPLIST topic I was not able to define a proper one.

Here the example:

#include <inttypes.h>
#include <m-tuple.h>

#define uint128_t __uint128_t
// first attempt:
// #define M_OPL_uint128_t() M_DEFAULT_OPLIST

// second attempt:
#define M_OPL_uint128_t()                                                      \
    (INIT(M_INIT_DEFAULT), INIT_SET(M_SET_DEFAULT), SET(M_SET_DEFAULT),        \
     CLEAR(M_NOTHING_DEFAULT))

TUPLE_DEF2(my_tuple, (x, int), (y, uint128_t))
#define M_OPL_my_tuple_t()                                                     \
    TUPLE_OPLIST(my_tuple, M_DEFAULT_OPLIST, M_OPL_uint128_t())

int main() {
    M_LET(t, my_tuple_t) {}
}

Recursive dependent definition of containers.

Currently, it is not possible with the interface to define easily a type such as
type = variant of (integer|string|array of 'type|dict of (string, 'type))
without hacking around using pointers.

Being able to create such types could be great (it can model a JSON object for example).

Generalisation of snapshots

snapshot are limited to one thread producer and one thread consumer. It will be handy to remove this limitation. However it is not easy as one major feature of a snapshot is to avoid any copy and avoid any waiting for data availability (even if theoretically there is a wait loop for a CAS...)

To do so, instead of a triple buffer, a 1+C+P buffer is needed with C the number of thread consumers and P the number of thread producers. The API should allow to keep track of the used buffers by the different threads, and the different threads should be able to lock/unlock a particular buffer (This a quite similar to a shared pointer mechanism to keep track of the used buffers and the free ones).

standard bool among native types?

Assuming to use the C11 standard, the bool type should be among the native types supported by the core of m*lib (using generics).

atomic shared pointer

Extension to the SHARED_PTR API :

  • New type atomic_shared_ptr
  • name_init_atomic_set (&atomic_shared_ptr, shared_ptr);
  • name_init_set_atomic (shared_ptr, &atomic_shared_ptr);
  • name_init_atomic_set_atomic (&atomic_shared_ptr, &atomic_shared_ptr);
  • name_atomic_set (&atomic_shared_ptr, shared_ptr);
  • name_set_atomic (shared_ptr, &atomic_shared_ptr);
  • name_atomic_set_atomic (&atomic_shared_ptr, &atomic_shared_ptr);
  • name_atomic_clear

No _ref or direct _init: we need to init first a normal shared_ptr then the atomic (TBC)

_atomic_set

It can be implemented by incrementing the non atomic shared pointer reference, then performs a compare_and_swap to the data of the atomic shared pointer, finally decrement and dec/free the swapped previous data of the atomic .

_set_atomic

It needs to perform the following atomic operation : <read the pointer, deref pointer and increment the pointed value> I don't known how to do it properly.

See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4162.pdf

Dict with OA using string_t as key

I'm using a dictionary to map some strings (from 6 to a dozen of chars) to some records; in my scenario such set of key strings doesn't grow during the execution: it is known at startup but I need to make a lot of look-up for fetch or update.
I was trying to optimize the LUT time. Switching on hashing with "stored hash" (I'm not sure what it exactly does... the description didn't help) helped a bit: about 5-10% of time saved.
Can I setup the number of buckets in advance to an optimal value? I can't spot a function like _reserve.

I was wondering if it could help to switch on the OA variant. On my attempt I tried something like:

static inline bool string_oor_equal_p(const string_t s, unsigned char n) {
    return (s->ptr == NULL && s->size == (size_t)(n + 1) &&
            s->alloc == (size_t)(n + 2));
}
static inline void string_oor_set(string_t s, unsigned char n) {
    s->ptr = NULL;
    s->size = n + 1;
    s->alloc = n + 2;
}
#define STRING_AS_OA_KEY_OPLIST                                                \
    M_OPEXTEND(STRING_OPLIST, OOR_EQUAL(string_oor_equal_p),                   \
               OOR_SET(string_oor_set))
DICT_OA_DEF2(my_dict, string_t, STRING_AS_OA_KEY_OPLIST, record_t, RECORD_OPLIST)
#define MY_DICT_OPLIST                                                  \
    DICT_OPLIST(my_dict, STRING_AS_OA_KEY_OPLIST, RECORD_OPLIST)

In this case I get an error like error: unknown type name ‘my_dict_key_type_t’. I need an oplist because I embed such dictionary in other containers.

out_str support for C native types

Hi,
I'm using your M*LIB with great satisfaction! I like its style but also the "array[1] trick" that I love and already use in GMP/MPIR, PBC and in all my own code. I've started to use it in one new big project and I'm wondering about some details.

I'm trying to get an out_str support for any new defined type: mainly for debugging purpose. It looks that string_t got the out_str method in the STRING_OPLIST. I was, wrongly, assuming that all the native numeric type (int, float, ...) in C got such support but I can't get any composition including such types printable in automatic way: I have to define a specific out_str method. Is it normal?

Other marginal questions/curiosities:

  • can I ask the meaning of the _DEF macros with the suffix 2 (like TUPLE_DEF2)? I saw in the commitments log a renaming of a macro from _DEF2 to _DEF: is it something like "it is a draft"?!
  • I there something like "API breakage vs versioning" in your mind? Can I develop a software saying that it requires m-lib with version greater-equal to something and expect that the code will not break?
  • If your last answer is like "something could change on new releases", I wondering if you could include in the main header m-core.h a define with the numeric version; in this way I can make a check in meson to verify such version to a specific one.

PS: maybe this should go into an email but I was not able to find you email address.

Thanks.

OPLIST for m-mutex.h elements

I understand that I'm trying to do OOP using C and m*lib containers as base (data + common methods). Maybe I'm going too far...

More or less with the same motivations on #34 , I need to embed an m_mutex_t and an m_cond_t into a tuple. I'm saying it is related because the buffer is indeed a data-structure plus some mutex/condition variables to provide its blocking/atomic features. Now I need a blocking/atomic behavior for a dictionary so I wanted to embed all the necessary (dictionary, mutex, condition variable) into a tuple. I'm stuck because the latter are intended to be used as global variables (like buffer until yesterday).

Tell me if I'm going to be cumbersome and/or if what I prospect has flaws.

mempool usage

This is a question about the way to use mempool.
I can spot in the main documentation that it is possible to automatically use mempool structures associated to some type of linked list and that they are declared as thread-unsafe.
I can also see an header m-c-mempool.h and it looks related to the future lock free lists.

Now my question: I use on different threads many temporary containers (of some few types) that I usually create/destruct with macro M_LET. I'm wondering if it could be possibile to specify a mempool (specific per container type) to automatically use on such contexts. If they are thread-safe, they could be shared among threads or one-per-thread if unsafe.

User defined & global OPLIST

It may be good to allow the user to define global OPLIST to use like this:

    #define M_OPLIST_mpz_t (INIT(mpz_init),CLEAR(mpz_clear), NEW(mpz_new), DEL(mpz_free) )
    #define M_OPLIST_mpq_t (NEW(mpq_new), DEL(mpq_free), REALLOC(none))

and modify M*lib to handle theses global definitions in some place like:

  • macro M_LET which could accept a type instead of an oplist as the last argument (with a little bit of macro programming, we should be able to differentiate between an oplist or a global type),
  • macro M_EACH (same reason but less usefull),
  • new layer for the memory allocation: M_MEMORY_ALLOC should be able to detect that we want to allocate a type T, and that the type T have globally overloaded its NEW operator.
  • containers with no given oplist should check if such global oplists exist.

First to evaluate the PRO and the CON.

Supported types will be limited more or less to 'typedef' ones only.

missing _ptr definitions

I can spot in tuples the definition of name_ptr pointer type. It looks that such support is missing in all the other containers.
It would be useful for me to define a pointer to a container (without allocation) keeping the comfortable syntax of the "array[1] trick".

_get_str conflict for tuple with str field.

A tuple definition will generate the following methods:

  • _get_str: to transform the tuple into a string_t
  • _get_##field: to get the value of the field. This is done for each field.

If one field of the tuple is 'str', there is a conflict of method...

The issue is in the API definition and only an API breakage may solve it. I see multiple solution:

  • rename all _get_str methods into _convert_str for all types. I don't like this solution too much impact.
  • rename the _get_##field into something else. in fact, current function return a pointer to constant type; which means the _get_ function should have been called _cget_.

So I propose to change the functions _get_##field to _cget_##field

for M_EACH cycles and clang-format: a strange behavior

I'm used to heavily use automatic formatting of my code (on save). At the moment I'm using clang-format inside Atom. I had no problem with M*LIB expect for the for M_EACH(...) block. clang-format doesn't correct the indentation level of such block (it is untouched): I have to move it on the right column. This is funny.

Example: clang-format is not able to correct the excessive indentation of the for M_EACH(...) block.

    M_LET(list, insertion, my_list_t) {
        //
                                        for
                                            M_EACH(item, list, my_list_t) {
                                                //
                                                //
                                            }
    }

non-heap allocation for small string_t

Currently, string_t always allocates the char array needed for a string on the heap (or equivalent). If the string is small enough, it is a waste of memory and induces an indirection (cache miss). The structure string_t itself can be used to store the string if the string is not too big (kind of stack allocation). The trick is to implement it without duplicating all the code for both allocations and without too many new branches (penalizing the execution time).

A tentative implementation of string_t with such characteristic has been implemented and is available here: https://gist.github.com/P-p-H-d/002560b5b7e558d60c18d7169b6d5f36
All tests pass with this new implementation.

Some bench have been done:

  • bench-string: this bench is really no good and lets too much the compiler to overwrite. GCC 8 does the best job:
OPERATOR NEW operations per second OLD operations per second
empty constructor 648252177.7 555829826.8
non-empty constructor 48662108.8 49258338.8
small non-empty constructor 648785372.6 49283624.4
Char * assignment 462453892.4 428323271.5
char extraction 486312551.9 555987881.8
scan 86451378.4 90490016.7
concatenation 274178844.1 216660394.5
replace 64858959.8 65919877.7

size of the executable is the same (!) (due to page cache alignment of the executable ?).
The result doesn't make much sense (for example concatenation) but it is due to the fact that the bench has some representative issues.

With GCC 4.9, the size of the executable increase by 3KB (probably a worst case).

  • bench-mlib / Dictionary over string.
    GCC 7:
    time of the bench goes from 780 ms (Heap Alloc) to 656 ms (Stack for small string). [-16%]
    size of the executable is nearly the same (from 42848 to 42952) [1%]

  • parse of a 181 MB JSON file
    OLD: 7.8s (42200 bytes)
    NEW: 7.5s (42296 bytes)
    I was expecting more performance gain, but it is probably because it never reads back the data in memory. The memory consumption of the data is greatly reduced however.

I need help to test the new implementation and compare it to the previous one to check if it can be integrated in M*LIB:

  • its speed
  • the size of the executable.

automatic _clean method for tuples

Even if all the elements of a tuple container got the _clean method, the resulting container lacks such method. I'm wondering if it make sense to automatically define it for tuples like for the other kinds of list in m*lib.

regression

Today I updated m-lib on latest commit (2e3b312). I'm not able to compile my project now: it looks related on recent changes on the M_LET macro. Doing some bisect it looks introduced on commit ce948bd where clang started to report the following error:

error: array type 'my_tuple_t' (aka 'struct my_tuple_s [1]') is not assignable
                    M_LET(tuple, my_tuple_t) {
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../custom-libs/include/m-core.h:2217:8: note: expanded from macro 'M_LET'
  M_ID(M_LETI1 M_INVERT(a, __VA_ARGS__,                                 \
  ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../custom-libs/include/m-core.h:1082:35: note: expanded from macro 'M_ID'
#define M_ID(...)                 __VA_ARGS__
                                  ^~~~~~~~~~~
../custom-libs/include/m-core.h:2226:3: note: expanded from macro 'M_LETI1'
  M_LETI2(cont, M_GLOBAL_OPLIST(oplist), __VA_ARGS__)
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The error is unchanged even with current HEAD on git. gcc reports a different message but it should be related to the same problem.

I hope you can catch the error with the information above; otherwise I can try to reproduce the error on a minimal example.

a bug in m-array _insert_v() method?

I'm facing something that looks a bug. I need to make space inside an array-based list , so I'm using _insert_v method to create some new initialized entries. As I understand from the documentation and from a quick view on the code: it should manage the case when the reserved/allocated space in the list is insufficient to support the insertion.

Something like this works:

my_list_resize(list, 201);
my_list_reserve(list, 512);
my_list_insert_v(list, 200, 300);

Instead this doesn't:

my_list_resize(list, 201);
my_list_reserve(list, 512);
my_list_insert_v(list, 200, 600);

Is it a bug?

In my original code I'm facing assertion violations on the contract clause: v->size <= v->alloc.

M_LOCK within a structure or malloc

The interface of M_LOCK prevents it from being used as a member of a structure or being allocated by malloc: it shall always be a global variable.
This limitation may not be pertinent.

[feature request] multiple heterogeneous M_LET with a single macro/block

I'm starting to heavily using M_LET and M_EACH macros. The first one is very useful to keep the code minimal and to forget about init/clean dues.
Using it in block that declare many M*LIB container instances it happens that the block got too many nesting levels: sometimes this badly squeeze the code. In my case is forced by the automatic use of beautification tools like clang-format inside the editor.

I appreciate the possibility to declare with M_LET multiple instances with the same type. I'm wondering if it could be possible to have a macro (maybe M_LET itself) that is able to declare multiple instances of heterogeneous types. This would permit to concentrate in a single block all the necessary containers.

Something like: M_LET ((x1, x2, type1_t), (y1, type2_t), (z1, z2, z3, type3_t)) { /* code */ }

warning with latest changes in master

This is a minimal example raising the warning not present with previous version:

TUPLE_DEF2(symbol, (type, int), (name, string_t))
#define SYMBOL_OPLIST TUPLE_OPLIST(symbol, M_DEFAULT_OPLIST, STRING_OPLIST)
#define M_OPL_symbol_t() SYMBOL_OPLIST

ARRAY_DEF(symbol_list, symbol_t)
#define SYMBOL_LIST_OPLIST ARRAY_OPLIST(symbol_list, SYMBOL_OPLIST)
#define M_OPL_symbol_list_t() SYMBOL_LIST_OPLIST

TUPLE_DEF2(record, (fields, symbol_list_t))
#define RECORD_OPLIST TUPLE_OPLIST(record, SYMBOL_LIST_OPLIST)
#define M_OPL_record_t() RECORD_OPLIST

this is the error:

warning: passing 'const symbol_list_t' (aka 'struct symbol_list_s const[1]') to parameter of type 'struct symbol_list_s *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
TUPLE_DEF2(record, (fields, symbol_list_t))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

get_str/parse_str methods for C native types

In a similar way as in report #16, I've found that the core base doesn't provide get_str/parse_str methods for native types. This is a pity as the containers provide them if the elements do.
I'm trying to use such methods to get a quick non-binary serialization of the containers. This would allow to send them as network messages.

Empty containers & IN_STR operator

Currently most containers don't support properly the IN_STR operator if there is nothing to read:

  1. the sub-operator IN_STR will fail to read any item, report its failure to the caller, but it doesn't restore the stream to its initial state.
  2. Then the caller fails to realize that the character in the stream is ']' as it has already been read and purge from the stream by the called operator.

attempt to redefine parse_str for an enum

I was trying to define a type from an enumeration but with a more informative output on the out_str/in_str/get_str/parse_str methods. Instead of a plain number something like number-SOME_TEXT.
I faced some problems on the OPLIST definition, indeed the methods works but I can't use such type in other containers. I can't spot the problem.

The toy example follows:

#include <m-string.h>
#include <m-tuple.h>
#include <stdio.h>

typedef enum { RED = 0, WHITE, BLUE, BLACK } color_t;
const char *color_str[] = {
    [RED] = "red", [WHITE] = "white", [BLUE] = "blue", [BLACK] = "black"};

#define color_out_str(f, x) (fprintf(f, "%u-%s", x, color_str[x]))
#define color_in_str(x, f) (fscanf(f, "%u-%*[a-z]", &x))
#define color_get_str(s, x, a)                                                 \
    ((a ? string_cat_printf : string_printf)(s, "%u-%s", x, color_str[x]))
bool color_parse_str(color_t *ptr, const char str[], const char **endptr) {
    char *end;
    *ptr = strtoul(str, &end, 10);
    if (end != str) {
        if (*end != '-')
            return false;
        end++;
        const char *ptr2 = color_str[*ptr];
        while (*ptr2 != '\0' && *ptr2 == *end) {
            ptr2++;
            end++;
        }
        if (endptr != NULL)
            *endptr = end;
        return (*ptr2 == '\0');
    } else
        return false;
}
#define COLOR_OPLIST                                                           \
    M_OPEXTEND(M_DEFAULT_OPLIST, OUT_STR(color_out_str), IN_STR(color_in_str), \
               GET_STR(color_get_str), PARSE_STR(color_parse_str))
#define M_OPL_color_t() COLOR_OPLIST

TUPLE_DEF2(my_tuple, (name, string_t), (color, color_t))
#define MY_TUPLE_OPLIST TUPLE_OPLIST(my_tuple, STRING_OPLIST, COLOR_OPLIST)
#define M_OPL_my_tuple_t() MY_TUPLE_OPLIST

int main() {
    color_t color = WHITE, color2;
    M_LET(string, string_t) {
        color_out_str(stdout, color);
        fputc('\n', stdout);
        color_get_str(string, color, false);
        assert(color_parse_str(&color2, string_get_cstr(string), NULL));
        printf("%u -> %s -> %u\n", color, string_get_cstr(string), color2);
        string_set_str(string, "2-wrong");
        assert(!color_parse_str(&color2, string_get_cstr(string), NULL));
    }
}

I'm getting a warning like incompatible integer to pointer conversion passing 'color_t' to parameter of type 'color_t *'; take the address with & [-Wint-conversion] on the tuple definition.

dictionary of pointers (LUT) vs compilers behavior

I'm trying to create a LUT (look up table) that maps a string to a generic pointer. I can compile and use the following minimal example with gcc (8.1.1) without errors/warnings:

#include <m-dict.h>
#include <m-string.h>
#include <stdio.h>

DICT_DEF2(lut, string_t, STRING_OPLIST, const void *, M_DEFAULT_OPLIST)

int main() {
    lut_t lut;
    string_t label;
    int a = 10;

    lut_init(lut);
    string_init(label);

    string_set_str(label, "a key");
    lut_set_at(lut, label, NULL);

    string_set_str(label, "another key");
    lut_set_at(lut, label, &a);

    string_clear(label);
    lut_clear(lut);
}

but here I was forced to use const void* instead of, a more useful for me, void * because of some errors with gcc like:

../test/test-mlib.c: In function ‘lut_cget’:
../custom-libs/include/m-core.h:1334:49: warning: returning ‘const void **’ from a function with incompatible return type ‘void * const*’ [-Wincompatible-pointer-types]
   (((union { type *ptr; const type *cptr; }){n}).cptr)
...

using void * with clang I get a different (maybe equivalent) error:

../test/test-mlib.c:5:1: warning: returning 'const void **' from a function with result type 'void *const *' discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
DICT_DEF2(lut, string_t, STRING_OPLIST, void *, M_DEFAULT_OPLIST)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../custom-libs/include/m-dict.h:39:3: note: expanded from macro 'DICT_DEF2'
  DICTI_DEF2_P1(M_IF_NARGS_EQ1(__VA_ARGS__)                             \
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../custom-libs/include/m-dict.h:111:28: note: expanded from macro 'DICTI_DEF2_P1'
#define DICTI_DEF2_P1(arg) DICTI_DEF2_P2 arg
                           ^~~~~~~~~~~~~~~~~

finally, if I rollback to const void* with clang it works but I can't get rid of this annoying warning:

../test/test-mlib.c:5:41: warning: duplicate 'const' declaration specifier [-Wduplicate-decl-specifier]
DICT_DEF2(lut, string_t, STRING_OPLIST, const void *, M_DEFAULT_OPLIST)

A side note: I was trying to create containers with pointers to other types (like mpz) but the default OPLIST with generics (gnu11) can't clearly manage the out_str method. I'm not interested in printing the linked structure (the raw pointer would be good) but I don't want to define a new out_str method every time. On the other side, using void * I have to cast to the correct type the pointer every time I extract something from the LUT. I hope it is clear what I mean.

Thanks.

missing const specification in dictname_equal_p

The documentation reports a prototype for methods like dictname_equal_p where the arguments are (correctly) passed as `const'. The implementation seams to lack such specification.

Such missing could be in other containers/methods.

issues in m-mutex.h

I'm starting to use some containers in m-buffer.h so m-mutex.h is involved too. On first use I faced some warnings (using gcc but also clang):

In file included from m-buffer.h:29,
m-mutex.h: In function ‘m_thread_join’:
m-mutex.h:102:22: warning: passing argument 1 of ‘thrd_join’ makes integer from pointer without a cast [-Wint-conversion]
   int rc = thrd_join(t, NULL);

In file included from m-mutex.h:33,
                 from m-buffer.h:29,
/usr/include/threads.h:107:30: note: expected ‘thrd_t’ {aka ‘long unsigned int’} but argument is of type ‘thrd_t *’ {aka ‘long unsigned int *’}
 extern int thrd_join (thrd_t __thr, int *__res);
                       ~~~~~~~^~~~~

I found also a, I guess, copy&past error that doesn't permit to compile anything using it: I'm wondering how it didn't raise from your test units (the file looks 3 months old). There is a double definition of the m_cond_clear function.

M_KEEP redefined

Seems like one of your internal macros are actually defined in stdlib =)

image

Great work on the project, BTW!

Makefile doesn't install m-concurrent.h

It looks that the actual Makefile doesn't install this recent header (others?). This hits the latest 0.2.0 release to it is buggy: maybe it is better to release a new minor release to fix this (IMHO).

Minor issues using m-array.h and m-concurrent.h

Trying to write an ad-hoc benchmark for #39 I spotted few minor issues trying to get a thread-safe array container:

  • there is a const-related warning, example:
ARRAY_DEF(key_list, string_t)
#define M_OPL_key_list_t() ARRAY_OPLIST(key_list, STRING_OPLIST)
CONCURRENT_DEF(key_list_ts, key_list_t)
  • the method _size is not exposed in the concurrent version: I can't get the number of elements in such kind of array.

Possibility to use _ptr or _srcptr to manage element references from containers

Methods like _front/_back/_get offered by many containers allow to get a reference to an element without extraction. It reports a pointer to the type: for the "array[1]-style" that would be not strictly necessary but I understand that it has to work also with native types.

Now the point: I would use the _ptr/_srcptr pointer types to host such references so I can continue to use the syntax pointer->field.

Here an example:

#include <m-array.h>
#include <m-tuple.h>

TUPLE_DEF2(my_tuple, (x, int), (y, char))
#define M_OPL_my_tuple_t()                                                     \
    TUPLE_OPLIST(my_tuple, M_DEFAULT_OPLIST, M_DEFAULT_OPLIST)

ARRAY_DEF(my_array, my_tuple_t)
#define M_OPL_my_array_t() ARRAY_OPLIST(my_array, M_OPL_my_tuple_t())

int main() {
    M_LET(t, my_tuple_t)
    M_LET(array, my_array_t) {
        my_tuple_ptr ptr;
        ptr = t;

        t->x = 100;
        t->y = 'a';

        assert(ptr->x == 100 && ptr->y == 'a');

        my_array_push_back(array, t);

        ptr = my_array_get(array, 0); // warning!
        assert(ptr->x == 100 && ptr->y == 'a');

        ptr = my_array_back(array); // warning!
        assert(ptr->x == 100 && ptr->y == 'a');

        my_tuple_srcptr ptr2;
        ptr2 = my_array_back(array); // warning!
        assert(ptr2->x == 100 && ptr2->y == 'a');

        my_tuple_t *ugly_ptr;
        ugly_ptr = my_array_get(array, 0);
        assert((*ugly_ptr)->x == 100 && (*ugly_ptr)->y == 'a');

        const my_tuple_t *ugly_ptr2;
        ugly_ptr2 = my_array_cget(array, 0);
        assert((*ugly_ptr2)->x == 100 && (*ugly_ptr2)->y == 'a');
    }
}

The only way to host the reported pointer (without warnings) is to use my_tuple_t *ugly_ptr or its const version. Is it possibile?

Can you provide a "complex" struct example?

Say I have a struct with a few uint32_t, a string_t and some json from https://github.com/DaveGamble/cJSON:

typedef struct rel { uint32_t id; uint32_t incoming_node_id; uint32_t outgoing_node_id; string_t type; cJSON* properties; } relationship;

and you wanted a dynamic array of them:
ARRAY_DEF(array_relationship, relationship, (INIT(relationship_init),SET(relationship_set),INIT_SET(relationship_init_set),CLEAR(relationship_clear)))

What would the relationship_init, relationship_set, relationship_init_set and relationship_clear look like to make it work?

a proper way to break from a M_LET block

In order to respect the following understandable claim about M_LET usage:

NOTE: The user code can not perform a return or a goto outside the {} otherwise the clear code of the object won't be called .

it is often necessary to write pedantic and inelegant code inside. It would be great to have an official way to "break" from such code jumping at the end in a proper way. Something like a new M_BREAK macro that will not break in the future release (not a simple trick).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.