nemequ / portable-snippets Goto Github PK

View Code? Open in Web Editor NEW

658.0 24.0 63.0 265 KB

Collection of miscellaneous portable C snippets.

License: Other

CMake 3.91% C 95.12% C++ 0.97%

c public-domain intrinsic builtin atomic clock time monotonic portable overflow

portable-snippets's People

Contributors

Stargazers

Watchers

Forkers

pskocik templeblock gaoxiaojun sgeto rmccullagh sdwfrost liumorgan aneury1 jbenden karbyshev vincentlao steveshaw amallia a-ma72 opencai c-and-cpp-libraries miller2018 bryanlopezrr uchihasr cqu611 junftnt wangkang6s junxie6 frymax xiaoxiang781216 forksnd jwcs nakrakiiya vilocnv theburn inndy nexomichael pramberg externalrepositories nealrichardson light7734 cbdesignsbybre hugmyndakassi nimius-debug pchemguy sungjjinkang mtdcy spkorhonen dosworld ckhfor clayne q4a lindauer-humain gerhobbelt brycem1221 spoopyando sayama3 vbs100 digitallibertieslab 33uy crimson1st ror3d gpg-dev tzleon nvoronetskiy billyoneal

portable-snippets's Issues

Breakpoint module

__builtin_trap is _Noreturn, so it doesn't work as a starting point to attach a debugger and step through the code.

Other code:

scottt/debugbreak looks cool. It's BSD2, and the code is a little sloppy (see scottt/debugbreak#4), but there is a great table in the README which could be used to write something cleaner.
GLib has G_BREAKPOINT, which explicitly supports different platforms than debugbreak (alpha vs arm).

Wouldn't be too difficult to merge the knowledge from both projects into a new (CC0) module for psnip…

Wrong detection of intsafe.h at MinGW (with GCC 4.8.1)

It was reported in modelica/ModelicaStandardLibrary#3556 that the following lines

portable-snippets/safe-math/safe-math.h

Lines 22 to 28 in f596f8b

    
           #  if defined(__has_include) 
        
           #    if __has_include(<intsafe.h>) 
        
           #      define PSNIP_SAFE_HAVE_INTSAFE_H 
        
           #    endif 
        
           #  elif defined(_WIN32) 
        
           #    define PSNIP_SAFE_HAVE_INTSAFE_H 
        
           #  endif

are not protable enough for MinGW with GCC 4.8.1 (not exclusively to that version). The issue probably is, that old GCC 4.8.1 has no __has_include why it then sets PSNIP_SAFE_HAVE_INTSAFE_H in the else branch since MinGW defines _WIN32. AFAIK, there is no intsafe.h on MinGW yet.

Add a function for securely erasing a buffer

After using any sensitive data such as passwords, the buffers which store them must be cleared by the code to prevent it from being misused by a malicious actor by performing core dumps or using other means to access the memory of the application. However, most compilers optimize away the memset call as they interpret the call as dead code because the memory being written to is not being used afterwards.

I have already added a psnip_memset_explicit function in https://github.com/arnavyc/portable-snippets/tree/arnavyc/memset-explicit/memset-explicit that can erase a buffer without fear of 'dead-store removal'

An error occurred when compiling portable-snippets with vcpkg

environment:

windows x64
msvc 14.36.32532

Steps to reproduce:

git clone https://github.com/microsoft/vcpkg.git
Enter the vcpkg directory and execute bootstrap-vcpkg.bat
Run the command ./vcpkg install portable-snippets:x64-windows
The following error occurs:

[1/5] C:\PROGRA~1\MICROS~3\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\HostX64\x64\cl.exe -IE:\Lily\jasper\buildtrees \portable-snippets\src\a6955d89c4-b026d6ec08 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /FoCMakeFiles\cpu.dir\cpu\cpu.c .obj /FdCMakeFiles\cpu.dir\cpu.pdb /FS -c E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\cpu\cpu.c
FAILED: CMakeFiles/cpu.dir/cpu/cpu.c.obj
C:\PROGRA~1\MICROS~3\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\HostX64\x64\cl.exe -IE:\Lily\jasper\buildtrees\portable-snippets\ src\a6955d89c4-b026d6ec08 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /FoCMakeFiles\cpu.dir\cpu\cpu.c.obj /FdCMakeFiles\ cpu.dir\cpu.pdb /FS -c E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\cpu\cpu.c
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(101): error C2054: expected '(' to follow '_Noreturn'
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(101): error C2085: 'thrd_exit': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(104): error C2085: 'thrd_join': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(105): error C2085: '_thrd_sleep32': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(107): error C2085: '_thrd_sleep64': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(116): error C2085: 'thrd_sleep': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(116): error C2143: syntax error: missing ';' before '{'
[2/5] C:\PROGRA~1\MICROS~3\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\HostX64\x64\cl.exe -IE:\Lily\jasper\buildtrees \portable-snippets\src\a6955d89c4-b026d6ec08 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /FoCMakeFiles\random.dir\random\random.c .obj /FdCMakeFiles\random.dir\random.pdb /FS -c E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c
FAILED: CMakeFiles/random.dir/random/random.c.obj
C:\PROGRA~1\MICROS~3\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\HostX64\x64\cl.exe -IE:\Lily\jasper\buildtrees\portable-snippets\ src\a6955d89c4-b026d6ec08 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /FoCMakeFiles\random.dir\random\random.c.obj /FdCMakeFiles\ random.dir\random.pdb /FS -c E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(101): error C2054: expected '(' to follow '_Noreturn'
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(101): error C2085: 'thrd_exit': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(104): error C2085: 'thrd_join': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(105): error C2085: '_thrd_sleep32': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(107): error C2085: '_thrd_sleep64': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(116): error C2085: 'thrd_sleep': not in formal parameter list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.36.32532\include\threads.h(116): error C2143: syntax error: missing ';' before '{'
ninja: build stopped: subcommand failed.

Then I added set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std:c11" ) in cmakelists.txt, and recompiled the following problems:

[1/5] C:\PROGRA~1\MICROS~3\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\HostX64\x64\cl.exe    -IE:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP  -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /FoCMakeFiles\cpu.dir\cpu\cpu.c.obj /FdCMakeFiles\cpu.dir\cpu.pdb /FS -c E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\cpu\cpu.c
[2/5] C:\PROGRA~1\MICROS~3\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\HostX64\x64\cl.exe    -IE:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP  -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /FoCMakeFiles\random.dir\random\random.c.obj /FdCMakeFiles\random.dir\random.pdb /FS -c E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c
FAILED: CMakeFiles/random.dir/random/random.c.obj 
C:\PROGRA~1\MICROS~3\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\HostX64\x64\cl.exe    -IE:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP  -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 -std:c11 /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /FoCMakeFiles\random.dir\random\random.c.obj /FdCMakeFiles\random.dir\random.pdb /FS -c E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.h(30): error C2057: expected constant expression
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.h(30): error C2466: cannot allocate an array of constant size 0
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c(34): error C2057: expected constant expression
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c(34): error C2466: cannot allocate an array of constant size 0
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c(139): error C2057: expected constant expression
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c(139): error C2466: cannot allocate an array of constant size 0
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c(300): error C2057: expected constant expression
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c(300): error C2466: cannot allocate an array of constant size 0
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c(372): error C2057: expected constant expression
E:\Lily\jasper\buildtrees\portable-snippets\src\a6955d89c4-b026d6ec08\random\random.c(372): error C2466: cannot allocate an array of constant size 0
[3/5] cmd.exe /C "cd . && C:\PROGRA~1\MICROS~3\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\HostX64\x64\lib.exe  /machine:x64 /nologo /out:psnip-cpu.lib CMakeFiles\cpu.dir\cpu\cpu.c.obj  && cd ."
ninja: build stopped: subcommand failed.

Rotation code is UB for zero shift counts

If I understand it correctly existing portable rotation code will emit a shift count of 32 if you rotate a 4-byte value by 0:

#define PSNIP_BUILTIN_ROTL_DEFINE_PORTABLE(f_n, T, ST)	\
  PSNIP_BUILTIN__FUNCTION				\
  T psnip_intrin_##f_n(T value, ST shift) {		\
    return						\
      (value >> ((sizeof(T) * 8) - shift)) |		\
      (value << shift);					\
}

Shifts of amounts equal or greater than the size of the underlying value are UB in C and C++, so a shift of 32 is not legal here.

Here is a good summary of portable, legal ways to implement this. They are generally just as fast (i.e., recognized) on current compilers, but using it on older compilers may be slow.

Dependencies between components

@nemequ let's say I want to use a particular component of portable-snippets, say "clock" - is there some way to determine or pull to in all the modules that clock needs? I have tried the brute force approach of adding stuff until it compiles, and that is a bit annoying - but more importantly it doesn't seem reliable because the exact things a module depends on may depend on the platform: compile it somewhere else and it may want atomic when the original platform didn't.

Just throwing this out there to see if you have any thoughts.

I face this issue since for a small projects of 3 or 4 files, protable-snippets is still very useful but I don't want to bring in 20 files to make just clock work. It's still better than writing it myself though :).

Intel/PGI compiler notes.

Per Intel Compilers for Linux: Compatibility with GNU Compilers (page 6), Intel compilers prior to OneAPI would attempt to guess the version of of gcc on the system based on what was found in the ${PATH} environment variable and silently degrade its feature set to be compatible what that version of gcc.

This is almost certainly not a problem unless you happen to work in a certain US Department of Energy nuclear lab where the default version of gcc is still 4.9.3. If that's the case and you have the latest version of icc in your path, __builtin_[add|sub|mul]_overflow will not be defined and anything using math-safe.h fail to compile.

The workaround is to use -gcc-name=${PATH_TO_MORE_RECENT_GCC} during compilation.

Based on a much more cursory search, PGI compilers not not appear to implement the necessary builtins.

Happy to write this up as a PR for the readme if there's interest.

random module needs convenience functions

µnit has some, like munit_rand_int_range. @jibsen, IIRC you did some work on those functions, would you mind if I relicensed them to public domain (CC0) and merged them into psnip?

It would probably also be a good idea to provide psnip_random_int, psnip_random_double… Ideally I'd like to just have a macro like

#define psnip_random_value(source, T) \
  ({ T psinp_random__tmp; \
     psnip_random_bytes(src, sizeof(T), &tmp); \
     psnip_random__tmp;  })

But that's a GCC extension; IIRC clang and icc support it, but I don't think there is a way to implement it with MSVC. I think the best we'll be able to do is something like

#define PSNIP_RANDOM_DEFINE_VALUE_FUNC(T, func_name) \
  __attribute__((__unused__)) \
  static inline T func_name (enum PSnipRandomSource source) { \
    T tmpval_; \
    psnip_random_bytes(source, sizeof(T), &tmpval_); \
    return tmpval_; \
  }

PSNIP_RANDOM_DEFINE_VALUE_FUNC(char, psnip_random_char)
PSNIP_RANDOM_DEFINE_VALUE_FUNC(short, psnip_random_short)
PSNIP_RANDOM_DEFINE_VALUE_FUNC(int, psnip_random_int)
...

With appropriate macros for the unused attribute and the inline keyword, of course.

High-precision time

There is some code in µnit for this but it's geared towards timing the difference between two events.

Migrate CI away from from Travis CI

I have been waiting for CI checks in #40, that resolves #39, to complete for about 15 days. However it always shows Expected — Waiting for status to be reported.

I would be nice to have a few more "how to use" lines in the readme

For example, it seems like .travis.yml uses cmake (for the tests), so this is a cmake project that pops out a library that I should link against (it seems not - cmake is just for tests?)? If I look at a module like cpu that has cpu.h and cpu.c do I just drop those into my project (probably not that simple since they may depend on other modules)?

Maybe something like a default-use case scenario, and then for any modules that don't follow that pattern a note in the readme.

Wrong intrinsics for GCC in atomic/atomic.h

When PSNIP_ATOMIC_IMPL is equal to PSNIP_ATOMIC_IMPL_GCC, the wrong intrinsics are used for the _int64_add & _int64_sub implementations.

In https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html , you can see these intrinsics exist:

__atomic_add_fetch
__atomic_sub_fetch

__atomic_fetch_add
__atomic_fetch_sub

To be consistent with the other implementations of psnip_atomic_int64_add and psnip_atomic_int64_sub, we want to fetch and then add or subtract. However, the intrinsics used in PSNIP_ATOMIC_IMPL_GCC do things in the opposite "order".

Memory mapped files

A decent mmap-like API which works on Windows would be quite helpful.

Runtime CPU feature detection (SSE, AVX, NEON, etc.).

I have some code for this. Need to clean it up a bit, remove any proprietary bits, and add support for some more architectures (currently it's only x86 and ARM).

Would be great if it also included an ifunc-like mechanism for calling different versions of a function.

Rotation code should use intrinsics on platforms other than MSVC

As far as I can tell, compiler rotate intrinsics are only used on MSVC.

As mentioned here, however, most platforms offer those intrinsics at least for x86 in <x86intrin.h>, so it seems like they could be used when available.

pragma message

Not sure in which module this would fit into, or if it's even interesting, but it would be nice to have portable ways of emitting messages at compile time, along the lines of what gcc offers with #pragma message since about the 4.4 version. I guess you'd have to emulate it with warnings before that?

Clarity on C++ support

It would be nice to have some clarity on C++ support, at worst here in this issue, but better a line in the README.

Based on my scan of a couple of header files is seems like the necessary #ifdef __cplusplus ... extern "C" { stuff is there, so I guess the intent is to support C++, but confirmation would be ideal.

math module

It could be useful to have a module which implements functionality missing from libm, similar to what we do for builtins/intrinsics in the builtin module.

https://github.com/GNOME/gtk/blob/master/gtk/fallback-c89.c would be a good starting point, but it's LGPLv2+. @fanc999, it looks like you're the sole author… would you be willing to release that code into the public domain (CC0) so I could steal it for this repo?

builtin.h does not compile with Arduino IDE g++

I am compiling psnip using the Arduino IDE for use on a Teensy, but it results in a compilation error when including psnip/builtin/builtin.h in a .cpp file. When I include it in a .c file it works fine, so that it suggests it has something to do specifically with C++.
In another issue I've read that you intend to support C++, but that it's just less well-tested.

In file included from C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\test.cpp:2:0:
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h: In function 'unsigned char psnip_intrin_BitScanForward64(long unsigned int*, uint64_t)':
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:636:63: error: '_' was not declared in this scope
 #  define psnip_builtin_ctz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,ctz)(x)
                                                               ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1253:61: note: in expansion of macro 'psnip_builtin_ctz64'
   return PSNIP_BUILTIN_UNLIKELY(Mask == 0) ? 0 : ((*Index = psnip_builtin_ctz64 (Mask)), 1);
                                                             ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:636:65: error: 'ctz' was not declared in this scope
 #  define psnip_builtin_ctz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,ctz)(x)
                                                                 ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1253:61: note: in expansion of macro 'psnip_builtin_ctz64'
   return PSNIP_BUILTIN_UNLIKELY(Mask == 0) ? 0 : ((*Index = psnip_builtin_ctz64 (Mask)), 1);
                                                             ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:636:68: error: 'PSNIP_BUILTIN__VARIANT_INT64' was not declared in this scope
 #  define psnip_builtin_ctz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,ctz)(x)
                                                                    ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1253:61: note: in expansion of macro 'psnip_builtin_ctz64'
   return PSNIP_BUILTIN_UNLIKELY(Mask == 0) ? 0 : ((*Index = psnip_builtin_ctz64 (Mask)), 1);
                                                             ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h: In function 'unsigned char psnip_intrin_BitScanReverse64(long unsigned int*, uint64_t)':
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:481:63: error: '_' was not declared in this scope
 #  define psnip_builtin_clz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,clz)(x)
                                                               ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1287:97: note: in expansion of macro 'psnip_builtin_clz64'
   return (PSNIP_BUILTIN_UNLIKELY(Mask == 0)) ? 0 : ((*Index = ((sizeof(Mask) * CHAR_BIT) - 1) - psnip_builtin_clz64 (Mask)), 1);
                                                                                                 ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:481:65: error: 'clz' was not declared in this scope
 #  define psnip_builtin_clz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,clz)(x)
                                                                 ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1287:97: note: in expansion of macro 'psnip_builtin_clz64'
   return (PSNIP_BUILTIN_UNLIKELY(Mask == 0)) ? 0 : ((*Index = ((sizeof(Mask) * CHAR_BIT) - 1) - psnip_builtin_clz64 (Mask)), 1);
                                                                                                 ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:481:68: error: 'PSNIP_BUILTIN__VARIANT_INT64' was not declared in this scope
 #  define psnip_builtin_clz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,clz)(x)
                                                                    ^
C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1287:97: note: in expansion of macro 'psnip_builtin_clz64'
   return (PSNIP_BUILTIN_UNLIKELY(Mask == 0)) ? 0 : ((*Index = ((sizeof(Mask) * CHAR_BIT) - 1) - psnip_builtin_clz64 (Mask)), 1);

sample code:

test.cpp
---
#include <stdint.h>
#include "psnip/builtin/builtin.h"
#include "psnip/endian/endian.h"

void test_do_nothing(void)
{
  uint16_t be = psnip_endian_be16(200);
}

How g++ is actually called by the Arduino IDE (newlines added for clarity):

"C:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" 
-c 
-O3 
-flto 
-fno-fat-lto-objects 
-g 
-Wall 
-ffunction-sections 
-fdata-sections 
-nostdlib
-MMD 
-fno-exceptions 
-fpermissive 
-felide-constructors 
-std=gnu++14 
-Wno-error=narrowing 
-fno-rtti 
-mthumb 
-mcpu=cortex-m4 
-fsingle-precision-constant 
-D__MK20DX256__ 
-DTEENSYDUINO=153 
-DARDUINO=10813 
-DARDUINO_TEENSY32 
-DF_CPU=24000000 
-DUSB_SERIAL_HID 
-DLAYOUT_US_ENGLISH
"-IC:\\Users\\ninjoh\\AppData\\Local\\Temp\\arduino_build_148318/pch" 
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\cores\\teensy3"
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\MFRC522\\src" 
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\libraries\\SPI"
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\Base64\\src" 
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\ArduinoBearSSL\\src" 
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\libraries\\EEPROM" 
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\EDB"
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\libraries\\Wire" 
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\libraries\\SD"
"-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\ArduinoJson\\src" 
"C:\\Users\\ninjoh\\AppData\\Local\\Temp\\arduino_build_148318\\sketch\\test.cpp"
-o "C:\\Users\\ninjoh\\AppData\\Local\\Temp\\arduino_build_148318\\sketch\\test.cpp.o"

"gnu" mode is needed for some modules like clock

Today I got all kinds of weird compile errors while building with clock/clock.h.

The underlying cause is that I had -std=c11 on the compile command line, which means certainly POSIXY things not in C are not available, including clock_gettime.

As a simple solution maybe clock could error out in that case, with a message about needing gnu mode, rather than give the compiler errors.

More comprehensive fixes are possible, but none are necessarily perfect. For example, you could declare the required macros yourself, but this won't to work if time.h was already included by someone else earlier.

I worked around it by putting a #define _GNU_SOURCE prior to including clock/clock.h.

CC0

According to https://opensource.org/faq#cc-zero, the OSI does not recommend to release OSS as public domain or CC0. Do you see any chance to change the license to BSD style, MIT or ISC?

Reconsider prioitizing use of CLOCK_MONOTONIC_RAW over non-RAW

In clock.h it seems that CLOCK_MONOTONIC_RAW is favored over CLOCK_MONOTONIC (the latter is selected only if the former doesn't exist).

I think in most cases CLOCK_MONOTONIC is better. Most importantly, it is much faster than CLOCK_MONOTONIC_RAW since it is has been implemented in the vDSO in all recent Linux versions, but CLOCK_MONOTONIC_RAW is a full system call. The difference is something like 10 nanoseconds versus 300ish nanoseconds (after Meltdown and Spectre anyways - the act performance depends a lot of what mitigations you have selected and other features).

CLOCK_MONOTONIC is still monotonic, but it is subject to NTP "rate" adjustments (i.e., when the local oscillator rate is slewed by some factor in order to cancel out error wrt network time). These are very gentle and preserve monotonicity. It isn't subject to NTP "jumps" so it never goes backwards (or big jumps forwards either, but backwards is "detectable"). Except for very unusual cases, that's what you want from a monotonic clock.

_RAW doesn't do the rate adjustment.

safe-math.h(850): error: argument is incompatible with formal parameter PSNIP_SAFE_DEFINE_BUILTIN_BINARY_OP(size_t, size, mul)

Using inteloneapi I am unable to build matio that uses portable-snippets. The platform is RHE79 with gcc 4.8.5. Any ideas how to fix this? Is there minimum requirement for gcc? This is possibly related to #45 .

Thank you.

exact-int 24-bit int

Some hardware has support for 24-bit ints, such as the eZ80 processor. Some Texas Instruments calculators use this processor (such as the TI-84 Plus CE) but exact-int doesn't give any way to use it's 24-bit ints. Could you consider adding 24-bit int support to exact-int? There's an overview of the CPU here, along with the size of its integers.

Secondly, (maybe this should be another issue), I think it would make sense to have a "min-int" mode (maybe via a define to enable it, or to have different macros along with the regular ones, eg. prefixed with m) where instead of the ints being exactly n-bits, it's only guaranteed to be at least that many bits. Most of the time, I wouldn't care if an int was bigger than I asked, only smaller. Otherwise, if you only need a 24-bit int and want to support both regular CPUs and awkward ones, you would need to check if 24-bit ints are supported, when all you care about is that it is at least that many bits. Eg. When you use a 24-bit int on a normal CPU, it would actually be 32-bits.

Thanks

`__has_builtin(__debugbreak)` does not exist in code nature, what is this?

portable-snippets/debug-trap/debug-trap.h

Lines 21 to 22 in 84abba9

    
           #  elif __has_builtin(__debugbreak) 
        
           #    define psnip_trap() __debugbreak()

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2514r0.html#_implementation_experience

Unreachable statements in clock/clock.h

From building munit-plus on Travis CI (built with C++ 2011 on PGI C++ compiler):

pgc++  --c++11 -mp -o example munit.cpp example.cpp
munit.cpp:
"munit.cpp", line 596: warning: statement is unreachable
    return 0;
    ^
"munit.cpp", line 654: warning: statement is unreachable
    return 0;
    ^
"munit.cpp", line 714: warning: statement is unreachable
    return 0;
    ^

The relevant lines on this repository seem to be in clock/clock.h (commit 5834798):

line 298 in psnip_clock_wall_get_time
line 356 in psnip_clock_cpu_get_time
line 416 in psnip_clock_monotonic_get_time

Feel free to label as wontfix.

Here is a link to the relevant build job on Travis CI: https://travis-ci.com/github/codylico/munit-plus/jobs/300147973

Here is a link to the GitHub repository for the project:
https://github.com/codylico/munit-plus

C11 backend detection for once is broken due to missing underscore

At line 26 in once.h we auto-detect use of C11 threads:

#    if __has_include(<threads.h>)
#      include <threads.h>
#      define PSNIP_ONCE_BACKEND PSNIP_ONCE_BACKEND_C11
#    endif

However, PSNIP_ONCE_BACKEND_C11 should be PSNIP_ONCE__BACKEND_C11 (note the extra underscore). Same issue occurs on line 36.

Originally reported by user jmesmon over here.

Check module clarification

Maybe I am being pedantic, but in the documentation for the check module it says (bolding mine):

Additionally, unless you define PSNIP_CHECK_FAIL_DEFINED, a failing assertion will considered undefined behavior even when NDEBUG is defined.

The "even" there seems wrong to me - I think it is saying "a failing assertion is UB even when NDEBUG is defined [and obviously also in the case it is not]", the part in brackets being implied by the "even"?

However a failing assertion should not be UB when NDEBUG is not defined, it should do the well-defined assert thing of aborting the program with a diagnostic.

If you agree, I think just removing the word even fixes it entirely.

Benchmarking program for unaligned accesses

We need a program to benchmark the different methods for the unaligned module. Shouldn't be difficult now that the clock module is in reasonable shape…

@Cyan4973, did you do anything special for that blog post, or just benchmarking xxhash? If the former, I don't suppose you still have the code sitting around somewhere (and would be willing to share it)?

exact-int.h `_STDC_VERSION_` instead of `STDC_VERSION__`

Is a typo, the very first instance in if defined(...) is missing a single leading underscore.

portable-snippets/exact-int/exact-int.h

Line 45 in 84abba9

# if defined(_STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)

Limit macros for unsigned values

In exact-int.h, I believe the limits.h macros for unsigned values are USHRT_MAX, UINT_MAX, ULONG_MAX, and ULLONG_MAX.

Also, the LLONG versions are only (required to be) available in C99/C++11 and forward.

I think technically, you are not guaranteed that the signed integer types are two's complement, so they could theoretically not work as base for the fixed-width types. But this is pretty far out on the what-if scale.

Builtins & intrinsics

Most compilers implement small, high-performance "intrinsics" (MSVC) or "builtins" (GCC) in their compilers.

Unfortunately, there are basically two sets; GCC's and MSVC's, and compilers generally only implement one. It would be great to have implementations of whatever we can, using either the current compiler's intrinsics or a portable fallback. Bit Twiddling Hacks has some code which could be useful for some of the bit twiddling builtins.

Add versioning

Hello,

Any chance to add versioning for these libraries? I use safemath and it's a bit hard to tell when a new version is "changed" or "released".

Thanks

I don't know squat about codefresh so I can't help too much here...

Modules (dlopen/LoadLibrary)

Constructor module

As mentioned in travisdowns/uarch-bench#39, it would be nice to have a macro which attempts to register a constructor function.

GLib has some code to do this already for GCC, Windows, and suncc. I'd probably want to at least a C++ version… something like (untested):

#if defined(__cplusplus)
#define PSNIP_CONSTRUCTOR(func_name) \
  static class func_name#_psnip_constructor_class_ { \
    public: \
      func_name#_psnip_constructor_class_(int x) { \
        (void) x; \
        func_name(); \
      } \
  } func_name#_psnip_constructor_(42);
#elif ...
/* ... */
#endif

	# if defined(__has_include)
	# if __has_include(<intsafe.h>)
	# define PSNIP_SAFE_HAVE_INTSAFE_H
	# endif
	# elif defined(_WIN32)
	# define PSNIP_SAFE_HAVE_INTSAFE_H
	# endif

	# elif __has_builtin(__debugbreak)
	# define psnip_trap() __debugbreak()