bvdberg / ctest Goto Github PK

I ran some tests compiled with gcc memory sanitizer options -fsanitize=address -fsanitize=leak -fsanitize=undefined and the output was fine except for the following error at the beginning of the output:

ctest.h:500:14: runtime error: load of address 0x000106c07ef8 with insufficient space for an object of type 'unsigned int'
0x000106c07ef8: note: pointer points here
 00 00 00 00  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  88 67 29 59 ff 7f 00 00  98 67 29 59

This corresponds to the line 500 in ctest.h:

if (t->magic != __CTEST_MAGIC) break;

I've never used the "deadbeef" #define __CTEST_MAGIC (0xdeadbeef) magic pointer technique so I'm not sure what's going on here. Anyone know if this is a real issue or if it can be easily fixed?

Hi,

Neat library!

As the tittle suggests, I was wondering if it's possible to have a setup/teardown function per each suite of tests instead of per each tests inside a suite?

Thanks!

If I define fixture with my own static variables, and use CTEST_SETUP / CTEST_TEARDOWN without CTEST_DATA, it will report conflict in expansion of macro 'CTEST_SETUP' in
ctest.h:140:17: note: in expansion of macro 'CTEST_IMPL_SETUP_FNAME'
and
ctest.h:138:17: note: in expansion of macro 'CTEST_IMPL_SETUP_FNAME'

Hello. Can't manage to make it build since a mac os X update

makefile

cmake_minimum_required(VERSION 3.14)
set(BIN athena_core)
project(athena_core C)

enable_testing()

set(TEST_SOURCES tests/test_core.c)
add_executable(athena_core_test ${TEST_SOURCES})
add_test(NAME athena_core_test COMMAND athena_core_test)

target_link_libraries(athena_core_test athena_core)

set(SOURCES
        casts.c
        ath_ticker.c strings.c uint128.c threads.c arrays.c)
add_library(${BIN} ${SOURCES})

test file

#include <stdlib.h>

#define CTEST_MAIN 1

#include "../../../vendor/ctest/ctest.h"

#include "../include/core.h"

int main(int argc, const char **argv) {
    return ctest_main(argc, argv);
}

CTEST(suite, test1) {
    ath_uint128 val;
    val.l = 0xAAAAAAAAAAAAAAAA; 
    val.r = 0x5555555555555555;  
    ath_core_uint128_shiftleft(&val, 0);
    ASSERT_EQUAL(val.l , 0xAAAAAAAAAAAAAAAA);
}

Result of build

====================[ Build | all | Debug ]=====================================
/Applications/CLion.app/Contents/bin/cmake/mac/aarch64/bin/cmake --build /Volumes/fswork/www/athena/src/athena_core/cmake-build-debug --target all -j 6
[2/2] Linking C executable athena_core_test
FAILED: athena_core_test 
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -g -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names  CMakeFiles/athena_core_test.dir/tests/test_core.c.o -o athena_core_test  libathena_core.a && :
ld: warning: alignment (1) of atom '_ctest_suite_test' from '/Volumes/fswork/www/athena/src/athena_core/cmake-build-debug/CMakeFiles/athena_core_test.dir/tests/test_core.c.o' is too small and may result in unaligned pointers 
ld: warning: alignment (1) of atom '_ctest_suite_test1' from '/Volumes/fswork/www/athena/src/athena_core/cmake-build-debug/CMakeFiles/athena_core_test.dir/tests/test_core.c.o' is too small and may result in unaligned pointers 
ld: building fixups: pointer not aligned at _ctest_suite_test+0x0 from /Volumes/fswork/www/athena/src/athena_core/cmake-build-debug/CMakeFiles/athena_core_test.dir/tests/test_core.c.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

WIth a modern C-compiler (GCC 5.4.0), and the simple test source:

#include "ctest.h"
CTEST_SETUP(Error) {}
CTEST(Error, test) {}

you get:

ctest.h:66:31: warning: ‘struct ctest_Discoverer_data’ declared inside parameter list
#define CTEST_IMPL_NAME(name) ctest_##name
                              ^
ctest.h:89:64: note: in expansion of macro ‘CTEST_IMPL_NAME’
    void CTEST_IMPL_WEAK CTEST_IMPL_NAME(sname##_setup)(struct CTEST_IMPL_NAME(sname##_data)* data)
                                                               ^
discoverer_tests.c:20:1: note: in expansion of macro ‘CTEST_SETUP’
 CTEST_SETUP(Discoverer) {
 ^
ctest.h:66:31: warning: its scope is only this definition or declaration, which is probably not what you want
 #define CTEST_IMPL_NAME(name) ctest_##name
                                                        ^
ctest.h:89:64: note: in expansion of macro ‘CTEST_IMPL_NAME’
 void CTEST_IMPL_WEAK CTEST_IMPL_NAME(sname##_setup)(struct CTEST_IMPL_NAME(sname##_data)* data)

Examples are not clear on if this is a valid case. Is there something else that is required to make this correct?

AddressSanitizer complaining about global-buffer-overflow at https://github.com/bvdberg/ctest/blob/master/ctest.h#L414

In certain test-cases it's necessary to test for an exact float-point value. ASSERT_EQUAL is not suitable for this because it assumes arguments to be integers. ASSERT_FLT_NEAR is not suitable either, because it will accept not equal values.

The workaround I currently use is ASSERT_TRUE(function_under_test() == 1.0f), which works as expected, but is not as readable as ASSERT_EQUAL(function_under_test(), 1.0f).

I mean this one. Is this intentional, a WIP, an error, something else?

Hi,

I started seeing test cases failing out of nowhere when they were executed after another test that had just failed. I thought I might be something wrong with my code - maybe I wasn't cleaning up my pointers or memory correctly... after a bit of research I found out that indeed the memory was not being clean up because the teardown function was not being called after a failing test case.

This simple example shows how the "bye from teardown" message is only being printed once.

#include <stdio.h>
#include "ctest.h"

CTEST_DATA(foo) {
};

CTEST_SETUP(foo) {
    puts("\nhi from setup");
}

CTEST2(foo, test1) {
    ASSERT_FAIL();
}

CTEST2(foo, test2) {
    ASSERT_TRUE(1);
}

CTEST_TEARDOWN(foo) {
    puts("bye from teardown");
}

The output is

$ ./test
TEST 1/2 foo:test2 
hi from setup
bye from teardown
[OK]
TEST 2/2 foo:test1 
hi from setup
[FAIL]
  ERR: mytests.c:13  shouldn't come here
RESULTS: 2 tests (1 ok, 1 failed, 0 skipped) ran in 0 ms

If you are OK wit it I could send you a PR to fix it?

Thanks for ctest, I absolutely love it!
Nico

Hi,

first, thank you for this project is really nice!!!!

I have a couple of questions:

1. there's a "nice way" without changing the ctest.h header to redefine the vsnprintf function?
   in my system I have to use a specific test_msg() function (for various reasons) to report text messages to the system so I cannot use any "printf" function. how can I extend ctest to use custom printf functions?

2. Also I have another function test_results() to report results to the main test framework.
   there's a way to call a user function to report test results for the single test? how will you handle this?

3. how can I define different test suites and skip/execute those depending on a "configuration":

for example I can have

suite1
suite2
suite3

configuration1 -> execute 1, skip 2 and 3
configuration2 -> execute 1, skip 2 execute 3
configuration3 -> execute 1, 2 and 3

switch(test_configuration)
case NO_STRESS_TEST
result = ctest_main(configuration1)
case TEST_ALL
result = ctest_main(configuration3)

ctest is not distributed with any licensing information. I'm hesitant to use it on my own projects without that info. Is it possible to distribute it under a permissive license? I'm a fan of MIT :-).

User has to pay extra attention since setup/teardown would not work for CTEST.

Hi!

Can you make a release, please? It'll help a lot with packaging your library into different dependency managers (like Conan).

Thank you!