arturassiscomp / ctest

CTEST

An unit test lib for C.
Explore the docs »

Report Bug · Request Feature

Table of Contents

1. Description
2. How to contribute
3. Requirements
4. How to use the makefile
5. Using ctest
   • Example
6. Documentation
   index
   Overview
   Types
   Variables
   Constants
   Functions
   • Single data types
   • unsigned_integer
   • assert_unsigned_integer_equal
   • assert_unsigned_integer_notEqual (implement)
   • assert_unsigned_integer_greater (implement)
   • assert_unsigned_integer_greaterEqual (implement)
   • assert_unsigned_integer_less (implement)
   • assert_unsigned_integer_lessEqual (implement)
   • assert_unsigned_integer_bitMaskEqual (implement)
   • integer
   • assert_integer_equal (implement)
   • assert_integer_notEqual (implement)
   • assert_integer_greater (implement)
   • assert_integer_greaterEqual (implement)
   • assert_integer_less (implement)
   • assert_integer_lessEqual (implement)
   • floating_point
   • assert_floating_point_almostEqual (implement)
   • assert_floating_point_notAlmostEqual (implement)
   • assert_floating_point_greater (implement)
   • assert_floating_point_greaterEqual (implement)
   • assert_floating_point_less (implement)
   • assert_floating_point_lessEqual (implement)
   • bool
   • assert_bool_equal (implement)
   • assert_bool_notEqual (implement)
   • assert_bool_true (implement)
   • assert_bool_false (implement)
   • char
   • assert_char_equal (implement)
   • assert_char_notEqual (implement)
   • assert_char_greater (implement)
   • assert_char_greaterEqual (implement)
   • assert_char_less (implement)
   • assert_char_lessEqual (implement)
   • file
   • assert_files_binaryContentEqual (implement)
   • assert_files_contentEqual (implement)
   • assert_file_path_contentEqual (implement)
   • Void pointer
   • assert_pointer_isNULL (implement)
   • assert_pointer_notIsNULL (implement)
   • Array data type
   • unsigned_integerArray
   • assert_unsigned_integerArray_equal (implement)
   • assert_unsigned_integerArray_notEqual (implement)
   • assert_unsigned_integerArray_sorted (implement)
   • assert_unsigned_integerArray_notSorted (implement)
   • assert_unsigned_integerArray_permutation (implement)
   • assert_unsigned_integerArray_notPermutation (implement)
   • assert_unsigned_integerArray_isPartialPermutation (implement)
   • assert_unsigned_integerArray_notIsPartialPermutation (implement)
   • assert_unsigned_integerArray_hasPartialPermutation (implement)
   • assert_unsigned_integerArray_notHasPartialPermutation (implement)
   • assert_unsigned_integerArray_setEqual (implement)
   • assert_unsigned_integerArray_setIn (implement)
   • assert_unsigned_integerArray_notSetIn (implement)
   • assert_unsigned_integerArray_setHas (implement)
   • assert_unsigned_integerArray_notSetHas (implement)
   • assert_unsigned_integerArray_isSubarray (implement)
   • assert_unsigned_integerArray_notIsSubarray (implement)
   • assert_unsigned_integerArray_hasSubarray (implement)
   • assert_unsigned_integerArray_notHasSubarray (implement)
   • integerArray
   • assert_integerArray_equal (implement)
   • assert_integerArray_notEqual (implement)
   • assert_integerArray_sorted (implement)
   • assert_integerArray_notSorted (implement)
   • assert_integerArray_permutation (implement)
   • assert_integerArray_notPermutation (implement)
   • assert_integerArray_isPartialPermutation (implement)
   • assert_integerArray_notIsPartialPermutation (implement)
   • assert_integerArray_hasPartialPermutation (implement)
   • assert_integerArray_notHasPartialPermutation (implement)
   • assert_integerArray_setEqual (implement)
   • assert_integerArray_setIn (implement)
   • assertd_integerArray_notSetIn (implement)
   • assert_integerArray_setHas (implement)
   • assert_integerArray_notSetHas (implement)
   • assert_integerArray_isSubarray (implement)
   • assert_integerArray_notIsSubarray (implement)
   • assert_integerArray_hasSubarray (implement)
   • assert_integerArray_notHasSubarray (implement)
   • boolArray
   • assert_boolArray_equal (implement)
   • assert_boolArray_allTrue (implement)
   • assert_boolArray_anyFalse (implement)
   • assert_boolArray_allFalse (implement)
   • assert_boolArray_anyTrue (implement)
   • charArray
   • assert_charArray_equal (implement)
   • assert_charArray_notEqual (implement)
   • assert_charArray_sorted (implement)
   • assert_charArray_notSorted (implement)
   • assert_charArray_permutation (implement)
   • assert_charArray_notPermutation (implement)
   • assert_charArray_isPartialPermutation (implement)
   • assert_charArray_notIsPartialPermutation (implement)
   • assert_charArray_hasPartialPermutation (implement)
   • assert_charArray_notHasPartialPermutation (implement)
   • assert_charArray_setEqual (implement)
   • assert_charArray_setIn (implement)
   • assertd_charArray_notSetIn (implement)
   • assert_charArray_setHas (implement)
   • assert_charArray_notSetHas (implement)
   • assert_charArray_isSubarray (implement)
   • assert_charArray_notIsSubarray (implement)
   • assert_charArray_hasSubarray (implement)
   • assert_charArray_notHasSubarray (implement)
   • string
   • assert_string_equal (implement)
   • assert_string_notEqual (implement)
   • assert_string_greater (implement)
   • assert_string_greaterEqual (implement)
   • assert_string_less (implement)
   • assert_string_lessEqual (implement)
   • assert_string_sizeEqual (implement)
   • assert_string_notSizeEqual (implement)
   • assert_string_sizeGreater (implement)
   • assert_string_sizeGreaterEqual (implement)
   • assert_string_sizeLess (implement)
   • assert_string_sizeLessEqual (implement)
   • assert_string_sameSize (implement)
   • assert_string_hasChar (implement)
   • assert_string_notHasChar (implement)
   • assert_string_sorted (implement)
   • assert_string_notSorted (implement)
   • assert_string_permutation (implement)
   • assert_string_notPermutation (implement)
   • assert_string_isPartialPermutation (implement)
   • assert_string_notIsPartialPermutation (implement)
   • assert_string_hasPartialPermutation (implement)
   • assert_string_notHasPartialPermutation (implement)
   • assert_string_setEqual (implement)
   • assert_string_setIn (implement)
   • assertd_string_notSetIn (implement)
   • assert_string_setHas (implement)
   • assert_string_notSetHas (implement)
   • assert_string_isSubarray (implement)
   • assert_string_notIsSubarray (implement)
   • assert_string_hasSubarray (implement)
   • assert_string_notHasSubarray (implement)
   • assert_string_empty (implement)
   • assert_string_notEmpty (implement)
   Macros

(back to top)

Description

This repository contains a library that can be used to implement unit tests for C code.

(back to top)

How is a contribution made?

1. Create a branch from an issue;
2. Clone the repository:

git clone  https://github.com/ArturAssisComp/ctest.git

3. Change to the desired branch:

cd ctest
git checkout <branch-name>

4. For each meaningful change, make a commit;

-> Fist, add the files that were changed:

git add <filename1> <filename2> ... <filenameN>

-> Commit the changes:

git commit -m "Implemented function foo()"

5. Push the changes:

git push origin <name-of-the-branch>

6. If more changes are necessary, go to 4. Else, go to 7;
7. Create a pull request and assign someone to review the changes that were made in the branch;

=> Other useful commands:

Check the history of commits

git log

Check the status of the repository

git status

Get changes from remote repository from branch master

git pull origin <master-branch-name>

(back to top)

Requirements

make 4.3

gcc 11.2.0

(back to top)

How to use the makefile?

Run the following command in the terminal in the root directory of the ctest project:

make

The headers, object files, and archive lib object will be saved into ./build/ folder. They may be used to implement unit tests using ctest lib in other projects.

To clean ./build/ folder, run:

make clean

In order to run all tests:

make test

or, to print the results to txt files in ./build/test_results/

make test_to_txt

(back to top)

How to use ctest in other projects?

Run the command make from the ctest base directory:

make

then, the headers and the archive ctest.a files will be created and saved into the folder ./build/ctest_lib/

The files are organized as follows:

headers --> ./build/ctest_lib/include/

ctest.a --> ./build/ctest_lib/lib/

In order to use the library, the user must copy the content from ctest_lib to the desired folder in the target project, import the header ctest.h, and compile the file with the unit tests using as input, among others, the ctest.a object file.

Example of usage

Let us suppose that the base directory for ctest is CTEST_BASE_DIR and the base directory for the target project, in which the unit tests will be implemented, is BASE_DIR. In an actual case, it is necessary to exchange BASE_DIR and CTEST_BASE_DIR to the respective actual directory name.

1 - Execute the make command from CTEST_BASE_DIR:

cd CTEST_BASE_DIR

make

2 - Copy the folder CTEST_BASE_DIR/build/ctest_lib and its contents to the target project:

cp -r CTEST_BASE_DIR/build/ctest_lib BASE_DIR/

3 - Create the unit test file with name unit_test.c:

//C file that implements the unit tests
#include "ctest_lib/include/ctest.h"


#define MSG_SIZE 100

int add(int a, int b)
{
    return a + b;
}

int main(void)
{
	char suite_name[] = "Unit test";
	char suite_description[] = "Basic test case.";
	char *functions_tested[] = {
        "add",
		NULL
	};
	char error_msg[MSG_SIZE] = "";
	char msg[MSG_SIZE] = ""; 


	//------------------------------------------------------------------------------
	start_suite(suite_name, suite_description, functions_tested);
		//------------------------------------------------------------------------------
		start_module("EDGE CASES", "Module for testing edge cases.", functions_tested);
            /* 0 + 0 */
            assert_integer_equal(add(0, 0), 0, __LINE__, "add(0, 0) == 0");
            /*------*/
            /* -1 + 0 */
            assert_integer_equal(add(-1, 0), -1, __LINE__, "add(-1, 0) == -1");
            /*------*/
            /* -1 + 1 */
            assert_integer_equal(add(-1, 1), 0, __LINE__, "add(-1, 1) == 0");
            /*------*/
            /* 1 + 1 */
            assert_integer_equal(add(1, 1), 2, __LINE__, "add(1, 1) == 2");
            /*------*/
            /* -1 + 2 */
            assert_integer_equal(add(-1, 2), 1, __LINE__, "add(-1, 2) == 1");
            /*------*/    
		end_module();
		//------------------------------------------------------------------------------
    		//------------------------------------------------------------------------------
		start_module("Domain CASES", "Module for testing domain cases.", functions_tested);
            /* 123 + 44 */
            assert_integer_equal(add(123, 44), 167, __LINE__, "add(123, 44) == 167");
            /*------*/
            /* -14 + 10 */
            assert_integer_equal(add(-14, 10), -4, __LINE__, "add(-14, 10) == -4");
            /*------*/ 
		end_module();
		//------------------------------------------------------------------------------
        
		
	end_suite();
	//------------------------------------------------------------------------------
}

4 - Compile the unit test file and execute it:

cd BASE_DIR

gcc -o unit_test unit_test.c ctest_lib/lib/ctest.a

5 - Execute unit_test and check the results:

unit_test

(back to top)

Documentation

Overview

• About assert functions: every assert function has the signature in the following format:

void assert_<type>_<description>(..., int line_number, char custom_message[]).

<type> describes the type of the target that will be used in the assertion process. <description> specify the type of assertion that will be made. As an example, <type> may be integer and <description> may be equal, resulting in assert_integer_equal.

line_number and custom_message are useful for printing meaningful failure messages. It is recommended to use the macro __LINE__ as the input line_number. custom_message may be NULL if no custom message will be used for failure message.

(back to top)

Types

(back to top)

Variables

(back to top)

Constants

(back to top)

Functions

Single data types

unsigned_integer

assert_unsigned_integer_equal

• Signature:

void assert_unsigned_integer_equal (unsigned_integer target, unsigned_integer reference, int line_number, char custom_message[]);

• Description: this function checks if the unsigned_integer target is equal to the unsigned_integer reference.

Example

#include "ctest_lib/include/ctest.h"
int main(void)
{
    char *functions_tested[] = { "None", NULL };
    start_suite("Unit test", "Basic test case.", functions_tested);
        start_module("Success module", "Basic test case.", functions_tested);
            assert_unsigned_integer_equal(2, 2, __LINE__, NULL); //pass
        end_module();	
        start_module("Fail module", "Basic test case.", functions_tested);
            assert_unsigned_integer_equal(0, 2, __LINE__, NULL); //fail
        end_module();	
    end_suite();
}

(back to top)

integer

(back to top)

floating_point

(back to top)

bool

(back to top)

char

(back to top)

file

(back to top)

void pointer

(back to top)

Array data types

unsigned_integerArray

(back to top)

integerArray

(back to top)

boolArray

(back to top)

charArray

(back to top)

string

(back to top)

Macros

(back to top)

ctest's Issues

Create a "goto finish:" to handle all the errors at the same place and make the code more readable.

The function is in std_assert_functions.c and is an auxiliary function. Its name is unsigned_integerArray_generate_aligned_str. The link is below.

ctest/ctest_library/assert/std_assert_functions.c

Line 2632 in c85037a

//------------------------------------------------------------------------------

Refactor the formatting string lib.

The functions from the formatting string auxiliary libs are poorly implemented. It is necessary to refactor them.

Create a function to print messages during the result of the tests.

Create a function that makes it easier to print messages during the result of the tests. If printf is directly used, the format of the result breaks.

The limit for the size of the module name and the suite name is too small.

Increase the limit size for the module name and for the suite name.

Create a detailed readme for the project

• Create instructions about how to contribute to the project;
• Create instructions about how to use the project, including configuration details;
• Create examples and use cases;

Bug in the start_module function from ctest_functions.h

The bug was noticed when testing assert macros.
The following piece of code was used in two different files (test_std_assert_macros_success.c and test_std_assert_macros_fail.c) and an error happened in one of them (test_std_assert_macros_fail.c).

start_module("UI ARRAY-notIsPartPerm", "Every assert in this module must pass.",\
(char *[]){"ASSERT_UNSIGNED_INTEGER_ARRAY_NOT_IS_PARTIAL_PERMUTATION", NULL});
(...)
end_module();

The error does not happen when the third parameter is changed from

(char *[]){"ASSERT_UNSIGNED_INTEGER_ARRAY_NOT_IS_PARTIAL_PERMUTATION", NULL} to
(char *[]){"A_UNSIGNED_INTEGER_ARRAY_NOT_IS_PARTIAL_PERMUTATION", NULL}

Empty custom message bug

If custom message is an empty string, the test failure message is not formatted properly.
Example:

assert_integer_equal(0, 1, __LINE__, "");

Is it necessary to have a global variable for the result of each unit test? (design issue)

Each assert function of this library uses a global variable called global_result to store the results of the test. It stores a bool that says if the test was successful and the details of the result. After that, it calls the print_result() function passing some arguments from the current test but, for some reason, it does not pass the result and the detailed message. Maybe it is a good idea to eliminate this global variable.

Finish the documentation of the project

Finish the writing process of the remaining entries of the documentation of the project that is in the readme.

Bug in ctest.c functions start_module() and start_suite()

When an empty string is used as description or name parameters the report loses its formatting.

Implement a better floating point almost equal

The assert functions almostEqual used to compare floating points were implemented using a naive algorithm. It is interesting to implement a better and more sophisticated algorithm.
The implemented algorithm uses the approach of comparing the difference between two floating points with an absolute value. That is a very naive and inefficient approach. The idea is to implement a technique that uses the concept of UPL.
To do list:

• Read the paper: "What Every Computer Scientist Should Know About Floating-Point Arithmetic";
• Implement an algorithm for almostEqual comparison between floating points using the concept of "units in the last place" (ULP);