http://fctx.wildbearsoftware.com/static/fctx/doc/1.6.1/api_fct.html#quick-test

The end of the second test should say FCT_QTEST_END -- it says FCT_TEST_END

Sometimes the following pattern,

char _data;
FCT_SUITE_BGN(xxx) {
FCT_SETUP_BGN() {
data = malloc(sizeof(char)_N);
}FCT_SETUP_END();
FCT_TEARDOWN_BGN() {
free(data);
} FCT_TEARDOWN_END();
FCT_TEST_BGN() { /* do test */ } FCT_TEST_END();
FCT_SUITE_END();

is akward. It is especially akward when the "data" is a C++ object that lacks a default constructor. Perhaps you have an object you would like to do a one-time initialization on, for instance:

config_t config("name", 123);
config.add(some_object_t(someparam));
config.add(other_object_t(otherparam));

Those "add" commands can't be in the "global" space, but you don't really want to call them over and over again with the setup/teardown. Its a config object that is read-only for the test suite for example.

The documentation module index refers to the "Python Module Index". Could be a bug in Sphinx (which grew up from Python), or a missing configuration parameter.

The Junit/XML reporter captures stdout/stderr by redirecting into a pipe, then reading it back in the same process. If an individual test prints enough output to fill up the pipe, the test process deadlocks waiting for a reader that is never going to drain the pipe.

Reproduce on Ubuntu 10 by:

#include "fct.h"

FCT_BGN()
{
    FCT_QTEST_BGN(too_much_output)
    {
        int x;
        for (x =0; x<100000; x++ ) 
            printf("xxx\n");
    }
    FCT_QTEST_END();
}
FCT_END();

and then:

./a.out --logger junit

This affects the junit logger and custom loggers that use the same mechanism to capture stdout/stderr.

If I get a chance, I will try replacing the pipe with a temp file, but I'm not sure I will get to it any time soon.

The simple workaround is: Don't let the test blab so much.

Sometimes you have a test that you know is wrong, but are waiting for some further changes to the API in order to finally bring that test back online.

It would be useful if we can tell FCTX to skip a certain test, and describe why to skip it.

Rough idea is,

fct_skip(<test_name>, "reason why");

That might suffer in a "typo" not causing a skip, so,

fct_skipnext("reason why");
FCT_TEST_BGN(xyz) {
} FCT_TEST_END();

and in the log you would get a "SKIP" and the next line would be a reason why you skipped it.

The goal of this is to be able to somehow manage data in a pristine state from test to test. Some tests require files in order run tests against them. It would be nice if you could somehow take care to load/unload these files and not have to worry about making sure the file hasn't been corrupted by some previous unit test.

Be able to execute some subset of tests based on a tag that was supplied for each test.

FCTest needs to be able to handle exceptions well. It should be able to survive a system failure with minimal issues.

Ideas:

Launch the program in a separate process? Using the full test name would technically make this doable.
Look into structured exception handling, can write some wrappers? What do to on a Linux system?

Optional?

define FCT_OPTION__USE_EXCEPTION_HANDLING

include "fct.h"

I'd like all the standard string tests to support the std::string and std::wstring, I would also like to add a checker for verifying that an exception is correctly generated.

It would be great if FCTX grew an option to generate XML style logs that integrate with tools like Ant and Cruise Control.

See this thread for more info:

http://n3.nabble.com/JUnit-Integration-td453995.html#a453995

Pretty straightforward. Just run the fct_timer object for each test as well as for the entire test run.

The main reason for this is to provide some immediate feedback on tests that appear to be getting longer and longer to execute.

Have the ability to use the result of a check to avoid a set of other checks.

For example,

{{{
if ( !fct_chk(CONDITION) ) {
/* No point running the following fct_chk, because that CONDITION was important. /
}
/ We can run other checks here... */

}}}

I found that this was an interesting idea because it helps flush out inter-dependencies between tests that may result in a "correct" test.

In a way it helps check for dependencies.

If we did this we would need to have the ability to,

1. Randomize with a specified seed.
2. Report the seed used to generate the error (so it is reproducible).

It would be nice if the all the error strings where displayed in an IDE "output window". This way when the user double clicks on the line with the error message it will jump them to the test in the code.

I know for Visual Studio if you use the WIN32 OutputString call it will go to the output window.

Would like to move the standard log reporting to use the T.A.P. (http://testanything.org/wiki/index.php/Main_Page).

The existing "standard" would become "fctx", and "tap" would become the new default.

Currently FCTX has the fct_chk_eq_str with its neq and case-insenstive variants. This blueprint is to track some additional checks, namely,

fct_chk_str_incl : determines if the target string includes the substring.
fct_chk_str_excl : determines if the target string excludes the substring.
fct_chk_str_starts : determines if the target string starts with the substring.
fct_chk_str_ends: determines if the target string ends with the substring.
fct_chk_str_empty: returns true if the string is empty.
fct_chk_str_full: returns true if the string is full.

These should help some common tests quickly indicate the reason for their failure.

Does anyone have a complete copy of the previous website? Do we want to duplicate that somewhere (GitHub?) so the information remains available? Obviously one can use archive.org to get to (most of) the old site. However, that doesn't seem like the best solution going forward.

Quite an ambitious plan, and may just be a matter of writing up the steps required to accomplish this with various compilers.

Inspired by the CMake "execute_process" function (http://www.cmake.org/cmake/help/cmake-2-8-docs.html#command:execute_process). Sometimes it would be handy to execute a program with different command line argument s and make sure it is "doing the right thing" by checking the output.

With the start of issue #10, it would be nice set up the ability to hook in different loggers. The first pass should look to be able to hook in "built-in" loggers, but the second pass should be too difficult for folks to extend fctx and register their own loggers.

If the fctcl_val2 was not defined on the command line it may not return the default value.

Please consider allowing the use of fct_req within FCT_SETUP_BGN "blocks".
This is useful to allow for sanity checking fixture setup conditions.

Currently, using fct_req within FCT_SETUP_BGN causes

fct_test__add: Assertion `test != ((void *)0)' failed.

for an example like the following

char * filename = NULL;

FCT_FIXTURE_SUITE_BGN(example_suite)
{
    FCT_SETUP_BGN()
    {
        /* filename = malloc(....) */
        fct_req(filename);
    }
    FCT_SETUP_END();

    FCT_TEARDOWN_BGN()
    {
        if (filename) free(filename);
    }
    FCT_TEARDOWN_END();

    FCT_TEST_BGN(debug)
    {
        /* NOP */
    }
    FCT_TEST_END();
}
FCT_FIXTURE_SUITE_END();

The FCTX code base should have been written with wchar_t* in mind, instead of using char*. This issue is to track support for it.

When trying to compile the test_big, with 1024 tests cases, using gcc with release flags, my system runs out of memory. The debug build proceeds happily.

I can only speculate that the release optimizer can not handle a single "function" with 1024 test cases within it, because this is what test_big looks like.

At the moment the test was modified to only generate 256 tests in release. The
number was picked out of the air, and was not precisely generated.

A workaround is not to let your test suites get this large, and to break up
your tests either with different EXE's or with the FCTMF (FCTX Multi-file API).