I have a test looking basically like this:

REQUIRE(foo() != static_cast<char*>(NULL));

the function foo return a pointer to a char (not a null-terminated string, just a single char), or null if the operation failed. However, Catch assumes that these pointers (on both sides of the !=) are null-terminated strings, and so it crashes with a segfault/access violation when it tries to print out the rhs, a null pointer of type char*.

I'm not really sure how this should be handled, since of course a char* often does represent a string. But automatically treating it as a null-terminated string is dangerous too (and can result in Catch crashing on 100% valid test code).

One option would be to treat char* as any other pointer type (print out the address), and if the user wants it treated as a string, he'll have to wrap it in a std::string or another string type, like this:

REQUIRE(std::string(foo()) != std::string("foo"));

Or of course a separate macro could be created, REQUIRE_STR, perhaps (or REQUIRE_PTR), to resolve the ambiguity. Ugly, but a possible solution.

Again, not sure how to handle this most correctly, but it just tripped me up, so I thought I'd pass it on at least.

I have a test which looks like this:

TEST_CASE("MyClass", "MyClass basic tests") {
    SECTION("MyClass::MyClass", "MyClass constructor tests") {
        SECTION("defaults", "Default values for member varirables") {
            MyClass obj;
            REQUIRE(obj.a==0);
        }
    }
}

The test compiles correctly, but no tests are run. If I remove the inner SECTION and move the code up to the outer SECTION block they are run correctly.

The overload on ResultBuilder for ->* with const char* should take it's argument by const ref. Ie change this:

Expression<const char*> operator->*
(
    const char* operand
)
{
    Expression<const char*> expr( m_result, operand );

    return expr;
}

to this:

Expression<const char*> operator->*
(
    const char* const& operand
)
{
    Expression<const char*> expr( m_result, operand );

    return expr;
}

I have a test which looks like this (simplified):

for (int i = 0; i < N; ++i) {
    REQUIRE(is_true(make_val(i), make_val(i+1));
}

because i isn't a literal, and it isn't used at the top-level in the REQUIRE macro, its value doesn't get printed out if the test fails, meaning that I don't know which iteration of the loop failed.

A similar problem could occur if you have a template function containing a REQUIRE (so you can run the same tests on a variety of types), in which case you'd want to be able to print out the current type on failure.

It would be helpful to have a mechanism allowing me to register objects or strings to be printed out if an assertion fails in the current scope

Perhaps something like this:

for (int i = 0; i < N; ++i) {
    ON_FAILURE(i); // would print out the value of i if a test fails, and be silent otherwise
    REQUIRE(is_true(make(i), make(i+1))
}

Of course, it would have to unregister itself when you leave the scope, or it would end up printing every value of i.

Compiling the Catch tutorial with g++ -Wall -Wextra and the stand-alone catch.hpp from f5c057e produces the following warning:

% g++ -Wall -Wextra -o tutorial tutorial.cpp
In file included from tutorial.cpp:2:0:
catch.hpp:3055:5: warning: unused parameter ‘descriptor’
%

REQUIRE(atan2(-862.42, 78.5149) == -1.480006);

The above code fails, with the output:

atan2(-862.42, 78.5149) == -1.48001 failed for: -1.48001 == -1.48001

I am not sure why, considering they are equal. I initially tried it with Approx, but the result was the same.

The provided solution (TestCatch) fails to compile as well. The errors are:

Error 1 error C2780: 'const _Ty &std::max(const _Ty &,const _Ty &,_Pr)' : expects 3 arguments - 2 provided d:\hgrepositories\twoloc\twolocdep\src\philsquared-catch-89d1e6c\include\internal\catch_approx.hpp 47 TestCatch

Error 2 error C2782: 'const _Ty &std::max(const _Ty &,const _Ty &)' : template parameter '_Ty' is ambiguous d:\hgrepositories\twoloc\twolocdep\src\philsquared-catch-89d1e6c\include\internal\catch_approx.hpp 47 TestCatch

I have looked through the code briefly, and it seems like the error is due to the member variables being double thus confusing the fabs and max functions. If I do a c-style cast to make the line like this:

return fabs( (lhs - (T)rhs.m_d) ) < (T)rhs.m_epsilon * ((T)rhs.m_scale + (std::max)( fabs(lhs), (T)fabs(rhs.m_d) ) );

Then everything is fine, although that is unsafe. My suggestion would be make the Approx class a template, and typedef basic types such as Approxf for Approx and Approxd for Approx. Of course, this suggestion is based on limited knowledge of your testing framework. I did do that in my code currently, and the above mentioned line is back to the way it was before, and everything compiles/runs/tests fine.

On MSVC, this triggers an assertion failure (subscript out of range) in std::string:

TEST_CASE("case", "")
{
    SECTION("section", "")
    {
        INFO("hi");
        REQUIRE(true);
    }
}

it's being accumulated with |= when it should be &=

REQUIRE( true ); ( REQUIRE is regarded as error and have to remove the ifdef windows, since window is default as vc++ now )

Invalid arguments '
Candidates are:
enum Catch::ResultAction::Value acceptExpression(const Catch::MutableResultInfo &)
'

Required for JUnit output - but useful in general

Can it give me a compile error when I've done something stoopid?

The Minimal case below doesn't compile:

#import<Foundation/Foundation.h>
#define CATCH_CONFIG_MAIN
#import"catch.hpp"

TEST_CASE("basic/first", "Suppose to fail") {
    REQUIRE(2==0);
}

Compile command used (assumed that "catch.hpp" is in the same directory as mm file):

gcc -o test BasicTest.mm -framework Foundation -lobjc

This is using gcc version 4.2.1 on OSX 10.6.8 (64-bit).

When text is encoded for attributes or text nodes double-quote characters are encoded as their entity reference representation.

That should be &quot;, but it was incorrectly being written as &quote;

Most of the time this is fairly benign - until something (like CruiseControl.Net) attempts to validate the reference (in CC.Net's case it sticks it in a CDATA - so it is ignored by XSLTs - silently dropping the error report! - rather nasty really).

This should simple to remediate.
It might also be worth suppressing quote escaping except for attributes anyway.

Currently, exceptions not derived from std::exception (other than raw strings) are caught by CATCH, but it doesn't know how to get any messages out.
A mechanism is needed to be able to specify custom exception types so more informative messages can be given.

I can't think of any clean (portable) ways to do this OTTOMH, but one way to "hack it" would be to have a macro, CUSTOM_EXCEPTION_TYPE, as well as CUSTOM_EXCEPTION_HANDLER, which would allow the user to specify a type and how to get the string out respectively.

This has a number of drawbacks. Apart from being very ugly it only allows one custom exception type to be specified (unless we have CUSTOM_EXCEPTION_TYPE2 etc), and the caller has to make sure the macros are defined before catch is included.

Any better ideas are very welcome!

I've just seen this in a CI context - not had a chance to drill in to the root cause, but it looks like there's definitely a bug in the stream handling when using -o to write to a file. I had the last line end up on stdout instead, which is not great when using the xml reporter to feed into an xslt :-s

Simple workaround for now is to use > to redirect stdout to a file instead of using -o.
Will try and fix it properly soon.

I was doing some testing for my presentation and accidently forgot to use CHECK instead of require. This code goes into an infinite loop (using single header btw). If you replace the REQUIRE with CHECK it's fine so I guess the exception is causing the looping. VC10 & gcc both fail.

define CATCH_CONFIG_MAIN

include "catch.hpp"

int g_count = 0;

struct counter
{
counter()
{
++g_count;
}
};

TEST_CASE("counter/sections", "Show multiple constructor calls")
{
counter c;
REQUIRE( g_count == 1 );

SECTION( "s1", "" ) { }

SECTION( "s2", "" )
{
    SECTION( "s3.1", "" ) { }
    SECTION( "s3.2", "" ) { }
    SECTION( "s3.3", "" ) { }
}

}

Malcolm

I had code like this and somehow in Debug Build my tests failed:

REQUIRE(task->isTerminated() == false);
REQUIRE(task->isRunning() == false);
REQUIRE(task->execStatus() == Task::Created);

The following tests the scenario just with true and false statements.

#include "catch_with_main.hpp"

ANON_TEST_CASE()
{
    CHECK(false == false); // Fails in Debug!!!
    CHECK(true == true);   // Fails in Debug!!!

    // Extra parenthesis make it work!
    CHECK((false == false));
    CHECK((true == true));

    CHECK_FALSE(false == true);
    CHECK_FALSE(true == false);

    // Extra parenthesis make it fail!
    CHECK_FALSE((false == true)); // Fails in Debug!!!
    CHECK_FALSE((true == false)); // Fails in Debug!!!
}

Change:
m_messageStream << value;
to
m_messageStream << Catch::toString( value );

(line 579)

In setStreamBuf() a dynamic_cast is used to see if the current stream object is a custom one (derived from StreamBufBase). If so it deletes it before setting the new one.

However, if it is not a custom streambuf it may have been built without RTTI - and the call blows up when the dynamic cast is attempted. At least I have seen this in Windows with VS2008 (in a project with RTTI enabled).

The code should be refactored to an alternate means of determining ownership (probably by always wrapping in a custom stream object)

Title says it all.
A generator within a section only seems to "generate" its first value. I suspect this is because generators and sections both cause test_cases to rerun - and section reruns presumably trump generator reruns.

I wrote a simple test which tests the value of a boost::trilogic variable. It boils down to this:

#include <boost/logic/tribool.hpp>

class A {
    boost::trilogic flag;
public:
   A() : flag(boost::indeterminate) { }
   boost::trilogic hasFlag() const { return flag; }
};

TEST_CASE("A", "class A tests") {
    SECTION("A::hasFlag", "hasFlag behaviour") {
       REQUIRE(A().has_flag()==boost::indeterminate);
    }
}

Trying to compile that test blows up rather badly:

../../src/tests/article.cc: In function ‘void catch_internal_TestFunction8()’:
../../src/tests/article.cc:213: error: no match for ‘operator==’ in ‘Catch::ResultBuilder(((const char*)"../../src/tests/article.cc"), 213u, ((const char*)"REQUIRE"), ((const char*)"article.hasFormula()==boost::indeterminate"), 0).Catch::ResultBuilder::operator->* [with T = boost::logic::tribool](((const boost::logic::tribool&)((const boost::logic::tribool*)(& article. Article::hasFormula())))) == boost::logic::indeterminate’
../../catch/internal/catch_capture.hpp:360: note: candidates are: Catch::MutableResultInfo& Catch::Expression<T>::operator==(const RhsT&) [with RhsT = bool ()(boost::logic::tribool, boost::logic::detail::indeterminate_t), T = boost::logic::tribool]
/opt/local/include/boost/logic/tribool.hpp:361: note:                 boost::logic::tribool boost::logic::operator==(boost::logic::tribool, bool (*)(boost::logic::tribool, boost::logic::detail::indeterminate_t))
/opt/local/include/boost/logic/tribool.hpp:355: note:                 boost::logic::tribool boost::logic::operator==(bool (*)(boost::logic::tribool, boost::logic::detail::indeterminate_t), boost::logic::tribool)
/opt/local/include/boost/logic/tribool.hpp:350: note:                 boost::logic::tribool boost::logic::operator==(bool, boost::logic::tribool)
/opt/local/include/boost/logic/tribool.hpp:345: note:                 boost::logic::tribool boost::logic::operator==(boost::logic::tribool, bool)
/opt/local/include/boost/logic/tribool.hpp:334: note:                 boost::logic::tribool boost::logic::operator==(boost::logic::tribool, boost::logic::tribool)
../../catch/internal/catch_capture.hpp: At global scope:
../../catch/internal/catch_capture.hpp: In instantiation of ‘Catch::Detail::IsStreamable<boost::logic::tribool>’:
../../catch/internal/catch_capture.hpp:96:   instantiated from ‘std::string Catch::toString(const T&) [with T = boost::logic::tribool]’
../../catch/internal/catch_capture.hpp:330:   instantiated from ‘Catch::MutableResultInfo& Catch::MutableResultInfo::captureExpression(const T1&, const T2&) [with Catch::Internal::Operator Op = IsEqualTo, T1 = boost::logic::tribool, T2 = bool]’
../../catch/internal/catch_capture.hpp:362:   instantiated from ‘Catch::MutableResultInfo& Catch::Expression<T>::operator==(const RhsT&) [with RhsT = bool, T = boost::logic::tribool]’
../../src/tests/article.cc:215:   instantiated from here
../../catch/internal/catch_capture.hpp:50: error: ambiguous overload for ‘operator<<’ in ‘Catch::Synth [with T = std::ostream&]() << Catch::Synth [with T = const boost::logic::tribool&]()’
/usr/include/c++/4.2.1/ostream:177: note: candidates are: std::basic_ostream<_CharT, _Traits>& std::basic_ostream<_CharT, _Traits>::operator<<(bool) [with _CharT = char, _Traits = std::char_traits<char>]
../../catch/internal/catch_capture.hpp:39: note:                 Catch::Detail::NonStreamable Catch::Detail::operator<<(std::ostream&, const Catch::Detail::NonStreamable&)

Does this mean it is necessary to write glue logic for all non-standard types? If so, could you add some documentation for doing that?

Just grabbed the latest version of the single-evaluation branch, and get a warning on line 95 in catch_debugger.hpp.

C4800: int: forcing value to bool 'true' or 'false' (performance warning)

The fix is just to compare the result of IsDebuggerPresent() against 0, rather than converting it directly to bool:

return IsDebuggerPresent() != 0;

When testing a function f() == true (that can only be called once), this test passes:

bool t = f();
REQUIRE(t == true);

this test fails:

REQUIRE(f() == true);

Seems the condition is evaluated twice? Perhaps this is by design, but it should probably be noted somewhere since it's not the usual case.

Approx() takes its arguments as doubles - which is fine, except when one of them is a float and it picks up the epsilon value of a double.

Taking the arguments generically, and using epsilon for float when the argument is a float, should address the issue - even if the values are then stored as doubles.

The TestCase pointer member recieves a clone of the incoming, but if it already pointed at something else it leaks.

Op= should be implemented in terms of copy & swap anyway.

Thanks to Sam Saariste for spotting this one.

If an exception is thrown outside of one of Catch's assertion macros the file/ line info is obviously not available for the throw site.
However, the file/line of the test case itself (or the section) should be available and these should be used instead.

Using current git for Catch and compiling using a fair amount of warning flags Catch still produces a lot of warnings:

g++ -I../catch -g -fPIC -I. -W -Wall -Wfloat-equal -Wundef -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion -Wsign-compare -Wmissing-noreturn -Wmissing-format-attribute -Wpacked -Winline -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Wreorder -Wold-style-cast -Woverloaded-virtual -ffor-scope -o runtests tests/body.cc
In file included from ../catch/internal/catch_hub.h:18,
                 from ../catch/internal/catch_hub_impl.hpp:12,
                 from ../catch/catch_runner.hpp:16,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/internal/catch_interfaces_reporter.h:26: warning: ‘struct Catch::IReporterConfig’ has virtual functions but non-virtual destructor
In file included from ../catch/internal/catch_test_registry.hpp:16,
                 from ../catch/internal/catch_test_case_registry_impl.hpp:13,
                 from ../catch/internal/catch_hub_impl.hpp:14,
                 from ../catch/catch_runner.hpp:16,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/internal/catch_interfaces_testcase.h:44: warning: ‘struct Catch::ITestCaseRegistry’ has virtual functions but non-virtual destructor
In file included from ../catch/internal/catch_test_case_registry_impl.hpp:13,
                 from ../catch/internal/catch_hub_impl.hpp:14,
                 from ../catch/catch_runner.hpp:16,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/internal/catch_test_registry.hpp: In constructor ‘Catch::MethodTestCase<C>::MethodTestCase(void (C::*)())’:
../catch/internal/catch_test_registry.hpp:25: warning: declaration of ‘method’ shadows a member of 'this'
In file included from ../catch/internal/catch_hub_impl.hpp:14,
                 from ../catch/catch_runner.hpp:16,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/internal/catch_test_case_registry_impl.hpp: At global scope:
../catch/internal/catch_test_case_registry_impl.hpp:24: warning: ‘class Catch::TestRegistry’ has virtual functions but non-virtual destructor
../catch/internal/catch_test_case_registry_impl.hpp: In constructor ‘Catch::FreeFunctionTestCase::FreeFunctionTestCase(void (*)())’:
../catch/internal/catch_test_case_registry_impl.hpp:61: warning: declaration of ‘fun’ shadows a member of 'this'
In file included from ../catch/internal/catch_runner_impl.hpp:15,
                 from ../catch/internal/catch_hub_impl.hpp:15,
                 from ../catch/catch_runner.hpp:16,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/internal/catch_interfaces_runner.h: At global scope:
../catch/internal/catch_interfaces_runner.h:20: warning: ‘struct Catch::IRunner’ has virtual functions but non-virtual destructor
In file included from ../catch/internal/catch_runner_impl.hpp:17,
                 from ../catch/internal/catch_hub_impl.hpp:15,
                 from ../catch/catch_runner.hpp:16,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/internal/catch_config.hpp:28: warning: ‘class Catch::Config’ has virtual functions but non-virtual destructor
../catch/internal/catch_config.hpp: In member function ‘Catch::Config::List::What Catch::Config::listWhat() const’:
../catch/internal/catch_config.hpp:153: warning: use of old-style cast
../catch/internal/catch_config.hpp: In member function ‘Catch::Config::List::What Catch::Config::listAs() const’:
../catch/internal/catch_config.hpp:159: warning: use of old-style cast
../catch/internal/catch_config.hpp: In member function ‘void Catch::Config::setShouldDebugBreak(bool)’:
../catch/internal/catch_config.hpp:170: warning: declaration of ‘shouldDebugBreak’ shadows a member of 'this'
../catch/internal/catch_config.hpp: In member function ‘void Catch::Config::setShowHelp(bool)’:
../catch/internal/catch_config.hpp:182: warning: declaration of ‘showHelp’ shadows a member of 'this'
In file included from ../catch/internal/catch_hub_impl.hpp:17,
                 from ../catch/catch_runner.hpp:16,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/internal/catch_stream.hpp: In member function ‘int Catch::StreamBufImpl<WriterF, bufferSize>::overflow(int)’:
../catch/internal/catch_stream.hpp:44: warning: use of old-style cast
In file included from ../catch/catch_runner.hpp:18,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/internal/catch_commandline.hpp: In member function ‘void Catch::ArgParser::changeMode(const std::string&, Catch::ArgParser::Mode)’:
../catch/internal/catch_commandline.hpp:128: warning: use of old-style cast
../catch/internal/catch_commandline.hpp:130: warning: use of old-style cast
../catch/internal/catch_commandline.hpp:134: warning: use of old-style cast
In file included from ../catch/catch_runner.hpp:22,
                 from ../catch/catch_with_main.hpp:15,
                 from tests/body.cc:1:
../catch/catch_reporter_junit.hpp: In constructor ‘Catch::JunitReporter::TestCaseStats::TestCaseStats(const std::string&)’:
../catch/catch_reporter_junit.hpp:35: warning: declaration of ‘name’ shadows a member of 'this'
../catch/catch_reporter_junit.hpp: In constructor ‘Catch::JunitReporter::Stats::Stats(const std::string&)’:
../catch/catch_reporter_junit.hpp:49: warning: declaration of ‘name’ shadows a member of 'this'

The NULL is deduced as an int in a template context. So we need at least this overload to make it work:

///////////////////////////////////////////////////////////////////////////
template<Internal::Operator Op, typename T>
MutableResultInfo& captureExpression
(
    const T* lhs, 
    int rhs
)
{
    setResultType( Internal::compare<Op>( lhs, reinterpret_cast<const T*>( rhs ) ) ? ResultWas::Ok : ResultWas::ExpressionFailed );
    m_lhs = Catch::toString( lhs );
    m_rhs = Catch::toString( rhs );
    m_op = Internal::OperatorTraits<Op>::getName();
    return *this;
}

From Chris Jefferson:

• catch_runnerconfig.hpp includes catch_reporter_registry.hpp, rather than "internal/catch_reporter_registry.hpp"
• internal/catch_commandline.hpp inclues catch_runnerconfig.hpp, rather than "../catch_runnerconfig.hpp"

See this example:

[the current version runs "1.2" in the same pass as "1.3", which is not right]

define CATCH_CONFIG_MAIN

include "catch.hpp"

include

//#ifdef SETCIONS
std::vectorstd::string g_testlog;
bool g_done(false);

std::string make_result(const std::vectorstd::string& v)
{
std::string result;
for( std::vectorstd::string::const_iterator i = v.begin();
i != v.end(); ++i )
{
if( !result.empty() ) result += " + ";
result += *i;
}
return result;
}

TEST_CASE("1/1", "Test vectors")
{
g_testlog.push_back("start");
SECTION("1", "")
{
g_testlog.push_back("1");
SECTION("1.1", "")
{
g_testlog.push_back("1.1");
CHECK(g_testlog.size() == 3); // start + 1 + 1.1
CHECK(!g_done);
}
SECTION("1.2", "")
{
g_testlog.push_back("1.2");
SECTION("1.2.1", "")
{
g_testlog.push_back("1.2.1");
// start + 1 + 1.1 + end
// + start + 1 + 1.2 + 1.2.1
CHECK(g_testlog.size() == 8);
CHECK(!g_done);
}
SECTION("1.2.2", "")
{
g_testlog.push_back("1.2.2");
// start + 1 + 1.1 + end
// + start + 1 + 1.2 + 1.2.1 + end
// + start + 1 + 1.2 + 1.2.2
CHECK(g_testlog.size() == 13);
CHECK(!g_done);
}
}
SECTION("1.3", "")
{
g_testlog.push_back("1.3");
SECTION("1.3.1", "")
{
g_testlog.push_back("1.3.1");
// start + 1 + 1.1 + end
// + start + 1 + 1.2 + 1.2.1 + end
// + start + 1 + 1.2 + 1.2.2 + end
// + start + 1 + 1.3 + 1.3.1
CHECK(g_testlog.size() == 18);
CHECK(!g_done);
}
SECTION("1.3.2", "")
{
g_testlog.push_back("1.3.2");
// start + 1 + 1.1 + end
// + start + 1 + 1.2 + 1.2.1 + end
// + start + 1 + 1.2 + 1.2.2 + end
// + start + 1 + 1.3 + 1.3.1 + end
// + start + 1 + 1.3 + 1.3.2
CHECK(g_testlog.size() == 23);
CHECK(!g_done);
}
}
}
SECTION("2", "")
{
g_testlog.push_back("2");
SECTION("2.1", "")
{
g_testlog.push_back("2.1");
CHECK(!g_done);
SECTION("2.1.1", "")
{
g_testlog.push_back("2.1.1");
// start + 1 + 1.1 + end
// + start + 1 + 1.2 + 1.2.1 + end
// + start + 1 + 1.2 + 1.2.2 + end
// + start + 1 + 1.3 + 1.3.1 + end
// + start + 1 + 1.3 + 1.3.2 + end
// + start + 2 + 2.1 + 2.1.1
CHECK(g_testlog.size() == 28);
CHECK(!g_done);
}
SECTION("2.1.2", "")
{
g_testlog.push_back("2.1.2");
// start + 1 + 1.1 + end
// + start + 1 + 1.2 + 1.2.1 + end
// + start + 1 + 1.2 + 1.2.2 + end
// + start + 1 + 1.3 + 1.3.1 + end
// + start + 1 + 1.3 + 1.3.2 + end
// + start + 2 + 2.1 + 2.1.1 + end
// + start + 2 + 2.1 + 2.1.2
CHECK(g_testlog.size() == 33);
CHECK(!g_done);
}
}
}
SECTION("3", "")
{
g_testlog.push_back("3");
SECTION("3.1", "")
{
g_testlog.push_back("3.1");
// start + 1 + 1.1 + end
// + start + 1 + 1.2 + 1.2.1 + end
// + start + 1 + 1.2 + 1.2.2 + end
// + start + 1 + 1.3 + 1.3.1 + end
// + start + 1 + 1.3 + 1.3.2 + end
// + start + 2 + 2.1 + 2.1.1 + end
// + start + 2 + 2.1 + 2.1.2 + end
// + start + 3 + 3.1
CHECK(g_testlog.size() == 37);
CHECK(!g_done);
}
}
SECTION("4", "end of tests")
{
g_testlog.push_back("4");
// start + 1 + 1.1 + end
// + start + 1 + 1.2 + 1.2.1 + end
// + start + 1 + 1.2 + 1.2.2 + end
// + start + 1 + 1.3 + 1.3.1 + end
// + start + 1 + 1.3 + 1.3.2 + end
// + start + 2 + 2.1 + 2.1.1 + end
// + start + 2 + 2.1 + 2.1.2 + end
// + start + 3 + 3.1 + end
// + start + 4
CHECK(g_testlog.size() == 40);
CHECK(!g_done);
g_done = true;
}
g_testlog.push_back("end");

if( g_done )
{
    std::string expected("start + 1 + 1.1 + end"
    " + start + 1 + 1.2 + 1.2.1 + end"
    " + start + 1 + 1.2 + 1.2.2 + end"
    " + start + 1 + 1.3 + 1.3.1 + end"
    " + start + 1 + 1.3 + 1.3.2 + end"
    " + start + 2 + 2.1 + 2.1.1 + end"
    " + start + 2 + 2.1 + 2.1.2 + end"
    " + start + 3 + 3.1 + end"
    " + start + 4 + end");
    REQUIRE(g_testlog.size() == 41);
    std::string actual = make_result(g_testlog);
    REQUIRE( expected == actual );
}

}

Some ambiguities in the field

Seems to make sense

I'm trying to upgrade a project to the current (single-include) version of Catch, but am running into a compile error:

../../src/tests/catch.hpp:1063:67: error: ISO C++ forbids comparison between pointer and integer

The relevant catch.hpp section is this:

    template<typename T1, typename T2>
    struct Evaluator<T1, T2, IsNotEqualTo>
    {
        static bool evaluate( const T1& lhs, const T2& rhs )
        {
            return const_cast<T1&>( lhs ) != const_cast<T2&>( rhs );
        }
    };

The code which triggers this is a very simple test:

Body body;
REQUIRE(body.figureSummary()==NULL);

where figureSummary is a method returning const char*

Does something like this exist? Or would it be possible to add?

Suppose I need to run the same set of tests on a number of different types, effectively templating the tests on a type parameter.

I'm not sure what would be an appropriate syntax for it (Boost.Test uses Boost.MPL to create type lists, but adding such a dependency might be awkward, and it requires you to be comfortable with metaprogramming just to define your tests)

Perhaps something like this?

// indicate I want to define tests that can be templated on a number of different types
TEST_TEMPLATE()
{
    // specify a test case for some type T
    TEST_CASE_T(name, description, T)
    {
        // inside the test case, I can use T as a placeholder for the specific type
        std::vector<T> vec;
        REQUIRE(vec.size() == 0);
    }
    // Register the types I want to run the tests for
    FOR_TYPE(int);
    FOR_TYPE(float)
}

Or perhaps you can come up with a better syntax (or even better, the feature already exists ;))
In any case, the feature would be very useful.

std::string is not quoted, char* is single quoted - yet string literals captured as part of the expression capture show double quotes.

All strings should use double quotes.

I think this is just a case of changing the following overloads:

///////////////////////////////////////////////////////////////////////////////
inline std::string toString
(
const std::string& value
)
{
return """ + value + """;
}
///////////////////////////////////////////////////////////////////////////////
inline std::string toString
(
const char* const value
)
{
return value ? toString( std::string( value ) ) : std::string( "{null string}" );
}

(Reporting this from work where I don't have git access - hencing using the ticketing system instead)

Files inside 'include/reporters/' have lines like this:

include "internal/catch_capture.hpp"

But the folder 'internal' is in the parent folder:

include "../internal/catch_capture.hpp"

Thanks.

if no << overload or toString() specialisation are available for a type that is being logged it should use some generic method

The error produced is quite lengthy, but below is a snipet.

In file included from tests.cc:19:
../include/catch.hpp: In member function ‘virtual void Catch::TestRegistry::registerTest(const Catch::TestCaseInfo&)’:
../include/catch.hpp:3108: error: invalid use of incomplete type ‘const struct Catch::TestCaseInfo’
../include/catch.hpp:176: error: forward declaration of ‘const struct Catch::TestCaseInfo’
../include/catch.hpp:3111: error: invalid use of incomplete type ‘const struct Catch::TestCaseInfo’
../include/catch.hpp:176: error: forward declaration of ‘const struct Catch::TestCaseInfo’
../include/catch.hpp:3112: error: invalid use of incomplete type ‘struct Catch::TestCaseInfo’
../include/catch.hpp:176: error: forward declaration of ‘struct Catch::TestCaseInfo’
../include/catch.hpp:3112: error: return-statement with a value, in function returning 'void'

The error exists from commit 8350ef to 83224e (current HEAD). This seems to be a side-effect of the fix for Issue #47

To capture that what() message:

catch( std::exception& ex ) \
{ \
    INTERNAL_CATCH_ACCEPT_EXPR( ( Catch::ResultBuilder( __FILE__, __LINE__, macroName, #expr ) << ex.what() ).setResultType( Catch::ResultWas::ThrewException ), stopOnFailure ); \
} \

If a class provides an operator overload, typically ==, as a non-const method this will not be used by Catch if the operator is relied on in the assertion comparison. e.g.:

struct Test
{
  Test( unsigned int v ) : m_value(v) {}

  bool operator==( const Test&rhs )
  { 
    return (m_value == rhs.m_value);
  }
  unsigned int m_value;
};

TEST_CASE("example/non-const==", "Demonstrate that a non-const == is not used")
{
    Test t( 1 );

    bool ok = (t == 1); // (1)
    REQUIRE(ok);    
    REQUIRE( t == 1 );  // (2)
}

In (2) the == operator is not found at all (yet it is in (1)) - leading to a compiler error. Alternative problems are that both const and non-const overloads are provided but the const overload is selected for a non-const object - possibly resulting in different behaviour (although it's hard to think of a case where that this would be justified). Also, if the method takes it's argument by non-const ref, or, again, provides overloads.

This is because the operands are being conveyed through several layers as const-ref arguments.
This could be fixed by providing non-const overloads, but this would involve 3-4x as much code.

It is debatable whether non-const operator methods are good practice, although it is understood that the user may not always have the choice (if they don't own the code in question, for example).

I'm leaning towards const-casting the references when invoking the operators. This has two potential issues, AFAICS:

1. We might still select the wrong overload if both are provided (for either operand) - but this is no worse than the current case, and it's debatable whether this should matter.
2. This is perhaps the same issue, really. If const and non-const overloads are provided and the non-const overload modifies itself or its operand (as applicable), but the test actually passed a const object down, this could result in incorrect side-effects. Again I'm struggling to think of a case where this would really be done, but nonetheless the issue is there.

Comments or criticisms welcome.

I am seeing an odd segfault when writing a test for some lua glue code. I am using a tiny lua state wrapper:

#include <lua.hpp>
class Lua {
public:
    Lua() {
        L=luaL_newstate();
    }

    ~Lua() {
        lua_close(L);
    }

    operator lua_State*() {
        return L;
    }

protected:
    lua_State   *L;
};

To test this code I wrote a very simple test:

#include <catch_with_main.hpp>
#include "lua.hh"

TEST_CASE("lua-wrapper", "Lua integration") {
    stylist::detail::Lua lua;
    lua_createtable(lua, 0, 1);
    lua_pushnil(lua);
    REQUIRE(lua_gettop(lua)==2);
    REQUIRE(lua_istable(lua, -2));
    REQUIRE(lua_next(lua, -2)==0);
}

lua.hh is my include file which defines the Lua class. When running this I get a segfault. When I rewrite this test as a simple main function without using test I do not get a segfault:

#include <cassert>
#include "lua.hh"

int main() {
    stylist::detail::Lua lua;
    lua_createtable(lua, 0, 1);
    lua_pushnil(lua);
    assert(lua_gettop(lua)==2);
    assert(lua_istable(lua, -2));
    assert(lua_next(lua, -2)==0);
    return 0;
}

Which suggests that somewhere catch causes memory corruption or does something else that breaks lua. Unfortuantely valgrind was not able to find anything weird, so I'm hoping you have some ideas.

I'm having a rather strange problem with REQUIRE:

[Running: waitSet/waitAny]
testWaitSet.cpp(184): waitSet.waitAny(0, waitHandle) == WaitSuccess failed for: 0 == 0
[Finished: waitSet/waitAny 2 test(s) passed but 1 test(s) failed]

The line in question is nothing complicated:
REQUIRE(waitSet.waitAny(0, waitHandle) == WaitSuccess);

I tried adding another set of parentheses and switching the lhs and rhs, but it won't pass. Instead, it worked fine if I replaced it with:
if(waitSet.waitAny(0, waitHandle) != WaitSuccess)
REQUIRE(false);

The identifiers are defined as follows:
typedef int Handle;
enum WaitResult
{
WaitSuccess = 0,
WaitAbandoned = -1,
WaitTimeout = -2
};
WaitResult waitAny(uint32_t timeout, Handle& handle);

This is running in Linux with gcc as the compiler.
You can see the full context at https://github.com/Tryneus/Abstraction/blob/master/libCommon/test/testWaitSet.cpp

When compiled under MSVC, the code fails if windows.h is included.

That header defines a function DebugBreak() while CATCH checks for a macro with the same name.

On a related note, on MSVC, the intrinsic __debugbreak() should be preferred over DebugBreak() (the latter requires windows.h, and requires debug symbols or it'll break into the wrong stack frame)

Here's a quick MSVC DebugBreak() replacement I whipped up:

#if defined(_MSC_VER)
    extern "C" __declspec(dllimport) int __stdcall IsDebuggerPresent();
    #define DebugBreak() if (IsDebuggerPresent() ) { __debugbreak(); }
#endif

(Still kind of ugly to reuse the name DebugBreak though. Would be cleaner to rename it entirely to avoid conflicting with the windows.h one)

I am not sure if this is by design, but if I have a TEST_CASE that as a few tests, let's say 5, and let's assume test 3,4,5 will fail. Catch only reports that test 3 failed, and does not go over the test 4 and 5. I did notice that each TEST_CASE can fail independently which is as I expected/wanted. I would really prefer that Catch also goes over all the REQUIREs in a TEST_CASE rather than giving up after the first REQUIRE that fails.

Catch appears to be using local variables with very obvious names, making it likely to get conflicts. I ran into this in a test which has a local variable called action. This resulted in the compiler producing warning: declaration of ‘action’ shadows a previous local, and indeed my tests were failing as well.

I would suggest to use a prefix for any variables injected by Catch that is not likely to conflict. Perhaps something as simple as _catch_ ?

It should be possible to obtain xml reports that follow the schema for JUnit - for consumption by third party tools

stdout (and stderr) are redirected (internally to an ostringstream), so they can be reported later as part of the test results.
Sometimes it is desirable to see the output immediately - especially while debugging.

A command line switch could be provided to allow this option.

I'm having trouble compiling Catch with objective-c(++) on OS X 10.6.

Error:
catch.hpp: In function 'size_t Catch::registerTestMethods()': catch.hpp:6186: error: invalid use of incomplete type 'struct Catch::TestCaseInfo' catch.hpp:176: error: forward declaration of 'struct Catch::TestCaseInfo'

Minimal example:

#import<Foundation/Foundation.h>
#import "catch.hpp"
int main (void) {
    return 0;
}

Compiled with gcc 4.2.1:
gcc -o test main.mm -m32 -framework Foundation -lobjc

Thanks,
Jurnell