Equal equality failing in REQUIRE about catch2

[Tryneus]

I think the issue may be that the REQUIRE macro ends up evaluating the expression twice. One of the calls is failing, and one of the calls is successful.

I made a simple test to check this, and I get the same behavior:

#include "catch_with_main.hpp"
TEST_CASE("catch/require", "Test if require evaluates expressions twice")
{
  int i = 1;
  REQUIRE(--i == 0);
}

[Tryneus]

Looked into this a little more, the two times the expression is evaluated appears to be (the first time) at the call to

template<typename T> ResultBuilder& ResultBuilder::operator->*(const T & operand)

The second time would be the call to

virtual ResultAction::Value Runner::acceptResult(bool Result)

A fix for this isn't immediately obvious to me. I would consider the 'acceptResult' call to be extraneous if it would be possible to evaluate the result solely in the ResultBuilder. This would require a few changes to how the ResultBuilder parses expressions though, it would have to save the result data as well as the stringized version of each operand, then evaluate it once the expression is complete.

I noticed that at the moment ResultBuilder doesn't parse past a binary operator such as && or ||, and I think the fix to both of these things could be combined together.

[philsquared]

Hi Tryneus.

It is indeed the double evaluation issue.
It's been on my list for a while to come back and look at.
You're the second person to hit this within a week - so I'll take that as a prompt to work on it next ;-)

As for how it can be worked around I have two ideas:

1. The first is along the lines you suggested too. The ResultBuilder can capture the native values as well as the stringised ones - then recombined into the original comparison once it has both sides. There were a few reasons why I didn't do it this way in the first place - but I think the repeated evaluation side-effect is more pernicious.
2. An alternative I have toyed with the idea of is to run the whole test case for a second pass in the event of failure. The first pass would evaluate the expressions and the second pass would capture the decomposed parts. I'm not sure if this will introduce non-trivial overhead in some cases, have other side-effects or problems of its own.

Either way, looking at what you're doing suggests you're testing multi-threaded code. Is that the case? I don't currently support multiple-threads explicitly - although if the test code is confined to the calling thread there shouldn't be any issues.

[Tryneus]

I am indeed working with multi-threaded stuff, but when I was using CppUnit on a previous project, I learned to leave test framework calls out of the children threads. If there is a problem in the thread, it will throw an exception which is captured into a string that can be checked later in the main thread.

I've been investigating a fix for the multiple evaluation behavior, I'll try uploading it on a fork if I get it working. It's basically your alternative 1. Also to correct what I said earlier, I now realize that due to operator precedence, it isn't really possible to parse past && or ||.

I suppose it's too early to describe the changes I'm making because I'm not entirely sure if it will work, but I'll update this once I know.