v2.0.0, OS X 10.10: All benchmarks are run for only one iteration about celero

[DigitalInBlue]

I just tested v2.0.0 by instrumenting celeroDemoTransform using Visual Studio 2012 x64 Release. The iteration count seems to be correct for both Baselines and Benchmarks. Can you please verify on your end and also report what your compiler/platform is. If you have code that demonstrates the error, please post that as well.

[sgraf812]

OS X 10.10, for the following clang version:

Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.4.0
Thread model: posix

It happens when running the celeroDemoSimple for example, where iteration count is constantly 1. This doesn't happen for v1.2.0 which I'm currently using. I'd love to help, but am a little unmotivated to hunt for this without some pointers.

It definitely has to do with every sample just doing one operation instead of the passed amount (found that out through printf-debugging). So, while the averaging logic is probably correct, there are not enough iterations.

On a side note: there's also an error regarding usage of std::less<> without arguments which is C++14, but I already fixed that in my branch for this issue.

[DigitalInBlue]

Ok. Thanks. I assume this should also be a problem with the MSVC version I am running, so I'll dig into it and get it sorted out.

[sgraf812]

Oh, I just noticed that for celeroDemoTransform and various others there is no problem, e.g. more than one iteration was performed. I mistakenly assumed that when I was able to reproduce it for celeroDemoSimple that it was the general case.

Edit: It seems to work for BENCHMARK_F, but not for BENCHMARK which implies something may be wrong with TestFixture.

[DigitalInBlue]

The update to v2.0.0 improved the way experiment values and units were implemented. However, there was an oversight in that if these more complex features were not used, only a single iteration of the code under test was executed.

Minor documentation updates were also performed.

[sgraf812]

👍