Relax tolerance in test_exact_transient_solution_cpp about cap

[dalg24]

I was thinking we bump it up to 1e-5 percent and may be check that absolute difference is less than 1 microvolt

[dalg24]

On jenkins

GIT_COMMIT_HASH=6bfffde034c545d56c450e1e43316966a7fbc1be
GIT_REMOTE_URL=https://github.com/ORNL-CEES/Cap
[...]
Total Test time (real) =  27.89 sec

The following tests FAILED:
     11 - test_exact_transient_solution_cpp_1_procs (Failed)
Errors while running CTest

[Rombur]

I use a percent tolerance of 1e-3.

[dalg24]

Running 1 test case...
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{4.29972e-06} between computed_voltage{0.17259069351540293} and gold_solution[pos]{0.17259143560676579} exceeds 1e-05%
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{2.04154e-06} between computed_voltage{0.18593225566221308} and gold_solution[pos]{0.18593263524951811} exceeds 1e-05%
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{2.36768e-06} between computed_voltage{0.19059739274508819} and gold_solution[pos]{0.19059784401880361} exceeds 1e-05%
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.16667e-06} between computed_voltage{0.19460198487156083} and gold_solution[pos]{0.1946022119085378} exceeds 1e-05%
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{3.79018e-06} between computed_voltage{0.19815937216959012} and gold_solution[pos]{0.19816012322872489} exceeds 1e-05%
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{6.66338e-07} between computed_voltage{0.20139353082879491} and gold_solution[pos]{0.2013936650249285} exceeds 1e-05%
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.78799e-06} between computed_voltage{0.20438036420913069} and gold_solution[pos]{0.20438072963998949} exceeds 1e-05%
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{2.34171e-06} between computed_voltage{0.20716968626261728} and gold_solution[pos]{0.2071701713934283} exceeds 1e-05%
/home/jenkins/workspace/Bar/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{8.27384e-07} between computed_voltage{0.20979775467086972} and gold_solution[pos]{0.20979792825420379} exceeds 1e-05%

*** 9 failures are detected in the test module "ExactTransientSolution"

[dalg24]

it's insane it shouldn't have passed the BOOST_CHECK_SMALL with 1microvolt of tolerance...

[dalg24]

copy/paste from discussion on PR #123

test_exact_transient_solution_cpp_1_procs" start time: Apr 21 09:31 EDT

Output:

Running 1 test case...
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{2.36184e-06} between computed_voltage{0.17259184323958501} and gold_solution[pos]{0.17259143560676579} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.74836e-07} between computed_voltage{0.18020259512043804} and gold_solution[pos]{0.18020256361459411} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.10821e-06} between computed_voltage{0.18593242919793684} and gold_solution[pos]{0.18593263524951811} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{9.04486e-07} between computed_voltage{0.1905976716258401} and gold_solution[pos]{0.19059784401880361} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.15416e-07} between computed_voltage{0.19460223436876695} and gold_solution[pos]{0.1946022119085378} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{2.77022e-06} between computed_voltage{0.19815957428217496} and gold_solution[pos]{0.19816012322872489} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.16931e-06} between computed_voltage{0.20139390051686604} and gold_solution[pos]{0.2013936650249285} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.39484e-06} between computed_voltage{0.20438101471750267} and gold_solution[pos]{0.20438072963998949} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.36566e-06} between computed_voltage{0.20717045431816833} and gold_solution[pos]{0.2071701713934283} exceeds 1e-05%
/home/bt2/Documents/Cap/cpp/test/test_exact_transient_solution.cc(68): error: in "test_exact_transient_solution": difference{1.02468e-06} between computed_voltage{0.20979771327799024} and gold_solution[pos]{0.20979792825420379} exceeds 1e-05%

[dalg24]

How does 1e-6 exceed 1e-5?

[dalg24]

It is a total non sense, the test that should be failing is not the BOOST_CHECK_CLOSE but the BOOST_CHECK_SMALL...

[Rombur]

1e-6 > 1e-5% = 1e-7

[dalg24]

What is the commit hash?

[dalg24]

Oh yes percentage you are right

[dalg24]

But shouldn't the BOOST_CHECK_SMALL fail as well?

[dalg24]

OK I get it now.
Let's relax to 1e-3%

[dalg24]

The issue is addressed in PR #119 which should be merged shortly