Training error - huge difference (encog 3.1.0) about encog-dotnet-core

BTW: It is not a problem to reproduce the same error. It is enought to run the sample code 100x times and you can get the same problem.

I probably know, where is the problem, I used this (basic) code
...
do
{
train.Iteration();
Console.WriteLine("Epoch #" + epoch + " Error:" + Format.FormatPercent(train.Error));
Console.WriteLine("Evaluated error: " + Format.FormatPercent(network.CalculateError(trainingSet)));
epoch++;
} while (train.Error > 0.001);
...

and I got these outputs:
...
Epoch #35 Error:0,125270%
Evaluated error: 0,058253%
Epoch #36 Error:0,058253%
Evaluated error: 22,434797%
Final evaluated error: 22,434797%
Neural Network Results:
0,0, actual=0,023279943378334,ideal=0
1,0, actual=0,351834582261846,ideal=1
0,1, actual=0,309542538954124,ideal=1
1,1, actual=7,84406348522279E-05,ideal=0

The problem is, that in some situations the calculation of train.Error and network.CalculateError can generate huge difference as in my sample 0,058253% (train.Error) vs 22,434797% (CalculateError).

I didn't have problem with topic huge difference in situation when I used cycle with "while (network.CalculateError(trainingSet) > 0.001);". It takes more time for training, but output of training is corrent in all situations (it would be fine to have final solution not this work-around).

BTW: This problem is also in Java code, I tested C# and Java also.

[jeffheaton]

This is the way that the training code is designed. train.Error is the error at the beginning of a training iteration (before weights are updated), whereas CalculateError is the error AFTER an iteration. They will always move sort of lockstep like you have there. Your results above seem to follow this, as epoch 35's evaluated error becomes the regular error for epoch 36, same thing on 36 to the final.

More info here:

http://www.jeffheaton.com/2014/03/when-is-a-models-training-error-calculated/

Also, sometimes, the random weights will produce a network that cannot be trained for XOR. If it takes 100 or so runs to see a large difference, you might be seeing that case.

[XeonMan]

Why did Epoch#36 jump from train.Error:0,058253% to a whopping Evaluated error: 22,434797%, whereas Epoch#35 decreased from train.Error:0,125270% to Evaluated error: 0,058253%, which is to be more or so expected?