which is the right way to setup number of epochs? [NoobQuestion#2] about mlpack

[rcurtin]

The loss prediction that the ProgressBar is showing you is necessarily something of an estimate. If we want to compute an exact loss value, we need to actually compute predictions with the current state of the neural network on every point in in---and that can be a large amount of computation! So, we don't do this, and instead use the loss just on each minibatch of training points as the progress bar estimate (all other toolkits do something like this too). This means that the first minibatch loss estimate has a lot of variance, and then as we add estimates for other minibatches, it slowly converges to the full-data loss (but even then it is still something of an estimate). This explains the "jumpy" behavior you see at the beginning of the epoch.

The optimizer isn't actually being reset between iterations, and the model isn't getting worse---it's just a matter of the approximation of how we can efficiently get some kind of loss estimate.

One experiment you could do to see this is use the PrintLoss() callback instead of ProgressBar(). If you run for 10 epochs, this will print 10 loss values. Then, compare this with what happens if you use an adaptation of your single-epoch training loop:

int epochs = 1000;
for (int e = 0; e < epochs; e++)
{
  double loss = Train(in, out, opt);
  std::cout << loss << std::endl;
}

and this should print very similar numbers (they may not be exactly the same due to randomness in shuffling the data and other small effects).

[IgorWojas]

@rcurtin Thank you for thorough explanation!!! I have tried something similar to the code you proposed but i couldn't understand the slight differences in values. Now everything is clear. Sooo Resolved!!!

Thanks again
Cheers!!!