[question] MajorIterations settings about optimize

[btracey]

The direct setting you are looking for is settings.FuncEvaluations. The optimization will stop after 5 function evaluations.

The concept of "Iteration" is not synonymous with function evaluation, and is a more nebulous term. We have taken whatever the view is of the optimizer. In the quasi-Newton methods, an "Iteration" is "choose a direction and perform a linesearch" this often takes more than one function evaluation.

It sounds like you are just supplying Func and not Grad. At present, you will thus be using Nelder-Mead. It requires dim+1 function evaluations to even construct the original simplex, and sometimes an iteration of nelder mead (defined as updating the simplex size) requires dim+1 evaluations. This is how you could find such a big disparity between them.

In short, setting FuncEvaluations is likely what you want, but know that if your limit is as low as 5 you might not get very far without supplying a gradient.

[milosgajdos]

Thanks for the excellent answer! The outcome did seem a bit "suspicious" to me :-)

The results I've mentioned here actually did use gradient as well. The particular optimization method I have used was optimize.BFGS{}

Thanks a lot for you explanation, it now makes more sense to me!

[btracey]

Could you test if FuncEvaluations is doing what you think it should? I'm surprised with BFGS it's taking 100 function evaluations in 5 iterations. Perhaps the problem isn't very well scaled?

[milosgajdos]

Yes FuncEvaluations does seem to work as you described.

And again, yes, the problem is defo not scaled - if by scaling you mean some kind of normalization to some unit sizes. I tested this on some hand-crafted data with no normalization performed.

Thanks for your answers!