Isn't Method.Iterate() too coarse-grained? about optimize

[btracey]

Here's how it should be. xNext says where to evalute, EvalutaionType says how to evaluate it, and IterationType essentially captures the meta-information about the function evalutaion. It's supposed to be information about the upcoming function information, and not information about the previous information. I think a single function evaluation is the right way to go. In optimize's worth vision, an optimize.Method states the next evaluation it wants to see to work toward a (local) minimum. Directly, this is {xNext, EvalutionType}, and IterationType is meta-information to help out the rest of the process. Two different functions seems like it would just create complicated state.

This mindset does have a problem in the case of linesearch. In a linesearch, it ends that the last location in the minor iteration is also the initial location in the major evaluation. However, until you've seen the function value, you don't actually know it's true. I imagine this is the issue with my interpretation of IterationType (a description of the upcoming evaluation), and your interpretation (a description of what just happened). A way to fix this is to have an EvalutaionType that is NoEvaluation (name tbd). The Linesarch struct is performing the linsearch, sending new locations and MinorIteration. It concludes successfully (the previous point having been a MinorIteration), so now Linesearch sends a MajorIteration, as it would be. However, Linesearch doesn't actually need more information at that location; it already has it from the linesearch evaluation, so it sends the same x location and a NoEvaluation. Thus, checkConvergence() and Recorder are properly signaled when the MajorIteration started, but we don't waste additional function evaluations by evaluating things twice.

[vladimir-ch]

Here's how it should be. xNext says where to evalute, EvalutaionType says how to evaluate it, and IterationType essentially captures the meta-information about the function evalutaion. It's supposed to be information about the upcoming function information, and not information about the previous information.

I agree that it would be natural if it were like this, but at the moment this is definitely not the case, and it is not a matter of interpretation. Right now what the code does is that IterationType refers to what just happened, EvaluationType refers to the upcoming evaluation. We need to fix this, which means modifying Linesearch, Local() and minimize() so that they play well together. It will also help us learn how non-Linesearch minimizers should behave.

I think a single function evaluation is the right way to go. Two different functions seems like it would just create complicated state.

I agree, that's why I put this as a question, not as an issue.

A way to fix this is to have an EvalutaionType that is NoEvaluation (name tbd).

We already have NoEvaluation.

The Linesarch struct is performing the linsearch, sending new locations and MinorIteration. It concludes successfully (the previous point having been a MinorIteration), so now Linesearch sends a MajorIteration, as it would be. However, Linesearch doesn't actually need more information at that location; it already has it from the linesearch evaluation, so it sends the same x location and a NoEvaluation. Thus, checkConvergence() and Recorder are properly signaled when the MajorIteration started, but we don't waste additional function evaluations by evaluating things twice.

Good idea. Linesearch checks if the linesearch has concluded, it finds out that yes, it has concluded, and returns the reached location in xNext and MajorIteration, NoEvaluation. checkConvergence() and Record() are called, evaluate() does not evaluate anything, and Iterate() is called again. This time new direction is generated, xNext updated, and MinorIteration, xxxEvaluation returned. Sounds good.

[btracey]

We already have NoEvaluation.

Oops. Sorry, should have re-read the code.
The Linesarch struct is performing the linsearch, sending new locations and MinorIteration. It concludes successfully (the previous point having been a MinorIteration), so now Linesearch sends a MajorIteration, as it would be. However, Linesearch doesn't actually need more information at that location; it already has it from the linesearch evaluation, so it sends the same x location and a NoEvaluation. Thus, checkConvergence() and Recorder are properly signaled when the MajorIteration started, but we don't waste additional function evaluations by evaluating things twice.

Good idea. Linesearch checks if the linesearch has concluded, it finds out that yes, it has concluded, and returns the reached location in xNext and MajorIteration, NoEvaluation. checkConvergence() and Record() are called, evaluate() does not evaluate anything, and Iterate() is called again. This time new direction is generated, xNext updated, and MinorIteration, xxxEvaluation returned. Sounds good.

SGTM.

[vladimir-ch]

Resolved by 1ede5c9.

Closing.