optimize: allow constrained optimization about optimize

[btracey]

Not at the moment, though it's on the roadmap. For now, you can use a barrier function as a workaround.

thanks.

@btracey Am considering working on this. Should the bounds go in optimize.problem or optimize.settings?

[sbinet]

@btracey I have also a student working on an optimize.problem (via go-hep.org/x/hep/fit) that would require adding constrains to the optimization process.
any pointer to what you were considering as an implementation?

[btracey]

@vladimir-ch should be brought into this as well.

The spirit of what we should have is

type Problem struct {
    Func func(x []float64) float64
    Grad func(grad []float64, x []float64)
    Hess func(hess mat64.MutableSymmetric, x []float64)
    Status func() (Status, error)

    Bounds []Bound
    Eq []func(x []float64) float64   // Equality constraints that f(x) = 0
    Ineq []func(x []float64) float64 // Inequality constraints that f(x) <= 0
}

However, this doesn't quite work, because we want to support constraints where we also know the gradient and the hessian. So we really want something like

type Function struct {
    Func func(x []float64) float64
    Grad func(grad []float64, x []float64)
    Hess func(hess mat64.MutableSymmetric, x []float64)
}

type Problem struct{
    Status func() (Status, error)
    
    Objective Function
    Bounds []Bound
    Eq []Function   // Equality constraints that f(x) = 0
    Ineq []Function // Inequality constraints that f(x) <= 0
}

I think that's how we would want to change the representation.

It's not hard to support something to handle constrained optimization. We can have a Barrier optimizer that uses a barrier/penalty method to convert the constrained problem into an unconstrained problem, and uses one of our unconstrained optimizers.

It is hard to support a real gradient-based optimizer. The typical algorithm is an SQP method (like SLSQP), which requires a QP solver, which requires an LP solver. This is why optimize/convex/lp exists, but I haven't found the desire to do all of the cases of QP yet (unconstrained, only equality, only inequality, only bounded, any combination). I've started looking into this a few times, but it'll require factoring out large parts of the Simplex solver in order to find initial feasible points. Once QP exists, then we can figure out how to do a real SQP solver.

Rewriting SLSQP into a better solver was one of my main goals at the start of Gonum, and yet here we are...

[vladimir-ch]

Closing because this repository is no longer maintained. Development has moved to https://github.com/gonum/gonum.

The follow-up issue is gonum/gonum#354