pbenner / autodiff

Autodiff is a numerical library for the Go programming language that supports automatic differentiation. It implements routines for linear algebra (vector/matrix operations), numerical optimization and statistics

Personal Website

autodiff's Issues

Is there any reason that there is no concrete version of math operation on, say `DenseReal64Vector` ?

On the doc, it says

Methods, such as VaddV and MaddM, are generic and accept vector or matrix types that implement the respective ConstVector or ConstMatrix interface.
However, opertions on interface types are much slower than on concrete types, which is why most vector and matrix types in autodiff also implement methods that operate on concrete types.
For instance, DenseFloat64Vector implements a method called VADDV that takes as arguments two objects of type DenseFloat64Vector.
Methods that operate on concrete types are always named in capital letters.

But many types doesn't have the concrete version of the method. DenseReal64 is one of those type where it has MdotV but no MDOTV.

Is there any reason for this?

LICENSE file?

I believe this is as simple as:

curl -L https://www.gnu.org/licenses/gpl-3.0.txt > LICENSE
git add LICENSE
git commit -a -m 'Add GPL3 license'
git push

It would fix the documentation not showing on https://pkg.go.dev/github.com/pbenner/autodiff?tab=overview

Document the use of ADD vs. Add, SET vs. Set, etc.

It's somewhat difficult for an outsider to understand the capitalization conventions because they're not spelled out.

github.com/pbenner/autodiff/simple should have ConstScalar arguments wherever possible

I've started using this library and I've been very impressed so far, thanks for making it!

One thing I've noticed: it'd be a lot easier to use if the "simple" package took ConstScalar arguments wherever possible. This would allow const-y arguments to be put into expressions with the Simple package, and only after proving that a loop is hot with the profiler would you go back and remove the allocations.

A potential bug in dyadic() function?

First of all, this library is fantastic and the code is really clean that gives me a sense of a good library.
Thank you so much for making this available.

But I've found that Matrix by Vector multiplication fails on dyadic() function.
I still don't know if this is because my usage is wrong or there is a bug.

This error can be reproduced by the following code:

f := func(y *ad.DenseReal64Vector, x *ad.DenseReal64Matrix, w *ad.DenseReal64Vector) ad.MagicScalar {
	_, c := x.Dims()
	weightedInput := ad.NullDenseReal64Vector(c)
	weightedInput.MdotV(x, w)

	// Take mean
	mean := ad.NullReal64()
	mean.Vmean(weightedInput)

	return mean
}

xBack := []float64{
	1, 1, 1,
	1, 1, 1,
	1, 1, 1,
}
x := ad.NewDenseReal64Matrix(xBack, 3, 3)
wBack := []float64{
	0.5,
	0.5,
	0.5,
}
w := ad.NewDenseReal64Vector(wBack)
yBack := []float64{
	1,
	1,
	1,
}
y := ad.NewDenseReal64Vector(yBack)

_ = w.Variables(1)
_ = y.Variables(1)
_ = x.Variables(1)

z := f(&y, x, &w)

And here is where the error occurs:

func (c *Real64) dyadic(a, b ConstScalar, v0, v10, v01, v11, v20, v02 float64) *Real64 {
  c.AllocForTwo(a, b)
  if c.Order >= 1 {
    if c.Order >= 2 {
      // compute hessian
      for i := 0; i < c.GetN(); i++ {
        for j := i; j < c.GetN(); j++ {
          c.SetHessian(i, j,
              a.GetHessian(i, j)*v10 +
              b.GetHessian(i, j)*v01 +
              a.GetDerivative(i)*a.GetDerivative(j)*v20 +
              b.GetDerivative(i)*b.GetDerivative(j)*v02 +
              a.GetDerivative(i)*b.GetDerivative(j)*v11 +
              b.GetDerivative(i)*a.GetDerivative(j)*v11)
          c.SetHessian(j, i, c.GetHessian(i, j))
        }
      }
    }
    // compute first derivatives
    for i := 0; i < c.GetN(); i++ {
      c.SetDerivative(i, a.GetDerivative(i)*v10 + b.GetDerivative(i)*v01) <--- HERE!!!
    }
  }
  // compute new value
  c.setFloat64(v0)
  return c
}

This is because we have a.GetN() = 9 where as b.GetN() = 3 resulting in the following error:

panic: runtime error: index out of range [3] with length 3 [recovered]
	panic: runtime error: index out of range [3] with length 3

The code should be like this? I'm not expert on autodiff so I'm not sure but this runs without error.

// compute first derivatives
for i := 0; i < a.GetN(); i++ {
  for j :=0; j < b.GetN(); j++ {
    c.SetDerivative(i, a.GetDerivative(i)*v10 + b.GetDerivative(j)*v01)
  }
}

If this is okay, can I send a pull request?

The `Import` functions cannot support large files

The Import functions are currently limited to lines that are 64K long:

https://golang.org/pkg/bufio/#pkg-constants

When that line length is exceeded, for instance, a DenseRealVector will silently fail to read and return a 0-length result. In my testing, this means it'll load a file with 3,000 floats but not 4,000.

This wouldn't be a problem, you could simply store the vector as a column vector on disk with newlines, if it weren't for this statement:

https://github.com/pbenner/autodiff/blob/master/vector_dense_barereal.go#L363

Would you consider it to be a bug if the vector Import were to be permissive and load either column or row vectors? If not, that statement could be deleted.