grad of grad about torch-autograd

[lukealonso]

@davidBelanger thanks for the test. There are a few functions used internally in the gradient definitions that don't yet have a gradient definition themselves, so you get a missing gradient error when we try to compute the grad of grads of those functions. We'll be adding those soon - will update the ticket when it's done.

[davidBelanger]

I was just poking around a bit to see if I could implement the gradient for fillSameSizeAsInPlace myself. It's very confusing to me, since I can't see where util.fillSameSizeAsInPlace is called from anywhere, from either torch-autograd or any torch project more generally. Where is this call happening?

[brianzhang01]

If tests and maybe an example could be added for grad of grad when it's fixed, that would be great too.

David, I believe the call is coming through an aliased function called fillSameSizeAs.

[alexbw]

That's true. All of the "inPlace" operations are aliases used in the optimize=true codepath.
Would definitely appreciate PRs w/ tests and fixes!

[alexbw]

FYI, an update. We're only missing gradients for the following functions:

repeatTensor
setNotEqual
selectSliceCopy
narrowSliceCopy
indexAdd

Once those are defined, I believe 2nd and higher-order gradients will be fully supported in optimize=true mode. 2nd order gradients will not be supported in the default optimize=false mode, and will not be for the foreseeable future.

[alexbw]

Supported in #83. Still some Lua version-specific errors, will fix and merge soon.

[alexbw]

Fixed with PR #83