Why do you calculate the gradient for the conjugate gradient mannually? about modl

[hkaggarwal]

Hello KaiXuan Wei,

Some theoretical but mostly practical reasons were there.

1. First, theoretically, we could validate that backward operation is also a conjugate gradient on the corresponding "gradient" as an input.
2. It was faster if I do not reply on the automatic gradient of CG block. Auto gradients of some operations were automatically falling back on the CPU.
3. Since MRI data is inherently complex data type so the conjugate gradient was written to handle complex data as input and complex data as output. Two years back in TensorFlow some complex operations did not have GPU gradients.
4. Some complex operations had wrong gradients (imaginary part was becoming zero).

Bests

[Vandermode]

Thank you for the explanation!