How can I add custom gradients? about ad

[cartazio]

#15 may help

the code in that version was

erf1 = lift1 erf $ \x -> (fromInteger1 2 /! sqrt1 pi1) *! exp1 (negate1 x *! x)

https://github.com/ekmett/ad/blob/master/src/Numeric/AD/Jacobian.hs
hass the class which is used to facilitate that

[cartazio]

point being i believe this answers your question, though please chime in if you want more examples or links / hit a new problem.

but closing for now :)

[kai-qu]

Thanks! I have a nice minimal example working like this:

λ> import Numeric.AD
λ> import Numeric.AD.Jacobian
λ> import Numeric.AD.Internal.Forward
λ> (lift1 (\x -> x^2) (\x -> 2 * x)) (Forward 3 1)
Forward 9 6

However, the Jacobian module only seems to provide primitives for defining derivatives for scalar or two-argument functions. I'd like to define a custom gradient for a function f : R^n -> R, for example something like

liftN (\[x, y, z] -> x * y * z) (\[x, y, z] -> [y * z, x * z, x * y])

Any thoughts on how I could do this? It doesn't seem efficient to try and hack this functionality in terms of lift1.

[cartazio]

Ill try to mull myself. I think edwark is likely to have better thoughts if he has the time to muse em

…

On Fri, May 3, 2019 at 6:49 AM Katherine Ye ***@***.***> wrote: Thanks! I have a nice minimal example working like this: λ> import Numeric.AD λ> import Numeric.AD.Jacobian λ> import Numeric.AD.Internal.Forward λ> (lift1 (\x -> x^2) (\x -> 2 * x)) (Forward 3 1) Forward 9 6 However, the Jacobian module only seems to provide primitives for defining derivatives for scalar or two-argument functions. I'd like to define a custom gradient for a function f : R^n -> R, for example something like liftN (\[x, y, z] -> x * y * z) (\[x, y, z] -> [y * z, x * z, x * y]) Any thoughts on how I could do this? It doesn't seem efficient to try and hack this functionality in terms of lift1. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#77 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAABBQVVGL5CATONY35EOW3PTQKFBANCNFSM4HKIK3CA> .

[cartazio]

@hypotext it is a gap, and its currently not quite possible to do gradients etc with "custom rules", which is a problem i think we'd both agree

[kai-qu]

Yes, I would really appreciate if that were a feature!

…

On Mon, Jan 20, 2020 at 3:43 PM Carter Tazio Schonwald < ***@***.***> wrote: @hypotext <https://github.com/hypotext> it is a gap, and its currently not quite possible to do gradients etc with "custom rules", which is a problem i think we'd both agree — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#77?email_source=notifications&email_token=AAVCM5FYMM5AAO54JV4EWXLQ6YEGTA5CNFSM4HKIK3CKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJNYWCQ#issuecomment-576424714>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVCM5DHJRIKFZMLOERBW2DQ6YEGTANCNFSM4HKIK3CA> .

[cartazio]

ok, lets start with several sub questions

1. what should/would this api look like?
2. how should/would this interact with calculating higher derivatives etc?

what types would you expect/hope the interfaces to have?

[kai-qu]

I'd like to be able to specify that when the autodiff hits a function with a particular name, that that function is a "leaf" of the autodiff tree, and to use a custom function as its derivative. Can't remember other details now as this issue was from a while ago, but I would trust your judgement as library designers!

…

On Mon, Jan 20, 2020 at 3:52 PM Carter Tazio Schonwald < ***@***.***> wrote: ok, lets start with several sub questions 1. what should/would this api look like? 2. how should/would this interact with calculating higher derivatives etc? what types would you expect/hope the interfaces to have? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#77?email_source=notifications&email_token=AAVCM5FKT5RVKIZEALWFOJDQ6YFKTA5CNFSM4HKIK3CKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJNZHQY#issuecomment-576426947>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVCM5HY3L4NNRZB6JUSIBTQ6YFKTANCNFSM4HKIK3CA> .

[cartazio]

ok, :)

[cartazio]

I'm honestly thinking about experimenting with writing a ghc core plugin to support better optimization for autodiff computations on ghc core level sometime this jan/feb

[dschrempf]

Hi! May I ask about the status of this issue? I am in a similar situation as the author. Thank you!