Comments (7)
@barak @gbaydin The generalized power rule only applies when the x is non-zero. What's the right way to treat this in the variant of symbolic differentiation you're using? thx
from diffsharp.
@gbaydin @barak I took a look at fixing this more out of curiosity than anything else. Is adding the conditional check the right fix? https://github.com/DiffSharp/DiffSharp/pull/31/files#diff-419be3dee4710aa72c56268abcf97b05L63
from diffsharp.
The issue is the x**y is continuous and differentiable except at (x,y)=(0,0). But the equation needs to be special-cased for some of the cases x=0 or y=0. Depending on which is "active", to save unnecessary computation.
This is done reasonably carefully in the "FAD" package for Haskell, as I recall.
from diffsharp.
See lines 382 and following in https://github.com/NUIM-BCL/fad/blob/master/Numeric/FAD.hs
from diffsharp.
OK, thx
from diffsharp.
Actually looking at that Haskell FAD code again, the division by x on line 391 is wrong when x==0. Needs special cases for that as well, basically x^(n-1) cannot be calculated as (x^n)/x when x==0, so even when x^n is in hand one needs to calculate x^(n-1) from scratch. In the FAD context, this is a particular issue when x is not just a dual number, but an infinite power series, as is often the case there. I suppose the right thing in that context is to first calculate x^(n-1) and then multiply it by x to get x^n.
This is the kind of fiddly numerics that makes me avoid numeric analysis...
from diffsharp.
Closing because this is out of date for DiffSHarp 1.x, which is currently planned to be AD only
from diffsharp.
Related Issues (20)
- Reimplement nll loss with gather HOT 1
- Move to addcmul etc as RawTensor primitives? HOT 1
- Strange error regarding "HIP" backend HOT 4
- Implement parallel data loading HOT 1
- Missing dsharp.argmax overload HOT 1
- In the rnn.fsx example, languageModel.summary() throws an error after the model has been trained. HOT 1
- In rnn, buffer.nelement differences between newly created stateVector and saved stateVector cause `loadState` to fail HOT 2
- Write up contribution guidelines
- Improve the Marsaglia Gaussian generator HOT 1
- Should `model.state <- dsharp.load(modelFile)` change which device the model is on?
- Reference backend does not support from end index slicing HOT 2
- Optimizer ignores specified maximum number of epochs under certain conditions
- Rounding always returns NaNs
- ToString of model with Parameters causes a stackoverflow
- Conversion to/from ONNX models HOT 1
- derivation of Tensor.matmul
- Unable to use corrcoef (using v1.0.7) HOT 1
- Naming - fork of 0.8.4
- dsharp.save/load with BinaryFormatter is blocked on .NET Interactive
- Current state HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diffsharp.