Comments (1)
Hi Brent,
This definitely makes sense, since with the sum of squared errors the size
of the error signal is dependent on the number of samples. I am working on
a part 5 that explains stochastic gradient descent (minibatches), where you
would use a sample size independent error measure by averaging the error
signals instead of summing them.
Best regards,
Peter
On Sun, Jul 5, 2015 at 9:12 AM Brent De Weerdt [email protected]
wrote:
First, thank you for these articles!
However, when playing with the code, if I changed the number of input
samples to 40 I get this result:w(0): 0.1000 cost: 46.1816
w(1): 4.7754 cost: 92.1105
w(2): -1.8647 cost: 184.7509
w(3): 7.5657 cost: 371.6103
w(4): -5.8276 cost: 748.5129I solved this by using a learning rate inversely proportional to the
number of samples, i.e.
learning_rate = 2 / nb_of samples
instead of a fixed 0.1.I tested it with sample sizes from 5 to 10 million, and it seems to always
converge now.
I don't know if this makes any mathematical sense, just want to let you
know.—
Reply to this email directly or view it on GitHub
#5.
from peterroelants.github.io.
Related Issues (16)
- RNN part1 HOT 1
- convert ipython notebook to jekyll pages HOT 1
- Error in the math
- Equivalent implementation HOT 2
- missing - in negative log marginal likelihood HOT 1
- SyntaxError: only named arguments may follow *expression
- Wrong matrix dimensions HOT 1
- Wrong sigma for functions
- Problem when running gaussian-process-kernel-fitting.ipynb HOT 1
- Clarification with notes on "Understanding Gaussian Processes" HOT 1
- Error in GP example with noise HOT 1
- Gaussian process tutorial notebook not working on local machine HOT 1
- Link to the IPYNB notebook mentioned in https://peterroelants.github.io/posts/rnn-implementation-part02/ is broken HOT 1
- missing terms in partial derivatives? HOT 3
- Neural Network Intercept Bias HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from peterroelants.github.io.