zhuanxuhit / dlnd-your-first-neural-network

udacity first project

dlnd-your-first-neural-network

dlnd-your-first-neural-network's Issues

hidden nodes个数修改建议

The number of hidden units is chosen such that the network is able to accurately predict the number of bike riders, is able to generalize, and is not overfitting.

The number of hidden nodes is open and there is no right answer here. We'd like the network to be complex enough to capture the patterns in the training data. If for example there aren't enough hidden nodes, and model is not complex enough, no matter how much data you have, or how much training you do, the model won't be able to represent the boundaries in the data. A really simple case would be having data that falls on a parabola, and using a linear equation to make predictions. It may get some predictions right, but it just isn't able to capture the nature of the relationships.

So why don't we just add loads of hidden nodes? Increasing hidden nodes adds computational burden and can also affect the robustness of your model outside of the training data (for example due to overfitting). The trick is to find a balance between capturing the patterns in the data, and producing a lean model that works well outside of the training environment.

Check this out for more info:
http://cs231n.github.io/neural-networks-1/#arch

TIP: A good rule of thumb is the half way in between the number of input and output units and in this case the recommendation is at least 10.

There's a good answer here for how to decide the number of nodes in the hidden layer. https://www.quora.com/How-do-I-decide-the-number-of-nodes-in-a-hidden-layer-of-a-neural-network

test data后半段不理想优化

If you look at the results, you can clearly find your model is doing better in the first ten days than the last ten days. Remember the way you divided the training data set and test data set. You leave the data in the last 21 date as test data and train the model use the data in the earlier time period. So it makes sense that your model are doing good in the date that closer to the date in your training data and worse when it goes farther beyond your training data period. We can also argue the end of the Dec is Chrismas holiday and one can expect the customer behavior will get different than usual days. You can see it as a kind of over fitting. To improve the model, you can randomly split the data into training and testing data set and ignore the time ordering and rebuild your model.

There is no direct rule to do it, but hyper parameter optimization is one of the methods.
Play with the number of hidden layers, learning rate, the activation function etc...

Try checking the accuracy and improving the results using the following grid,
Learning rate,: 0.001,0.005, 0.1, 0.2
Number of hidden layers: 10,20,40,80,160
Activation function ,: sigmoid, tanh

Try all the above combinations and see which gives the best results..