danielkelshaw / concretedropout Goto Github PK
View Code? Open in Web Editor NEWPyTorch implementation of 'Concrete Dropout'
Home Page: https://arxiv.org/abs/1705.07832
License: MIT License
PyTorch implementation of 'Concrete Dropout'
Home Page: https://arxiv.org/abs/1705.07832
License: MIT License
Hi ! I read Yarin Gal's paper and I did not understand how the weight regulariser and dropout regulariser are initialized. The author provided a formula, but it is not very clear (e.g what means prior length scale ? and which value to assign for this variable ?). Could you explain how you find the values used to inizialize the weight regulariser and the dropout regulariser ?
An example of using ConcreteDropout
in training a network for MNIST classification should be included - this provides a demonstration of the capabilities of the network.
The model passes the logits through softmax in the output layer, but then the F.cross_entropy() function is used which combines log_softmax and nll. I think maybe the softmax in the output layer is not necessary?
In the initial development of this code I was experimenting with the use of class decorators to add functionality, namely the concrete_regulariser
. Coming back to this code several months later I realise that, although this works, it is not the nicest way of implementing a solution.
I would like to rework my implementation of the regularisation by making a base-class which accounts for the implementation, rather than requiring the user to wrap their model implementation with a pre-defined decorator.
At the moment the README.md
provides no particular indication as to what this project is about - it would be a good idea to update this to describe the purpose of the code that has been developed.
It would be good to demonstrate how the modules developed can be user in a very simple way. This could help users implement the code found in this repository.
In order to provide a suitable development environment the repository should be set up with an appropriate directory structure as well as functionality for automated continuous integration.
Implementation of the ConcreteDropout
class should inherit from nn.Module
to allow the functionality to be used inline with other PyTorch modules. Upon implementation the ConcreteDropout
module can be used in a regular neural network to be tested.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.