santoshbalija / bayes-resources

Plethora of knowledge (papers, books, videos etc) in my journey to learn bayesian methods in machine learning

Bayes Resources

Plethora of knowledge (papers, books, videos etc) in my journey to learn bayesian methods in machine learning

Books

• Machine Learning A Probabilistic Perspective (A good start to learn probabilistic machine learning, the first two introductory chapters are pretty good for starter) - Kevin P. Murphy

• Statistical Rethinking - Richard McElreath | course | numpyro implementation | (seems good read later)

• Surrogates - Robert B. Gramacy

• Bayesian Optimization Book - Roman Garnett

Tutorials and Course Notes

• Course Notes for Bayesian Models for Machine Learning (A must read note, the formula derivations are nice) - Columbia University Fall 2016

• Variational Inference: A Review for Statisticians - Blei et al

• An Introduction to Variational Methodsfor Graphical Models - Jordan et al

• Advanced methods of variational inference - Deep Bayes 2018 - Max Welling

• Matrix Cook Book (many usefull facts on linear algebra and beyond)

• Deep Probabilistic Modelling with with Gaussian Processes - Neil D. Lawrence

• Bayesian workflow - Gelman et,al.

• Neil Lawrence's Talks - Neil D. Lawrence

Bayesian Deep Learning

• Bayesian Learning for Neural Networks - Neal, Radford M.

Monte Carlo Dropout Related

• Uncertainty in Deep Learning (PhD Thesis) - Yarin Gal

• Dropout as a Bayesian Approximation:Representing Model Uncertainty in Deep Learning - Yarin Gal

note that this method is little controversial following Ian Osband note on NIPS 2016 workshop. more details on r/machinelearning

Other Methods

Gaussian Process and Bayesian Optimization

Gaussian Process

• Gaussian Processes for Machine Learning - Carl Edward Rasmussen. short intro : link. video lecture : link

• Sparse GP using pseudo Input - Snelson & Ghahramani

• Scalable Gaussian process inference using variational methods - Matthews Thesis. (good resources to learn variational GP)

• A Practical Guide to Gaussian Processes - Deisenroth, Luo, Van der Wilk

• A Visual Exploration of Gaussian Processes - Görtler et al

Bayesian Optimization

• A Tutorial on Bayesian OptimizationPeter - I. Frazier

• Taking the Human Out of the Loop:A Review of Bayesian Optimization Nando Defreitas . more comprehensive than above

Bandit Algorithm

• The Multi-Armed Bandit Problem and Its Solutions -Lilian Weng

Online Courses

• Bayesian Method for Machine Learning - National Research University Higher School of Economics (Comprehensive, compact and intuitive course on bayesian method for machine learning, this is good start for getting basic intuition on bayesian methods)

Summer School

• Deep|Bayes 2019 material and code: github video: youtube

Blogs

• betanalpha.github.io - Michael Betancourt's blog
• Notes on machine learning - Peter Roelants's blog (have good notes on GP, Multivariate Gaussian and Multi-Armed bandit)
• Knowledge Gradient Visualized - Louis Tiao (Nice visualization to understand KG)

Notes etc

• Optimal Transport and Wassertein Distance - Larry Wasserman
• Understanding ELBO - Xitong Yang
• More on Multivariate Gaussians - Chuong B. Do
• Gaussian Densities - Tony E. Smith
• Gumble Distribution - mrahtz
• Expected Improvement Derivation - Al-Dujaili
• Math for Machine Learning - Hal Daumé III

QnA

• How to choose prior
• Standardizing data for GP Regression