giovannipasserello / shgp

Sparse Heteroscedastic Gaussian Processes

SHGP

SHGP is a GPflow-based software package for Sparse Heteroscedastic Gaussian Processes. It was developed as an accompaniment to my final-year master's thesis at Imperial College London - 'Heteroscedastic Inducing Point Selection for Sparse Gaussian Process Classification'. The focus of this thesis is to compute an SGPR-style model for Gaussian process classification, in which gradient-based optimisation of the variational distribution over inducing points is not required. Further to this, we propose a method for the superior selection of inducing points for Gaussian process classification.

Contributions

• An implementation of a heteroscedastic Gaussian process regression model in which the variational distribution is analytically optimised. This is an extension of the collapsed model derived in [Titsias 2009].
• An implementation of a novel heteroscedastic Gaussian process classification model in which the variational distribution is analytically optimised. This model employs Pólya-Gamma data augmentation to form a conditionally conjugate lower bound to the likelihood. Importantly, the effective likelihood is an unnormalised Gaussian distribution with heteroscedastic variance.
• An implementation of a novel heteroscedastic inducing point selection method for both Gaussian process regression and classification.
• A robust procedure for stable Cholesky decompositions with dynamic jitter selection.
• Multiple experiments which investigate the benefits of the above contributions. In particular, we show that our collapsed Gaussian process classification model paired with our inducing point selection method permits much sparser models than alternative schemes. We also show that our selection method allows the number of inducing points required for a sufficiently accurate approximation to be automatically determined by the model.

License

This project is licensed under the MIT License - see LICENSE for details.

shgp's Issues

question for the invlink function in svgp_vs_pgpr.py file

I read the codes and have a question for prediction in svgp_vs_pgpr.py file:

    X_test_mean_pgpr_go, _ = pgpr_go.predict_f(X_test)
    P_test_pgpr_go = invlink(X_test_mean_pgpr_go)

According to the mathematic expression, the result should be the integration of N(f| mu, sigma)*N(f|y/2,w)p(w)dfdw.
How to understand the written codes? Why is invlink function here for pgpr model? Thank you very much!