y0ast / due Goto Github PK
View Code? Open in Web Editor NEWCode for "On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty".
Home Page: https://arxiv.org/abs/2102.11409
License: MIT License
Code for "On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty".
Home Page: https://arxiv.org/abs/2102.11409
License: MIT License
Dear authors,
I really like the idea of this paper. Thank you very much! However, while reading the code, I am a little confused by the implementation of DUE.
In the paper (Appendix A.2, right after Equation 9), it is mentioned that MC sampling is used for training the classification model. But according to the implementation, I couldn't find the location where MC sampling is implemented. Based on my observation of the code, the output of the model is a Gaussian distribution. Then this distribution is fed into the VariationalElbo directly to compute the loss. Therefore, could you please link me to the place where MC sampling is implemented?
Another question about MC sampling during training is more fundamental. I think MC sampling is non-differentiable. If MC sampling exists during training, how the gradient could be propagated to the feature extractor part?
Please let me apologize if I am asking wrong questions. (I am new to Gaussian process.)
Thank you very much
I have deleted the issue. sorry for bothering you.
I come from DUQ repo... After I understand the DUQ, I think it is difficult to apply duq to regression task.
Could you add a example about how to load trained modle and likelyhood model for prediction in toymodel_regression script?
When I apply DUE regression to my task, the expected_log_prob became negative gradually?
Does it represent overfitting of my model?
What is the difference between loss in step VariationalELBO
and loss in metric expected_log_prob
?
Is it a hyperparameter, could you give some experience for selecting this paprm.
I have implemented your DUE to my 6D pose regression task, but after training, in the evaluation, the predicted mean is a constant value, could you give me some inspirations? The input data is 128 diemensions. output is 6 diem.
When I try to extend to num_outputs > 1 based on the toy_regression script
if you can modify toy regression script to support num_outputs>1
, I will be very very appreciated!
Thanks ahead!
About def initial_values(train_dataset, feature_extractor, n_inducing_points)
:
In toymodel regression script, the train_dataset is the whole training datset, could it be the mini-batch instead of whole training dataset?
Hi, I have successfully run this code and it performed well without "--sngp". However, when I added "sngp", the accuracy fell to 0.1. It might make sense when it was 0.7 or 0.8, but it seemed too low to be 0.1.
Is it because I did something wrong, or the performance of GP layer is highly correlated with the feature extractor? Is such optimizer applicable for some bigger feature extractors, such as BERT for text classification for instant?
And by the way, is the performance significantly correlated with the optimizer? Is adam also okay?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.