y0ast / due Goto Github PK

Dear authors,

I really like the idea of this paper. Thank you very much! However, while reading the code, I am a little confused by the implementation of DUE.

In the paper (Appendix A.2, right after Equation 9), it is mentioned that MC sampling is used for training the classification model. But according to the implementation, I couldn't find the location where MC sampling is implemented. Based on my observation of the code, the output of the model is a Gaussian distribution. Then this distribution is fed into the VariationalElbo directly to compute the loss. Therefore, could you please link me to the place where MC sampling is implemented?

Another question about MC sampling during training is more fundamental. I think MC sampling is non-differentiable. If MC sampling exists during training, how the gradient could be propagated to the feature extractor part?

Please let me apologize if I am asking wrong questions. (I am new to Gaussian process.)

Thank you very much

I have deleted the issue. sorry for bothering you.

I come from DUQ repo... After I understand the DUQ, I think it is difficult to apply duq to regression task.

Could you add a example about how to load trained modle and likelyhood model for prediction in toymodel_regression script?

When I apply DUE regression to my task, the expected_log_prob became negative gradually?
Does it represent overfitting of my model?

What is the difference between loss in step VariationalELBO and loss in metric expected_log_prob?

Is it a hyperparameter, could you give some experience for selecting this paprm.

I have implemented your DUE to my 6D pose regression task, but after training, in the evaluation, the predicted mean is a constant value, could you give me some inspirations? The input data is 128 diemensions. output is 6 diem.

When I try to extend to num_outputs > 1 based on the toy_regression script

• No problem when num_outputs >1 with batch_size =1.
• But, when batch_size>1, error happens.

if you can modify toy regression script to support num_outputs>1, I will be very very appreciated!
Thanks ahead!

About def initial_values(train_dataset, feature_extractor, n_inducing_points):
In toymodel regression script, the train_dataset is the whole training datset, could it be the mini-batch instead of whole training dataset?

Hi, I have successfully run this code and it performed well without "--sngp". However, when I added "sngp", the accuracy fell to 0.1. It might make sense when it was 0.7 or 0.8, but it seemed too low to be 0.1.
Is it because I did something wrong, or the performance of GP layer is highly correlated with the feature extractor? Is such optimizer applicable for some bigger feature extractors, such as BERT for text classification for instant?
And by the way, is the performance significantly correlated with the optimizer? Is adam also okay?