Missing factor 0.5 in KL-divergence? about tutorial_bayesiancompressionfordl

[aswin-raghavan]

I believe this is true. log(1/var) becomes -log(var), the 0.5 should be there unless log(sd) was modeled instead of log(var).
EDIT: Seems to be correct in the fully connected layer on Line num. 141

[clouizos]

Apologies for the very late reply; I just noticed the issues here. Indeed you are correct, there should be a factor of 0.5 in front of the KL. We only tested the fully connected version for this release, so this fell through the cracks. Thanks for noticing!

[aswin-raghavan]

We used the solution posted before, but the LeNet model had lower performance than your paper. See https://arxiv.org/abs/1811.04985 I will try running this again. Aswin On Dec 11, 2018 12:10 PM, Christos Louizos <[email protected]> wrote: Closed #8<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FKarenUllrich%2FTutorial_BayesianCompressionForDL%2Fissues%2F8&data=01%7C01%7Caswin.raghavan%40sri.com%7C065f8c25ca82414e292008d65f8b8acc%7C40779d3379c44626b8bf140c4d5e9075%7C1&sdata=AafJUQUE18%2FMP%2BboY1ma1UBSFMSir5fonVcWKDsUwok%3D&reserved=0>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FKarenUllrich%2FTutorial_BayesianCompressionForDL%2Fissues%2F8%23event-2019315930&data=01%7C01%7Caswin.raghavan%40sri.com%7C065f8c25ca82414e292008d65f8b8acc%7C40779d3379c44626b8bf140c4d5e9075%7C1&sdata=q3hDvAO8MAPB%2FD7dFl1UDs%2B8wBDIwgeRgXLQb5ZhR5k%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAhK_yjqSw5u5DH_J3H4r5dt9YMEnTiMJks5u3-b-gaJpZM4V1QYi&data=01%7C01%7Caswin.raghavan%40sri.com%7C065f8c25ca82414e292008d65f8b8acc%7C40779d3379c44626b8bf140c4d5e9075%7C1&sdata=13QpGffhwU2ILoVKSuH6SXeb5tW12waJuWKr0uQD7NU%3D&reserved=0>.