Comments (2)
树哥你好,我把网络换了resnet50,发现还真的只有RMSprop的结果能正常一点点,Adam和SGD好像没法正常更新。Any insight on this?仅是讨论哈哈,不是Issue。
我实验发现只要存在形如$S_{ij}*\Theta_{ij} - log(1+e^{\Theta_{ij}})$的Loss,在PyTorch里就只有RMSprop效果还能看,其他的优化器效果很差。不过我当时在复现DPSH这篇论文的时候发现
SGD是可以优化的,不过momentum这一项需要置为0,并且U和B变量(分别负责保存CNN输出和优化出来的B)初始化非常重要,具体细节忘了,好像是其中一个必须zero初始化,另一个必须randn初始化,可能还要加sign之类的,SGD才可能优化,特别玄学。
我阅读过作者提供的Matlab代码,它们直接是用SGD就可以做,能和PyTorch这个框架有关?理论水平不够,分析不出来哈哈哈,现在遇到这种形式的Loss我就直接用RMSprop了= =
from dsdh_pytorch.
我佛了。。我现在遇到的问题是用了RMSprop,大概前50个epoch还是正常的map基本上是在提高,然后50个epoch之后,map反而开始下降了,但loss的值还是在变小,就很晕。。还没能正常复现出DPSH的实际性能。(突然发现Issue是在DSDH下,我跑的是DPSH)
我之前跑过其他的方法,个人感觉,可能log()的梯度在pytorch里有点问题,但我也说不清楚。
from dsdh_pytorch.
Related Issues (6)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dsdh_pytorch.