thaiminhpv / relahash

Official implementation of IEEE Access 2023 paper: "RelaHash: Deep Hashing with Relative Position"

👋 Hi there, I'm Phạm Vũ Thái Minh

• I began studying at the University when I was 16, got my first-authored Q1 journal article and graduated from my University when I was 19. My interests are deep learning and software development.

• I love solving problems and building software that impacts people around me. I have contributed to some non-profit/open-sourced organizations and communities such as Hugging Face, Langchain, AWS Academy (educator), ASAP Project (about psychology), 80% Project (about equality), Developh Vietnam, Japanese Software Engineers Club (teaching), etc.

• Additionally, I have participated and received awards in many competitions.

Connect with me!

• [email protected]
• Facebook
• LinkedIn

For detailed information about me, you can view my resume or my website

relahash's Issues

Hello , I have a question and I am really desire to get a reply

Thank you for sharing the code, and these days I have been trying to test Relahash's performers in another dataset, I found that in the "labels_scaled" part, It will be easy to get a NAN loss,

    def forward(self, logits, z, labels):
        if self.multiclass:
            if not self.one hot:
                labels = F.one_hot(labels, logits.size(1))
            labels = labels.float()

            margin_logits = self.compute_margin_logits(logits, labels)

            # label smoothing
            log_logits = F.log_softmax(margin_logits, dim=1)
            labels_scaled = labels/labels.sum(dim=1, keepdim=True)  # that is place
            loss = - (labels_scaled * log_logits).sum(dim=1)
            loss = loss.mean()

The way I came across this question was during my attempt to identify the main cause of the NAN loss. Throughout this process, I have preserved every output variable, such as log_logits and labels_scaled. Upon closer examination, I discovered that there are some Nan values in the labels_scaled output. I believe this discrepancy may be attributed to an incompatibility between my model and yours.

To elaborate, if I don't consider a large loss to be problematic, I could potentially remove the labels_scaled part without significantly impacting the model's performance. This is the most straightforward solution I can currently think of.

In my efforts to mitigate any negative effects, I have attempted to use the loss_scaled approach by utilizing the apex package from https://github.com/NVIDIA/apex. However, I am uncertain whether my approach is correct or entirely misguided. Therefore, I would greatly appreciate your feedback on the matter or any appropriate solutions you may provide.

Thank you sincerely,
An individual from China with limited proficiency in English. 🙏