Coder Social home page Coder Social logo

feiyuchen7 / m3net Goto Github PK

View Code? Open in Web Editor NEW
23.0 23.0 3.0 78 KB

Pytorch implementation for the paper: Multivariate, Multi-frequency and Multimodal: Rethinking Graph Neural Networks for Emotion Recognition in Conversation, CVPR 2023.

Python 100.00%

m3net's People

Contributors

feiyuchen7 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

m3net's Issues

复现时的疑问

您好,我对您的文章很感兴趣,在复现时遇到了一些问题,希望得到您的指点!
我在复现IEMOCAP数据集时,ACC和F1只能达到70,论文中写有72,我用的也是RTX 3090,实验中参数和您设置一样,这可能是由于不同参数有不同的效果,因此我也在尝试中。
image
我对超图神经网络应用于对话情绪识别上很感兴趣,因此我删掉了Multi-frequency propagation模块,ACC和F1特别低,分别为61和60,论文中论文中消融实验结果为69,想问下是否我的操作有问题,以下是我的操作过程,我仅在model_hyper.py文件中修改了相关代码,我在输出时只保留了Multivariate propagation模块的输出值。想问下正确的过程是怎样的,是否要在其他文件上做调整,这个问题一直困惑着我,希望得到您的指点,谢谢!
image
image
image

复现出现的一些问题

您好!我在尝试复现您的论文

按照您给出的参数设置
python -u train.py --base-model 'GRU' --dropout 0.5 --lr 0.0001 --batch-size 16 --graph_type='hyper' --epochs=80 --graph_construct='direct' --multi_modal --mm_fusion_mthd='concat_DHT' --modals='avl' --Dataset='IEMOCAP' --norm BN --num_L=3 --num_K=4
和指定的torch-geometric 版本1.7.2

在IEMOCAP数据集上,我得出的结果只达到了70.73,我也尝试过调整一些参数,例如use_modal、use_residue等,use_modal设置成True的话可以略微提高结果的accuracy,但是还是达不到论文的72,然后use_residue这个参数调整对结果没有明显的改变。

不知道是否哪里有遗漏,非常需要您的指点!拜托🙏

Namespace(no_cuda=False, base_model='GRU', graph_model=True, nodal_attention=True, windowp=10, windowf=10, lr=0.0001, l2=3e-05, rec_dropout=0.1, dropout=0.5, batch_size=16, epochs=80, class_weight=True, active_listener=False, attention='general', tensorboard=True, graph_type='hyper', use_topic=False, alpha=0.2, multiheads=6, graph_construct='direct', use_gcn=False, use_residue=False, multi_modal=True, mm_fusion_mthd='concat_DHT', modals='avl', av_using_lstm=False, Deep_GCN_nlayers=4, Dataset='IEMOCAP', use_speaker=True, use_modal=True, norm='BN', testing=False, num_L=3, num_K=4)
Running on GPU
construct hyper
Graph NN with GRU as base model.
<dataloader.IEMOCAPDataset object at 0x000001C9ED8ADF00>

image

Request for Feature Extractor scripts

Great job on the project! 👏 I'm interested in using this model on a custom dataset. Could you please provide a feature extractor script to facilitate this process? especially for dense net

Thanks in advance! 😊

Some doubts during the reproduction process.

Hello, congratulations on completing and publishing your work! I'm interested in graph methods, so I want to reproduce this method. However, I encountered the following issues while trying to replicate it:

1.The feature dimensions in the provided file do not match those in the code.
2.Even after modifying the dimension settings in the code based on the feature file, I still can't run it successfully. The core components of the code, model_hyper.py and HypergraphConv.py, are throwing errors.

Could you please help me with these questions? I'm looking forward to your response.

Problem of reproducing results of IEMOCAP and MELD

Hello, congratulations on completing and publishing your work! I appreciate your excellent work on ERC task, so I want to reproduce this method. However, I couldn't reproduce the results of the IEMOCAP and MELD datasets mentioned in your paper.
Here is what I have done and the results I have got, I would appreciate if you could tell me what's wrong with my implementation.

  1. download the feature files you offer and place it in proper position as set in dataloader.py:
image image image
  1. run the command you give in readme:
    Here are the results of IEMOCA, which is much lower than the paper as it only achieves around 66 in acc and F1-score. And results of paper is about 72 in F1-score and acc.
image image Here are the results of MELD, which is much lower than the paper as it only achieves around 66 in acc and 65 in F1-score. And results of paper is about 67 in acc and 68 in F1-score. image

Multi-frequency Graph construction problem

Hello, I'm trying to use your code, but I can't replicate your results. I downloaded your code and the dataset and used the code line that you gave on GitHub, but the metrics are different. Also, I have a problem finding the Multi-frequency Graph construction in the code. I would be grateful if you could help me.

Value Error

Thank you for your impressive work!
I try to reproduce the result using your training example, while it reports "ValueError: Encountered tensor with size 2607 in dimension 0, but expected size 917." in line 150 in HypergraphConv.py, which is "out = self.propagate(hyperedge_index, x=out, norm=D, alpha=alpha,
size=(num_edges, num_nodes))"
Then what should i do to fix it
Thanks!!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.