thu-coai / emotional-support-conversation Goto Github PK

View Code? Open in Web Editor NEW

223.0 4.0 32.0 72.42 MB

Data and codes for ACL 2021 paper: Towards Emotional Support Dialog Systems

License: Other

Python 92.57% Shell 0.09% Cuda 2.90% C++ 4.44%

emotional-support-conversation's Introduction

Emotional-Support-Conversation

Copyright © 2021 CoAI Group, Tsinghua University. All rights reserved. Data and codes are for academic research use only.

2024-05-14

Added

To support future research, we release FailedESConv.json, 196 negative samples of emotional support conversations that were primarily dropped when we were collecting ESConv.
- Our rule to determine an effectively negative sample: (# of turn > 2 AND # of speaker > 1) AND ((post-conversation survey was done AND (negative emotion intensity does not decrease OR empathy score <=2 OR relevance score <=2 )) OR (the support-seeker did not finish the post-conversation survey AND the support-seeker's last feedback score <= 2) )

Data and codes for the ACL 2021 paper: Towards Emotional Support Dialog Systems

If you use our codes or your research is related to our paper, please kindly cite our paper:

@inproceedings{liu-etal-2021-towards,
  title={Towards Emotional Support Dialog Systems},
  author={Liu, Siyang  and 
    Zheng, Chujie  and 
    Demasi, Orianna  and 
    Sabour, Sahand  and 
    Li, Yu  and 
    Yu, Zhou  and 
    Jiang, Yong  and 
    Huang, Minlie},
  booktitle={ACL},
  year={2021}
}

Data

The corpus file is ESConv.json. We have collected more conversations with more problem topics. ESConv now contains 1,300 conversations with 10 topic problems.

Statistics

Problem Category

Problem Category	ongoing depression	breakup with partner	job crisis	problems with friends	academic pressure	procras- tination*	alcohol abuse*	issues with parent*	sleep problems*	appearance anxiety*	school bullying*	issues with children*
Number	351	239	280	179	156	13	12	18	28	12	2	10

* denotes the new topics added during the second collection. We hope new data supports the future research in transferring the ability of models from old topics to new ones.

Strategy Category

Strategy Category	Number
Questions	3801(20.7%)
Self-disclosure	1713(9.3%)
Affirmation and Reassurance	2827(15.4%)
Providing Suggestions	2954(16.1%)
Other	3661(18.3%) / 3341(18.2%)
Reflection of feelings	1436(7.8%)
Information	1215(6.6%)
Restatement or Paraphrasing	1089(5.9%)

Model Implementation

We provide two versions of model implementation:

codes is the version that we used in the original experiments
codes_zcj is the version reproduced by @chujiezheng

emotional-support-conversation's People

Contributors

Stargazers

Watchers

emotional-support-conversation's Issues

找不到 strategy.json文件

按照readme运行 process.py提示找不到strategy.json，这个文件在哪找呢？

作者您好，不好意思打扰一下：可以问一下关于策略如何根据历史文本识别的吗？，以及如何根据已经识别出策略，如何生成相应的回复文本的？
我修改了8个策略的编号。重新从头训练推理，生成的结果中策略则不准确了。所以不太清楚是如何根据对话历史识别出策略的。
关于第二个问题：根据对话历史和策略生成response内容。根据文章得知 response内容应该不是根据策略分类匹配的内容，那应该是如何生成的? 在哪个代码文件里可以找到？

对您的研究非常感兴趣，麻烦作者可以帮忙解答一下，十分感谢

a question about train

在数据prepare的时候：
labels = ([strat_id] + response + [eos])[:max_decoder_input_length + 1]
decoder_input_ids = [bos] + labels[:-1]
label的第一个值是strat_id对应的token_id，decoder_input_ids 的第一个值是bos对应的token_id.
训练过程中计算loss的时候：loss = F.cross_entropy(lm_logits.view(-1, lm_logits.size(-1)), labels.view(-1), reduction='none')，lm_logits计算的原始输入是decoder_input_ids ，与label进行cross_entropy，每个response的第一个token_id是不是对应错了？？求大佬指导

多轮对话评估问题

作者，您好，从大模型过来学习，之前没有做过传统模型的多轮对话的自动化指标评估，关于自动化评估有一些问题想请教，如果冒昧，还请海涵！
想请问下论文中多轮对话这些指标是怎么计算的？ppl比较好理解，Rouge、bleu等指标是怎么计算得到的呢？按照计算来看应该是有模型生成的多轮对话与和ground truth的多轮对话比较，第一轮对话的生成比较好理解，后几轮的对话模型是如何生成的呢？

运行infer_vanilla.sh时出现valueError

错误如下：
ValueError: The following model_kwargs are not used by the model: ['batch_size']

运行bash RUN/infer_vanilla.sh报错

您好！
我在运行bash RUN/infer_vanilla.sh报以下错误：

这是由于以下代码

请问怎么修改呢？

Training loss suddenly decrease at the beginning of each epoch.

Hi,

I use the code in codes to train model. But I find a strange phenomenon that the loss suddenly decreases at the beginning of every epoch.

So, my question is that is this the same as your training curve? and why does it suddenly decrease at the beginning of every epoch? I thought about it for a long time, but I still can't figure it out. Thanks a lot!

运行infer_strat.sh时显示ValueError

您好，我运行infer_strat.sh时会出现下面的问题，想咨询下作者解决的方法，谢谢
Traceback (most recent call last):
File "/root/Emotional-Support-Conversation-main/codes_zcj/prepare.py", line 36, in
toker = build_model(only_toker=True, **names)
File "/root/Emotional-Support-Conversation-main/codes_zcj/utils/building_utils.py", line 27, in build_model
raise ValueError
ValueError

关于论文两处公式的一些疑问？

你好，非常期待作者开源代码~
这里在读论文的时候，发现两个公式中的变量似乎没有在论文中解释：
1）：$h_{1,1}^s$表示什么意思？${\hat{H}}^s$是把这些$h^s_{i,j}$一起concat吗？

2）：$C_1$表示的什么意思？

非常期望得到作者的回复~
谢谢~

由于运行文件是.sh文件，该如何debug运行

我想debug逐步分析学习代码，可是运行文件是.sh文件，我在。sh文件无法打断点，又在python文件里打断点，没有效果。请问应该如何做？

The training steps

在训练步骤这是会出现过拟合的问题么，可以解决么，

运行infer_strat.sh 脚本报错

在运行infer_strat.sh脚本时每次validation结束后，会报错：

TypeError: forward() got an unexpected keyword argument 'encoder_outputs'
inferring: 0%| | 0/195 [00:00<?, ?it/s]

是哪里出现了问题，谢谢

Some features of the data are missing

在1300个对话的ESCConv.json中，有150个对话没有“final_emotion_intensity”，请问能够补全吗？

why the parameters(max_src_len and max_tgt_len) in interact_vanilla.sh are not in interact.py?

Confusion about the Dataset

The meaning of "seeker_question1 and 2" as well as "supporter_question1 and 2" was not apparent in the paper. I kindly request the authors to provide an explanation for these terms for better clarity and understanding.

a question about the backbone models

Hi,

Great work!
I was wondering whether it is possible to change the English task into a Chinese task.
Do you think it is possible to make the strategy come true through the Chinese data and the Chinese model?

Thanks for your reply!

运行interact_strat.sh时，一直提示KeyError: 'strategy'。请求解决谢谢

错误如下：
用户无论输入什么，机器都提示策略错误。是找不到策略吗？求作者解答谢谢
A new conversation starts!
Human: hi
Traceback (most recent call last):
File "/root/Emotional-Support-Conversation-main/codes_zcj/interact.py", line 168, in
inputs = inputter.convert_data_to_inputs(history, toker, **dataloader_kwargs)
File "/root/Emotional-Support-Conversation-main/codes_zcj/inputters/strat.py", line 87, in convert_data_to_inputs
strat_id = process('[' + dialog[i]['strategy'] + ']')
KeyError: 'strategy'

a question about interact_strat

Hi! Great work!
I wanted to interact with the bot with strategy,
so I trained the model following train_strat.sh.
then I wanted to interact with the trained model and run interact_strat.sh.
However, I get errors in inputters/strat.py
How could I solve the problem?

/workspace/Dialogue/Emotional-Support-Conversation/codes_zcj# bash RUN/interact_strat.sh
06/07/2022 00:58:18 - INFO - utils.building_utils - loading finetuned model from /workspace/Dialogue/Emotional-Support-Conversation/codes_zcj/DATA/strat.strat/2022-06-07004140.3e-05.16.1gpu/epoch-1.bin
06/07/2022 00:58:19 - INFO - utils.building_utils - deploying model...

A new conversation starts!
Human: Hi!
Traceback (most recent call last):
File "interact.py", line 168, in
inputs = inputter.convert_data_to_inputs(history, toker, **dataloader_kwargs)
File "/workspace/Dialogue/Emotional-Support-Conversation/codes_zcj/inputters/strat.py", line 87, in convert_data_to_inputs
strat_id = process('[' + dialog[i]['strategy'] + ']')
KeyError: 'strategy'

Size mismatch

size mismatch for final_logits_bias: copying a param with shape torch.Size([1, 54953]) from checkpoint, the shape in current model is torch.Size([1, 54944]). size mismatch for model.shared.weight: copying a param with shape torch.Size([54953, 512]) from checkpoint, the shape in current model is torch.Size([54944, 512]).
size mismatch for model.encoder.embed_tokens.weight: copying a param with shape torch.Size([54953, 512]) from checkpoint, the shape in current model is torch.Size([54944, 512]).
size mismatch for model.decoder.embed_tokens.weight: copying a param with shape torch.Size([54953, 512]) from checkpoint, the shape in current model is torch.Size([54944, 512]).
size mismatch for lm_head.weight: copying a param with shape torch.Size([54953, 512]) from checkpoint, the shape in current model is torch.Size([54944, 512]).

我把从 https://huggingface.co/lsy641/ESC_Blender_noStrategy/tree/main 的模型放到blend-small文件夹下面之后，运行Python BlenderEmotionalSupport.py 文件，遇到了这个维度不匹配的错误，请问这是怎么回事呢？

About Backbone Models

Hi,

Great work !
I want to know whether CDial-GPT can be considered as one of the Backbone Models or not?

Thanks!

tr_loss1, logging_loss1 = 0.0, 0.0

您好，在复现代码的时候，有个细节请教您以下：tr_loss1记录的是什么？在源码中
if args.strategy:
tr_loss1 += outputs[1].mean().item()
计算得到tr_loss1，之后使用它计算ppl，这里的又是什么含义呢？（不是困惑度啊）

在inteact_strat.sh一直提醒 command not found

一直提醒--fp16 false 下一句的参数名称 command not found。谁在-fp16 false 下面就提醒谁找不到，麻烦作者指导一下，谢谢

有中文数据吗

请问这份数据有中文版本吗？谢谢

”glove.6B.300d.model.bin“ 文件缺失

./codes_zcj/metric/word2vec/evaluate.py 中第15行中：
class Embedding(object):
def init(self):
path = '/home/zhengchujie/wordvector/english/glove6B' # please modify the path manually
# path = ''
self.m = KeyedVectors.load(os.path.join(path, 'glove.6B.300d.model.bin'), mmap='r')
try:
self.unk = self.m.vectors.mean(axis=0)
except AttributeError:
self.unk = self.m.syn0.mean(axis=0)

'glove.6B.300d.model.bin'文件要怎么下载呢

BlenderEmotionalSupport.py代码有误

codes/BlenderEmotionalSupport.py：974行write拼写错误

$(7T19U{NX7_H1UB6U1_OE7J$

train

您好，在执行train_vanilla.sh步骤时出现的错误，前面都是按照redome进行操作，能麻烦进行解答一下么

No model file found in codes

when I ran the code python BlenderEmotionalSupport.py, I got an error:

OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] found in directory ./blender-small or from_tf set to False

After downloading model file in this link, then changing the self.output_dir in class Args() , I got following error

Traceback (most recent call last):
File "/data00/home/zhushaojin/work_dir/20210701_ESC/Emotional-Support-Conversation/codes/src/transformers/modeling_utils.py", line 1038, in from_pretrained
state_dict = torch.load(resolved_archive_file, map_location="cpu")
File "/home/zhushaojin/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 595, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/zhushaojin/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 764, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "BlenderEmotionalSupport.py", line 1020, in
generate()
File "BlenderEmotionalSupport.py", line 920, in generate
from_tf=False,
File "/data00/home/zhushaojin/work_dir/20210701_ESC/Emotional-Support-Conversation/codes/src/transformers/modeling_utils.py", line 1041, in from_pretrained
f"Unable to load weights from pytorch checkpoint file for '{pretrained_model_name_or_path}' "
OSError: Unable to load weights from pytorch checkpoint file for '/home/zhushaojin/work_dir/20210701_ESC/blenderbot_small-90M' at '/home/zhushaojin/work_dir/20210701_ESC/blenderbot_small-90M/pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

How can I get the correct model for running BlenderEmotionalSupport.py in codes?

Why not "next_strategy_id" used in Blender mode code?

I see "next_strategy_id" is not used in the Emotional-Support-Conversation/codes/src/transformers/models/blenderbot_small/modeling_blenderbot_small.py.

Except for useless code for role and turn id, I can't tell more code differences between codes/src/transformers and huggingface transformers.

That is to say, I can successfully reproduce your result only by data preprocess adding strategy. Right?