We organize an ECAC evaluation, namely Multimodal Emotion Cause Analysis in Conversations, as a shared task of SemEval-2024.
🎉🎉🎉 Our task paper is available here. Please cite it according to the official format.
🔥🔥🔥 Our CodaLab Competition website is available!
🌟🌟🌟 Interested participants are encouraged to join our Google Group to stay updated.
Based on the multimodal conversational emotion cause dataset ECF, we define the following two subtasks:
In this subtask, an emotion cause is defined and annotated as a textual span.
- Input: a conversation containing the speaker and the text of each utterance.
- Output: all emotion-cause pairs, where each pair contains an emotion utterance along with its emotion category and the textual cause span in a specific cause utterance, e.g., (U3_Joy, U2_"You made up!").
It should be noted that sometimes the cause can not be reflected in text only, and we accordingly propose a multimodal subtask to extract emotion cause in all three modalities (language, audio, and vision). For example, the cause for Phoebe’s Disgust in U5 is that Monica and Chandler were kissing in front of her, which is reflected in the visual modality of U5. In this case, cause is defined and annotated at the utterance level.
- Input: a conversation including the speaker, text, and audio-visual clip for each utterance.
- Output: all emotion-cause pairs, where each pair contains an emotion utterance along with its emotion category and a cause utterance, e.g., (U5_Disgust, U5).
🔔 Please note that the use of additional annotation data is not allowed for ECAC. However, we encourage participants to utilize publicly available Large Language Models, including ChatGPT, during the system development and evaluation phases.
- Training data.
- Trial data for the Practice phase on CodaLab
- Evaluation data.
📢 Trial data and evaluation data contain some noise instances that will not be evaluated.
📢 Our dataset is also available at Zenodo.
- Similar to the previous works, we evaluate the emotion-cause pairs of each emotion category with F1 scores separately and further calculate a weighted average of F1 scores across the six emotion categories (Anger, Disgust, Fear, Joy, Sadness and Surprise).
- For Subtask 1 which involves the textual cause span, we adopt two strategies to determine whether the span is extracted correctly: Strict Match (the predicted span should be exactly the same as the annotated span) and Proportional Match (considering the overlap proportion between the predicted span and the annotated one).
📢 You can find the details of the evaluation metrics on GitHub.
Our CodaLab Competition website is available! Please refer to the official instructions on how to participate in the competition. After registering for our competition on Codalab, please fill in your registered user information on the online form.
| Event | Date |
|---|---|
Note: All deadlines are 23:59 UTC-12 (AoE).
@ARTICLE{wang2023multimodal,
author={Wang, Fanfan and Ding, Zixiang and Xia, Rui and Li, Zhaoyu and Yu, Jianfei},
journal={IEEE Transactions on Affective Computing},
title={Multimodal Emotion-Cause Pair Extraction in Conversations},
year={2023},
volume={14},
number={3},
pages={1832-1844},
doi = {10.1109/TAFFC.2022.3226559}
}
@InProceedings{wang2024SemEval,
author={Wang, Fanfan and Ma, Heqing and Xia, Rui and Yu, Jianfei and Cambria, Erik},
title = "{S}em{E}val-2024 Task 3: Multimodal Emotion Cause Analysis in Conversations",
booktitle = "Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)",
month = jun,
year = "2024",
address = "Mexico City, Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.semeval-1.277",
pages = "2039--2050",
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent