aylai / gcdc-corpus

Grammarly Corpus of Discourse Coherence and accompanying code for discourse coherence models

Grammarly Corpus of Discourse Coherence (GCDC)

Description of this corpus and the accompanying code can be found in the following paper:

Discourse Coherence in the Wild: A Dataset, Evaluation and Methods
Alice Lai ([email protected]) and Joel Tetreault ([email protected])
Proceedings of the 19th Annual SIGDIAL Meeting on Discourse and Dialogue (SIGDIAL 2018)

GCDC was created in part using the Yahoo Answers corpus: L6 - Yahoo! Answers Comprehensive Questions and Answers version 1.0. The Yahoo Answers corpus can be requested free of charge for research purposes. Access to GCDC (and the accompanying code) will require users to first gain access to this Yahoo Answers corpus.

Once you have gained access to the L6 corpus, please forward the acknowledgment to Joel Tetreault ([email protected]), along with your affiliation and a short description of how you will be using the data, and we will provide access to the Grammarly Corpus of Discourse Coherence and accompanying code. Please let us know if you have any questions.

gcdc-corpus's Issues

Score prediction task

Hi Authors

In GCDC paper, I didn't get the exact procedure for the score prediction. For Spearman’s rank correlation coefficient, we need two inputs (gold_score, label_score), it's mentioned in the paper clearly that 'gold_score' is calculated by taking the mean of expert reviews but how 'label_score' is obtained? I can think of two different approaches as follows:

• use the class label predicted by the trained model (classification models)
• train another linear regression model to obtain the label score with gold_score as training output.

I am not certain whether any of the above methods are used or none of them are used. It will be great if you can clear this up.
Thanks for hearing me out.