ukplab / acl2019-bert-argument-classification-and-clustering Goto Github PK

Hi,
Thanks for sharing the codebase. I am trying to execute the following script: argument-similarity/evaluation_with_clustering.py. However, it is throwing an error at the following lines (the error being no such files are found):

dev_sim_scorer = PairwisePredictionSimilarityScorer("%s/%d/dev_predictions_epoch_%d.tsv" % (bert_experiment, split, epoch))
test_sim_scorer = PairwisePredictionSimilarityScorer("%s/%d/test_predictions_epoch_%d.tsv" % (bert_experiment, split, epoch))

How can I generate these files?

Hi,
thanks for sharing this work.
I tried to run one of the scripts, but seems that datasets are missing, so I take the following error
Traceback (most recent call last):
File "train.py", line 711, in
main()
File "train.py", line 571, in main
train_sampler = RandomSampler(train_data)
File "/home/ant/anaconda3/envs/pytorch_full/lib/python3.7/site-packages/torch/utils/data/sampler.py", line 94, in init
"value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integer value, but got num_samples=0

From where I am supposed to download the data?

Thanks,
Antonis

Hello,
could you kindly clarify which one is the expected ibm dataset? Is this one "Expert Stance" ?
Thank you

Hi,
for UKP ASPECT Corpus there are 4 labels:

DTORCD: Different Topic/Can’t decide
HS : High Similarity
NS : No Similarity 
SS : Some Similarity

Would it be beneficial to use -1,0,1,2 instead?
Is this treated as a regression problem? Where is the transition takes place?

Hi!

I tried to reproduce the performance of Glove Embeddings on the AFS dataset, but I failed.

The Pearson correlation and Spearman's rank correlation are .3240 and .3400 in the original paper, but I got around .15 ~.20 on both metrics with 10-fold CV.

Am I have to weight the embedding of each token by IDF, or follow additional details to reproduce the same result?

I also tried to reproduce the performance of Infersent(both of Glove and FastText) and their performance is same with paper.

Thank you!

Hello,
I find there is no File (make_splits.py)