talschuster / feversymmetric Goto Github PK

Hi,

Thanks very much for open-sourcing your code and data.
I found that the provided link of the processed Fever train and eval data is no longer valid. Could you please kindly provide an updated link? I would really appreciate the help.

Best,
Weihang

I am using the version of FEVER dataset that you have processed and released at the following addresses:
https://www.dropbox.com/s/v1a0depfg7jp90f/fever.train.jsonl
https://www.dropbox.com/s/bdwf46sa2gcuf6j/fever.dev.jsonl

The problem is that even though the training data is a three-class dataset, the validation data only includes "Support" and "Refute" samples. Can you please let me know what is the reason? Also, can we use this data as it is (training on three-class data and testing on binary data)?

waiting for reply, thanks!

Dear author,

Thanks for sharing your code. We have traced through the reweighting mechanism in the repository(pytorch_transformers).
In modeling_bert.py, we found the weighted_loss objective function with the hardcoded weights in the input fever training dataset. Unfortunately, we can not find the instance reweighting mechanism with learnable instance weights in both repositories. Can you kindly point us to the right segment in the code that implements this logic in your repository?
Thanks for your help!

Hi Tal,

I want to create symmetric datasets using the guidelines provided in the paper 'Towards debiasing fact verification models'. I want to ask if there are any other available annotator guidelines that we could follow for creating the symmetric datasets from various NLI corpora.

Thanks