j6mes / acl2021-factual-error-correction Goto Github PK

ACL 2021

License: Apache License 2.0

Shell 2.40% Python 97.60%

deep-learning distant-supervision natural-language-generation natural-language-processing natural-language-understanding

acl2021-factual-error-correction's People

Contributors

Stargazers

Watchers

Forkers

anshiquanshu66 judepark96 gruentee lijierui s-abdelnabi teddy309 anmahmood

acl2021-factual-error-correction's Issues

Training on multiple GPUs in parallel

Hello, I am able to run your code, however I do not have access to a TPU.

Do you know if it is possible to perform training of the masker-corrector model on multiple GPUs instead of a single one? Thank you!

Running the code on custom dataset without the FEVER DB file

Hi,

Is it possible to run the masker-corrector module of this code, without using the FEVER sqlite3 database file, in the following code file src/error_correction/modelling/error_correction_module.py?

I have my own dataset with the evidence text already retrieved, so I am hoping to avoid the step of retrieving information from the FEVER database. By any chance, are any intermediate output files generated after text has been retrieved from the FEVER database, that I can look at?

Thank you!

AttributeError while running masker-corrector with heuristic masking

Hi, thank you for sharing your code! While running the masker-corrector model with the following command: bash scripts/finetune_masked.sh t5-base 1e-4 heuristic true false all, I get the below error:

Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/t5-error-correction/src/error_correction/corrector/run.py", line 83, in <module>
    main(args)
  File "/home/ubuntu/t5-error-correction/src/error_correction/corrector/run.py", line 48, in main
    model: ErrorCorrectionModule = ErrorCorrectionModule(args)
  File "/home/ubuntu/t5-error-correction/src/error_correction/modelling/error_correction_module.py", line 65, in __init__
    super().__init__(hparams, num_labels=None, mode=self.mode, **kwargs)
  File "/home/ubuntu/t5-error-correction/src/error_correction/modelling/base_transformer.py", line 67, in __init__
    self.hparams = hparams
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 819, in __setattr__
    object.__setattr__(self, name, value)
AttributeError: can't set attribute

Could you please look into this and tell me where the issue might be? Thank you!

Why is the distantly-supervised corrector trained with "all" labels?

Hi,
Thanks a lot for providing your code.

As far as I understood from the paper, the corrector with distant supervision is trained as an autoencoder to reconstruct the claim [given the supporting evidence], such that it can be used later at test to make the claim consistent with the evidence.

If that is true, then shouldn't the corrector be trained on "Supports" examples only? since the "refutes" are not consistent with the evidence, so they should not be reconstructed as they. Per the instructions to train the masker-corrector, it is trained on all labels.

I hope you could please clarify this confusion.

Unable to run the FEVER task classifier scripts (LIME-based masker)

Hi,

To reproduce an experiments from the scratch that you've done, I was trying to run the LIME-based masker w/ bash scripts following as:

bash scripts/finetune_fever.sh 5e-5 fever_sub genre_50_2 true

finetune_fever.sh

lr=$1
reader=$2
ns=$3
dpr=$4

python -m fec.modelling.finetune_fever \
  --model_name_or_path bert-base-uncased \
  --learning_rate $lr \
  --num_train_epochs 3 \
  --output_dir output/fever/reader=${reader},lr=${lr},data=${dpr},negative_sampling=${ns}/seed=1/  \
  --train_file resources/retrieval/genre_50/${dpr}_train.jsonl \
  --val_file resources/retrieval/genre_50/${dpr}_dev.jsonl \
  --do_train \
  --train_batch_size 16 \
  --eval_batch_size 16 \
  --reader $reader \
  --negative_sampling ${ns} \
  --seed 1

But "ModuleNotFoundError" (fec.modelling.finetune_fever) comes out, and I also have tried to find fec module, but I couldn't find.

If you can, I hope you to check the bash script is wrong or not.

Thank you for your paper and code!

Unable to load a saved model and do predictions

Hi,

I am using the following command to load an already trained model and use it to make predictions on an input test file:

python -m error_correction.corrector.run \
    --model_name_or_path t5-base \
    --output_dir <path_to_trained_model_directory> \
    --do_predict \
    --test_file <path_to_test_file> \
    --train_file <path_to_train_file> \
    --val_file <path_to_val_file> \
    --reader mask \
    --mutation_source true \
    --mutation_target false \
    --labels all

I have to give the parameters train_file and val_file as input, since they are 'required' parameters according to the code. However, on running this command I get the following error:

Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/t5-error-correction/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ubuntu/anaconda3/envs/t5-error-correction/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/t5-error-correction/src/error_correction/corrector/run.py", line 83, in <module>
    main(args)
  File "/home/ubuntu/t5-error-correction/src/error_correction/corrector/run.py", line 74, in main
    trainer.test(ckpt_path=checkpoints[-1])
  File "/home/ubuntu/anaconda3/envs/t5-error-correction/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1281, in test
    results = self.__test_using_best_weights(ckpt_path, test_dataloaders)
  File "/home/ubuntu/anaconda3/envs/t5-error-correction/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1309, in __test_using_best_weights
    model.load_state_dict(ckpt['state_dict'])
  File "/home/ubuntu/anaconda3/envs/t5-error-correction/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1052, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for ErrorCorrectionModule:
	Missing key(s) in state_dict: "model.decoder.block.0.layer.1.EncDecAttention.relative_attention_bias.weight".

Can you please let me know what the issue might be here? Thank you!

Obtaining the "resources/mutation/train.jsonl" file

Hi authors,

Thank you very much for the great work. I am very interested in it. I tried to reproduce the fully supervised setting with gold evidence: https://github.com/j6mes/acl2021-factual-error-correction#fully-supervised-ceiling

using the following script:

bash scripts/finetune_supervised.sh t5-base 1e-4 true false all

In this scripts/finetune_supervised.sh file, it called "resources/mutation/train.jsonl" and "resources/mutation/dev.jsonl" as the training file and validation file respectively. However, these two files are not in the data you shared on google drive.

May I know how should I obtain these two file?

Thank you in advance.

Best regards,
Michael

j6mes / acl2021-factual-error-correction Goto Github PK

acl2021-factual-error-correction's People

Contributors

Stargazers

Watchers

Forkers

acl2021-factual-error-correction's Issues

Training on multiple GPUs in parallel

Running the code on custom dataset without the FEVER DB file

AttributeError while running masker-corrector with heuristic masking

Why is the distantly-supervised corrector trained with "all" labels?

Unable to run the FEVER task classifier scripts (LIME-based masker)

Unable to load a saved model and do predictions

Obtaining the "resources/mutation/train.jsonl" file

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent