This repo modifies the original repo for ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training.
Link for original repo: https://github.com/microsoft/ProphetNet
Tested with Python 3.7 via virtual environment.
Clone the repo, go to the repo folder, setup the virtual environment, and install the required packages:
$ python3.7 -m venv venv
$ source venv/bin/activate
$ pip install -r requirements.txt
Update: We can also use Metaflow to execute the whole process, by running:
$ python prophetnet-metaflow.py run
The data can be downloaded here. To run this program, you need to put the data files into the following directory: /data/cnndm_data/yourdata.zip
.
Also you can directly run the following command to download the data.
$ mkdir data
$ cd data
$ mkdir cnndm_data
$ cd cnndm_data
$ gdown --id 1jiDbDbAsqy_5BM79SmX6aSu5DQVCAZq1 --output data.zip
$ unzip data.zip
After downloading the data, firstly we run preprocess_cnn_dm.py
to tokenize CNN/DailyMail data.
$ python preprocess_cnn_dm.py
Then we need to generate the binary data files.
$ chmod +x cnndm-data-preprocess.sh
$ ./cnndm-data-preprocess.sh
$ chmod +x cnndm-fine-tuning.sh
$ ./cnndm-fine-tuning.sh
To generate the result:
$ chmod +x cnndm-generate.sh
$ ./cnndm-generate.sh
To evaluate the result
$ chmod +x cnndm-evaluation.sh
$ /cnndm-evaluation.sh
Here is the evaluation result I got for CNN/DM dataset:
---------------------------------------------
1 ROUGE-1 Average_R: 0.37606 (95%-conf.int. 0.37373 - 0.37846)
1 ROUGE-1 Average_P: 0.49294 (95%-conf.int. 0.48986 - 0.49598)
1 ROUGE-1 Average_F: 0.41450 (95%-conf.int. 0.41220 - 0.41674)
---------------------------------------------
1 ROUGE-2 Average_R: 0.18094 (95%-conf.int. 0.17857 - 0.18315)
1 ROUGE-2 Average_P: 0.24029 (95%-conf.int. 0.23736 - 0.24321)
1 ROUGE-2 Average_F: 0.20032 (95%-conf.int. 0.19788 - 0.20275)
---------------------------------------------
1 ROUGE-L Average_R: 0.35153 (95%-conf.int. 0.34924 - 0.35386)
1 ROUGE-L Average_P: 0.46082 (95%-conf.int. 0.45789 - 0.46372)
1 ROUGE-L Average_F: 0.38751 (95%-conf.int. 0.38526 - 0.38971)
>> ROUGE-F(1/2/l): 41.45/20.03/38.75
ROUGE-R(1/2/3/l): 37.61/18.09/35.15
If you meet any problems related to pyrouge
, you can refer to this useful solution.