sheffieldnlp / amr-preprocessing Goto Github PK

Create a new parameter and pipeline to remove all concept's semantics identifier

We would like to know whether the prediction actually performs better when beginning and ending parentheses of every sentence are discarded.

We need to perform tokenization for every sentence file.

• Concatenate all body document of each ID as one line.
• One file for body documents and one file for summaries. Duplicate body document if there are more than one summary for each ID

The filtering didn't go well and producing an empty output.

This setting is specific to the Proxy Report.

• Setup one folder named no-side, and another folder named side. The no-side will be input folder for the baseline with no side information, where the side will be input folder for our approach and the baseline with side information.
  • For the no-side folder, only a single file consisting of all summary.
  • For the side folder, contain documents of which comprise:
    • A summary sentence and its AMR.
    • A file consisting of the document body sentences and another file consisting of each of the document body sentence's AMR.

Create a new preprocessing and parameter pipeline for removing all parentheses

This pre-training dataset will only consist of Proxy's summary and non-Proxy dataset. A new output filename would be required to avoid overwriting the normal pre-training dataset.