amazing-j / structural-transformer Goto Github PK

Hi,
Thanks for your meaningful works, i'm sorry to ask you about how to preprocess our data, which consist of the amr parsing result by using "stog", the sample as follows. In your readme, you said that we need linearied the amr graphy, but i don't konw how to make it. So, i hope you can give me some details about achieving this. Thanks!

# ::id 0_0
# ::tok Good afternoon . This is Michelle Li speaking , calling on behalf of IBA . Is Mr Meng available at all ?
# ::abstract_map {}
(c0 / multi-sentence
      :snt1 (c1 / afternoon
            :ARG1-of (c4 / good-02)
            :domain (c5 / this))
      :snt2 (c2 / speak-01
            :ARG0 (c6 / person)
            :topic (c7 / call-02
                  :ARG0 c6
                  :ARG2 (c10 / i)))
      :snt3 (c3 / possible-01
            :ARG1 (c8 / available-02
                  :mod (c11 / at-all))
            :mode interrogative))

Hi @Amazing-J,

I tried to replicate the results of the baseline, feature-based and CNN/SA models. Here's the results that I got (I'm not sure which BLEU score you use, so I include a few of them

For the LDC2015E86
Baseline
Dev
Bleu_1: 0.495188
Bleu_2: 0.353920
Bleu_3: 0.261369
Bleu_4: 0.196578
METEOR: 0.309095
Test
Bleu_1: 0.504782
Bleu_2: 0.349076
Bleu_3: 0.251628
Bleu_4: 0.184996
METEOR: 0.307734

Features
Validation
Bleu_1: 0.482722
Bleu_2: 0.327856
Bleu_3: 0.228882
Bleu_4: 0.161975
METEOR: 0.297407
Test
Bleu_1: 0.488859
Bleu_2: 0.334304
Bleu_3: 0.236498
Bleu_4: 0.170757
METEOR: 0.297764

SA
Validation
Bleu_1: 0.524293
Bleu_2: 0.379137
Bleu_3: 0.282884
Bleu_4: 0.214684
METEOR: 0.327192
Test
Bleu_1: 0.531576
Bleu_2: 0.389128
Bleu_3: 0.295101
Bleu_4: 0.228584
METEOR: 0.327300

CNN
Validation
Bleu_1: 0.495188
Bleu_2: 0.353920
Bleu_3: 0.261369
Bleu_4: 0.196578
METEOR: 0.309095
Test
Bleu_1: 0.496904
Bleu_2: 0.355405
Bleu_3: 0.262857
Bleu_4: 0.198174
METEOR: 0.306878

The results are much worse for the LDC2017T10 dataset
Baseline
Validation
Bleu_1: 0.204362
Bleu_2: 0.105597
Bleu_3: 0.062237
Bleu_4: 0.038863
METEOR: 0.110788
Test
Bleu_1: 0.197050
Bleu_2: 0.104745
Bleu_3: 0.064901
Bleu_4: 0.042817
METEOR: 0.108762

Features
Validation
Bleu_1: 0.151187
Bleu_2: 0.079275
Bleu_3: 0.047097
Bleu_4: 0.030196
METEOR: 0.088049
Test
Bleu_1: 0.156906
Bleu_2: 0.084370
Bleu_3: 0.052182
Bleu_4: 0.034406
METEOR: 0.088722

SA
Validation
Bleu_1: 0.216386
Bleu_2: 0.125283
Bleu_3: 0.079592
Bleu_4: 0.052227
METEOR: 0.123734
Test
Bleu_1: 0.203880
Bleu_2: 0.123374
Bleu_3: 0.082714
Bleu_4: 0.058262
METEOR: 0.119108

CNN
Validation
Bleu_1: 0.234239
Bleu_2: 0.150478
Bleu_3: 0.103202
Bleu_4: 0.073315
METEOR: 0.146379
Test
Bleu_1: 0.228974
Bleu_2: 0.150096
Bleu_3: 0.103783
Bleu_4: 0.073553
METEOR: 0.144331

I followed all the step listed on README as well as the answers from the repo issues. Am I missing something here?

Also, can you share the vocab size and the sequence length you used for the LDC2017T10 dataset? There are only numbers for the LDC2015E86 dataset according to the code.

Hi,

I often ran into the following error when starting a multi-GPU training.

Traceback (most recent call last):
  File "train.py", line 116, in <module>
    main(opt)
  File "train.py", line 44, in main
    p.join()
  File "/home/xfbai/anaconda3/envs/torch1.0/lib/python3.6/multiprocessing/process.py", line 124, in join
    res = self._popen.wait(timeout)
  File "/home/xfbai/anaconda3/envs/torch1.0/lib/python3.6/multiprocessing/popen_fork.py", line 50, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
  File "/home/xfbai/anaconda3/envs/torch1.0/lib/python3.6/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
TypeError: signal_handler() takes 1 positional argument but 3 were given

The parameters I used are:

CUDA_VISIBLE_DEVICES=0,1 python3 train.py \
                        -data $data_prefix \
                        -save_model $model_dir \
                        -world_size 2 \
                        -gpu_ranks 0 1 \
                        -save_checkpoint_steps 5000 \
                        -valid_steps 5000 \
                        -report_every 20 \
                        -keep_checkpoint 50 \
                        -seed 3435 \
                        -train_steps 300000 \
                        -warmup_steps 16000 \
                        --share_decoder_embeddings \
                        -share_embeddings \
                        --position_encoding \
                        --optim adam \
                        -adam_beta1 0.9 \
                        -adam_beta2 0.98 \
                        -decay_method noam \
                        -learning_rate 0.5 \
                        -max_grad_norm 0.0 \
                        -batch_size 4096 \
                        -batch_type tokens \
                        -normalization tokens \
                        -dropout 0.3 \
                        -label_smoothing 0.1 \
                        -max_generator_batches 100 \
                        -param_init 0.0 \
                        -param_init_glorot \
                        -valid_batch_size 8

I got this error on Ubuntu16.04, Python3.6, Pytorch 1.0.1. Can someone help me understand what's the cause of it? I would really appreciate your help, thank you!

Hi @Amazing-J ,

Again thank you for releasing the code. I have a question regarding the sub-word AMR.

In case the original word doesn't have an in-coming edge, how should we encode the AMR for the corresponding subword, e.g., the word "adjust" => "ad@@ ju@@ st" in the following example?

(a / ad@@ ju@@ st-01 :ARG0 (g / girl@@ ) :ARG1 (m / mach@@ ine))

Thank you!

Hi,

I am trying to reproduce the results of the baseline NMT system in your paper, however, I found the examples given in Readme and that in corpus_example are not consistent. (e.g. different from the example in Readme, the :wiki link here is not removed, even BPE has been applied).

Now I'm confused about which one I should follow, any suggestions :)?

Hi,

Could you please release the preprocessing codes for generating the structural sequence and the commands for applying bpe? i.e., how to get the files in corpus_sample/all_path_corpus and corpus_sample/five_path_corpus.

Thanks.

Hi, thanks for the great work!

I try to run the code. However, I don't know how to do data preprocessing for AMR corpus. May I ask how can I do data preprocessing?

Hi,

I am trying to reproduce the result of your baseline model. I find that your tokenized target texts seems to be different from previous works (e.g., https://github.com/freesunshine0316/neural-graph-to-seq-mp). In your corpus_sample, the first target sentence in test set is "2007-08-21" while the same sentence aften tokenizing in prior works is "2007 - 08 - 21".

If your reference texts (gold answer) are different from prior works, the BLEU score of your system is not strictly comparable.

I would really appreciate it if you can explain it. Thank you!