cbaziotis / seq3 Goto Github PK

Source code for the NAACL 2019 paper "SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression"

Python 98.52% Shell 1.48%

abstractive-summarization autoencoder gumbel-softmax sentence-compression seq2seq seq2seq2seq summarization

seq3's People

Contributors

Stargazers

Watchers

Forkers

chl916185 wanghm92 agnibrainhack shibei00 hmxv2 dragomirradev legendtianjin rhoposit liangsi03 zhaoxiaoliang-clh julian-theis davidpqc1231 nthon akaranikolos venescu nlpcode pooooobzx composerlm

seq3's Issues

LM prior checkpoint naming

Hi, thanks for the code.
Just want to confirm, because the seq3 training is looking for lm_giga_articles_all_10K.pt, while the lm prior training saves as lm_prior.pt
I guess I can simply rename it to make it work?
Thanks!

Model size, convergence times, etc.

Hello,

I am trying to reproduce your results, but I can't seem to fit the model in memory. Could you please tell me your hardware specs as well as whether you ran into out-of-memory issues.

Also could you please motivate your choice of recurrent encoder-decoder models instead of the more recent transformer architecture.

Great work, looking forward to hearing from you.

Failed to run seq3 trainer

The file 'utils/eval.py' contains several errors:

rouge.Rouge has no init parameters like: 'max_n', 'limit_length', etc; even very old version doesn't contain these parameters;
rouge metrics has no 'rouge-n', but 'rouge-1', 'rouge-2';

$ python models/seq3.py --config model_configs/seq3.yaml

Traceback (most recent call last):-----------------] Time: 0m 8s (-23m 15s)
  File "models/seq3.py", line 423, in <module>
    train_loss = trainer.train_epoch()
  File "/home/jinzy/work/compression/seq3/modules/training/trainer.py", line 118, in train_epoch
    c(batch, losses, loss_list, batch_outputs)
  File "models/seq3.py", line 368, in eval_callback
    scores = rouge_file_list(config["data"]["ref_path"], hyps)
  File "./utils/eval.py", line 33, in rouge_file_list
    scores = rouge_lists(refs, hyps_list)
  File "./utils/eval.py", line 17, in rouge_lists
    stemming=True)
TypeError: __init__() got an unexpected keyword argument 'max_n'

Why using Glove in compressor instead of pretrained embeddings from LM prior

The compressor and reconstructor are using Glove as their embedding layer. I am curious about why did you use Glove instead of embedding from LM prior.

rouge init max_2 deprecated

max_n=2
limit_length

Total Params: 4.6M
Total Trainable Params: 4.6M
Epoch:1, Batch:100/15800 (1%) [--------------------] Time: 0m 10s (-28m 40s)
Traceback (most recent call last):
  File "models/seq3.py", line 423, in <module>
    train_loss = trainer.train_epoch()
  File "/users/jwilliams/code/seq3/modules/training/trainer.py", line 118, in train_epoch
    c(batch, losses, loss_list, batch_outputs)
  File "models/seq3.py", line 368, in eval_callback
    scores = rouge_file_list(config["data"]["ref_path"], hyps)
  File "./utils/eval.py", line 33, in rouge_file_list
    scores = rouge_lists(refs, hyps_list)
  File "./utils/eval.py", line 17, in rouge_lists
    stemming=True)
TypeError: __init__() got an unexpected keyword argument 'limit_length'

cbaziotis / seq3 Goto Github PK

seq3's People

Contributors

Stargazers

Watchers

Forkers

seq3's Issues

LM prior checkpoint naming

Model size, convergence times, etc.

Failed to run seq3 trainer

Why using Glove in compressor instead of pretrained embeddings from LM prior

rouge init max_2 deprecated

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent