sunyilgdx / sifrank Goto Github PK

Thank you for great work!

However, your paper said using macro F1 score. But i know real implementation is micro avg f1 score not macro.
Isn't right macro needs average f1 score each document?

Hi.

Thank you for the great work. I have one question that I hope you could help with.

In the original Embedrank paper, the variant EmbedRank d2v achieved an F1-score of 31.51 in the Inspec dataset (with N = 5). But your paper reported that it only achieved an F1-score of 27.20. Why is there such a difference? Did you run the original code of Embedrank and it only gave an F1-score of 27.20?

Thanks.

Hi Sun Yi,

thank you for sharing your valuable work. I have run the evaluation myself and the results I obtain slightly differ from the ones in your paper.
Below are the evaluation results for SIFRank on the SemEval2017 dataset.

N=5
P=0.4864097363083164
R=0.14057920037519053
F1=0.21811897398581043

N=10
P=0.43448275862068964
R=0.25114315863524445
F1=0.31830002228991755

N=15
P=0.39143979412163077
R=0.3388439441904092
F1=0.3632478632478633
totally cost 501.45456099510193

Could you explain what might be the reason?

Best regards,
Charles

Hi.
Thanks for your great work.
I check the paper which proposed EmbedRank, they use two different ways to generate the embedding: Sent2vec and Doc2vec. I find that the results of EmbedRank Sent2vec are better than Doc2vec on some datasets.
Did you compare EmbedRank Sent2vec on the SemEval2017 dataset?
Thanks
Kawamura

您好，非常感谢您的工作。近期通过替换 SIFRank中的elmo预训练模型（例如electra，bert），做了一些对比实践，但发现差距较大，不知道什么原因。其次如果其他的预训练模型的确表现不如传统的Elmo，那SIFRank方法进一步提升的可能性在什么方面？

Thank you for sharing your code. A quick question: where or how did you get the file enwiki_vocab_min200.txt ? The one that is in the directory auxiliary_data.
Thank you.

你好，
我复现代码的结果与论文里面不太一样，在Semeval2017上面，elmo L0，lamda=0.6，N=5 的结果是f1@5 20.21, 与论文中22.59有一定差距，有可能是什么原因呢？

Hello, thank you for sharing your code. I am trying to run your code, and am stuck in the Usagesection in the following line of code:
en_model = StanfordCoreNLP(‘stanford-corenlp-full-2018-02-27',quiet=True)

The code simply 'hangs' or gets stuck in this line and does not move forward. The execution of this line does not finish even after waiting for a few hours. Can you please provide any help with this? Thank you

Hello!

I try to install SIFRank, but I have error:

"No module named 'allennlp.commands.elmo'"

Do you know, what can I do with it?

I read that the current version of allennlp does not contain the elmo.py file, and whan i try to install previous version, i have error too.

I will be grateful for the help because SIFRank is very interested me.