evaluation question about sifrank

[sunyilgdx]

Sorry, there is an error here, we do use micro-F1(wrong in the paper). The metrics (micro-P-R-F1) we using are following previou works, such as Zhiyuan Liu's works1, 2
I apologize again for troubling you with your reading.

[qute012]

@sunyilgdx
Thanks to reply! It's okay, i can aware from this performance is better than others using macro-avg :D.

[qute012]

@sunyilgdx
I reopened this issue, because there are some mistake i think.
your evaluation metric doesn't consider duplicate candidate. https://arxiv.org/abs/1810.05241, generative model restrain duplicate keyphrase in this paper, but extractive model not.

So It's appeared that model extracted same keyphrase in several times. So It seems like precision is better that other papers. How do you think this?

I tested chunking noun phrases using stanford corenlp if duplicate pharse in document.

And how about using intersection?

Thanks.

[sunyilgdx]

Good question, but since time has passed for a long time, I can't remember some details, let me check it out. But all of our baselines use the same evaluation method and word segmentation tool (Stanford CoreNLP). I can upload the reproduction code of the baseline model first.

[sunyilgdx]

You can check out this code
The function get_final_dist(dist_all, method='average') will filt the duplicate keyphrase.

[qute012]

Thanks to kind reply!

I miss that function, it control duplicate using dictionary keys!