How to evaluate SpanBERT using sample test data? about coref

[jkkummerfeld]

The notebook does not have evaluation code. To add that you will need to look at the contents of this repository and make suitable additions.

On running on more data, I used the colab to run small experiments and am not sure how scaling would go. The problems you are hitting could be due to the limited resources provided for free with Colab. You could take the code and run it on your own server instead, which may also provide more information for debugging.

Unfortunately, I do not have time to help you with these changes, but hopefully the rest of the team / your advisor who tasked you with this can help. Good luck!

[brendanc2122]

The notebook does not have evaluation code. To add that you will need to look at the contents of this repository and make suitable additions.

On running on more data, I used the colab to run small experiments and am not sure how scaling would go. The problems you are hitting could be due to the limited resources provided for free with Colab. You could take the code and run it on your own server instead, which may also provide more information for debugging.

Unfortunately, I do not have time to help you with these changes, but hopefully the rest of the team / your advisor who tasked you with this can help. Good luck!

@jkkummerfeld Hi there, I appreciate your reply, can I get some insight on why there the metrics in the files but unable to produce evaluation results?

Is it possible to run on windows? As the front part of the notebook contains some commands that are not available on windows.

Thank you!

[jkkummerfeld]

Evaluation - The code I wrote does not call the functions that do evaluation. I didn't look at the evaluation set up at all, so I am not sure how it works in this codebase.

Windows - I haven't used Windows for programming in over a decade, so I'm not much help there. Those first few special commands are installing packages. Maybe you can use a virtual machine of some sort? Or ssh into a server that runs linux?