Performance Record Documents about chatgpt-sentiment-evaluation

[SinclairCoder]

hi, thanks for your attention and kind words.

Sorry for the confusion. Initially, OpenAI did not release the API of ChatGPT. So we attempt to benchmark its performance using part of the test set. For example, 50_test means we sample 50 samples from the test set. But, on the way, OpenAI released its API of ChatGPT therefore we decided to benchmark it with the full test set. You can refer to our paper for detailed statistics of the test set for each task.

If you have any further questions, please do not hesitate, feel free to leave words.

[Steven-1124]

Thanks for your immediate response! Do you mean that you store the prediction results in 50_test file? And, under different prefix folder, the 50_test record the results performance (not only the measurements index, but the details, like the sentences and the labels predicted) on differnt tasks and different datasets? Then, how about the prediction of other models if 50_test is the results of GPT?

Sorry for the confusion. Initially, OpenAI did not release the API of ChatGPT. So we attempt to benchmark its performance using part of the test set. For example, 50_test means we sample 50 samples from the test set.

It's very glad to hear that! I would check it quickly, but btw, can I access the detail of prediction results (as I mentioned above) from there?

But, on the way, OpenAI released its API of ChatGPT therefore we decided to benchmark it with the full test set. You can refer to our paper for detailed statistics of the test set for each task.

[SinclairCoder]

Hi, 50_test means the set sampling 50 test samples from the full test set, where only the original review sentence and corresponding label are provided.

We recommend rerunning the evaluation experiments for predictions by yourselves. This can be done through OpenAI playground, Chat Window, or using API. You will find that ChatGPT is quite powerful for these tasks, especially some long-tail domains.

[Steven-1124]

Thanks so much! I will check it! It's very helpful talking to you!