eternityyw / metacognitive-prompting Goto Github PK

Hi!
Thank you for the excellent work.

I have a question regarding the Confidence Analysis of this paper. The definition of the confusion matrix in the main text was as follows:

Within this matrix, the standard terminologies of ‘True Positive’, ‘False Positive’, ‘True Negative’, and ‘False Negative’ are redefined as follows:

• True Positive (TP): Represents instances where the model, using MP, expressed high confidence and produced a correct answer. These account for 58.3%.
• False Positives (FP): Denotes cases where the model exhibited high confidence but gave an incorrect prediction. These amount to 5.9%.
• True Negatives (TN): Refers to instances where the model signaled low confidence and its response was indeed incorrect. These stand at 27.1%.
• False Negatives (FN): Highlights cases where the model indicated low confidence but, surprisingly, delivered a correct answer. These tally to 8.7%.

However, isn't this different from the results in Figure 5?

• FP is 27.1% instead of 5.9%
• TN is 8.7% instead of 27.1%
• FN is 5.9% instead of 8.7%
  Is that correct?

Thank you again for the remarkable work!