experiment.evaluate() shows stale evaluation results about prompttools

[NivekT]

Your observation is correct. Currently, if a metric already exists (which is "similar_to_expected" in your case), it raises a warning (as seen in your notebook "WARNING: similar_to_expected is already present, skipping") rather than overwriting it.

If you change the metric name given in the second .evaluate call (i.e. experiment.evaluate("similar_to_expected_2", ...)), it will compute another column.

We are open to considering overwriting it even when the existing metric already exists. Let us know what you think.

[Sruthi5797]

Thank you for this issue, I changed the variable name but still, the response column is stale. Any leads on this issue? I use python version 3.11.5

[NivekT]

Hi @Sruthi5797,

Can you post a minimal code snippet of what you are running? Also, are you seeing any warning message?