huaxiuyao / lisa Goto Github PK

when I reproduce the code in CMNIST in subpopulation folders. I found a logger bug in subpopulation/confounder_dataset.py, which seemed that the color and class is reverse. I don't know whether the bug is just for CMNIST or for other Dataset too.

Since this is just the logger problems, it may not influence the results for the experiments.

the code link is here logger bug

Dear LISA authors,

Thanks for the impressive work, which inspires me a lot in studying OoD generalization. When I tried to reproduce the performance in FMoW of LISA using the code, I got only 33.49% of Worst group Acc. Perhaps I missed something. I was using 3 random seeds 0, 1, 2, to run the code. May I ask what the 3 random seeds you are using in the paper are? Could you give some hints on what items I might miss when reproducing the results?

BTW, there's a typo in the code, Line 130.

Thank you very much!

Best Regards,
Andrew