Reproducing MolBERT results on QSAR tasks about molbert

[pykao]

Hi @TWRogers ,

I tried to reproduce the QSAR tasks using the script scripts/run_qsar_test_molbert.py but I find it is hard to run CDDD and MolBert in the same script. MolBert is supported by python 3.7 and CDDD is supported by python 3.6. There is also some tensorflow issues such as cuda and version mismatch if i upgrade CDDD to python 3.7. How did you solve this issue?

Best regards,
Po-Yu

[pykao]

@TWRogers ,

I also get poor predictions compared to the MolBert paper. Fine-tuned MolBert sometimes gets worse performance than MolBert without any fine-tuning. Maybe we need to get the details setting for finetuning MolBert.

Best regards,
Po-Yu

[feiyang-cai]

I am using the same setting as @TWRogers using, and also cannot get the reported performance.

Here are some results for the regression tasks from my rerunning,

1. Lipop: 0.710, 0.607, and 0.643 for three seeds, but $0.561 \pm 0.03$ was reported in the paper;
2. ESOL: 0.551, 0.542, and 0.590 for three seeds, but $0.531 \pm 0.04$ was reported in the paper;
3. FreeSolv: 1.286, 0.940, and 1.680 for three seeds, but $0.948 \pm 0.33$ was reported in the paper.