s-sd / task-amenability Goto Github PK

Hi Shaheer, thanks for proposing such pioneering works.

I have some concerns about your training process. I can not find the codes for selecting or rejecting samples with high task amenability to retrain the actor (task predictor). I could only find similar codes for evaluating/testing in L125 of pneumoniamnist_experiment.py. Could you please indicate where similar codes for selecting and retraining are?

Best.

Hello, I have followed this work and got a poor result using the PneumoniaMNIST dataset.
due to the absence of preprocessed files, I write my own preprocess algorithm. to keep the consistency in train/val/holdout, each of them has the same proportion of corrupted images(24/524). There are 4 types of corrupting in my algorithm:

1. fully corrupt(100%) with value 255
2. half corrupt(the top half) with value 255
3. randomly corrupt(50%) with value 255
4. randomly corrupt(1/28) with value 255

After doing that, I get the npz file. Ecerything else keeps the same. However the controller predict 1 for all samples in holdout dataset, while there are about 28 samples which are corrupted. it seems controller predicts poorly about the image quality.
By the way, my npz file is 4493 kb, but the file in your repo is 11684 kb.