After you revised the project last time, I tried to run FTT-NAS search. It was normal at first, but after 10epochs were found, an error was reported when running the test() method... about aw_nas

[walkerning]

Thanks for you information. Since we have added more codes after the FTT-NAS work, It seems like a bug introduced in our later up modification of the shared component classes (the evaluator, and weights manager). I'll check it as soon as possible.

[walkerning]

I have fixed this bug in the latest commit a3834db.

The bug comes from: (If you are interested): In the early code, we aggregate the rewards/perfs. of each batch by mean. Later on, when we are implementing Detection NAS, when implementing mAP reward calculation, this behavior is no longer suitable. Thus, while still keep mean as the default, we permit the objective to provide special handling of the aggregate function of batch rewards. Previously, we check if the length of criterions and number of perfs (len(aggregate_fns)) is the same. However, the third criterion function return multiple perfs! After the results are flattened, the length of criterion results should match the number of perfs.

[BaronNashorMan]

Thank you for your careful hints and explanations. I will continue to run the code according to your instructions. Thank you very much and wish you all the best.