ruocwang / darts-pt Goto Github PK

For evaluation experiments on S1-S4, we are supposed to run:
'bash eval.sh --arch [genotype_name]'
What is genotype_name here?
Suppose I got genotype as
Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3',2)], reduce_concat=range(2, 6))
How to map this to genotype name?

Hi Ruochen, while I was learning your code, some errors came upcode, some errors came up before a new epoch started training. before a new epoch started training.

Please give me some tips about this, thank you~

Hi, thank you for sharing. I have a question about whether you have done the search for RNN? If it has been done, what result can it get?

Could you please state the difference and reason you choose the older version.

Hello!
First of all I would like to thank you both for the very clear paper, and for provision a code - does not often happen -.
I have a doubt about the pipeline to follow to run a complete NAS algorithm.

As far as I understood, the first step is for sure to run train_search.py with specified parameters. After this, what should I do?
I say there is a script darts-proj-sota.sh that calls the file projection.py which is not clear to me what it does, or to which step of NAS it corresponds.
Is that step mandatory?
Then, how to build the final network to train from scratch and evaluate? I saw that after calling darts-proj-sota.sh there's the final output that is a Genotype. I guess I need to insert that genotype in the python file genotype.py and run train.py, but I'm not sure of what's going on, if projection.py is mandatory step etc.

I hope I expressed my doubts clearly, if you could provide some more hints on a complete pipeline to follow it would be great.
Thanka a lot!

darts-pt/nasbench201/train_search.py

Line 394 in bf3ffdd

result = api.query_by_arch(genotype)

Here seems missing argument "hp" in api.query_by_arch() according to

https://github.com/D-X-Y/NAS-Bench-201/blob/8558547969c131f75af2725869ff1ece98e98f23/nas_201_api/api_201.py#L118`

Fixed after adding hp argument with 12 or 200
result = api.query_by_arch(genotype, hp='200')

Thanks for your work!
I ran DARTS-PT on NAS-Bench-201(cifar10) with darts-201.sh and darts-proj-201.sh, but I didn't get the hight accuracy in the paper. The accuracy I got was 85.03. Is this result normal? If not, can you give me some advice?

Are the sota result on CIFAR10 evaluate under --init_channels 36 --layer 20 or same as c100 at --init_channels 16 --layers 8

Hi, thanks for the good work, and for the performance in subspace. Are you run each experiment four times with 4 random seeds and select the best to report? Just as what previous work did? This part not be illustrated on paper, just wanna check if it's right.