ytongbai / vits-vs-cnns Goto Github PK

Cannot create DeiT Small model.

I have timm=0.3.2 installed. I try to evaluate DeiT Small on ImageNet and get the following error.

[xxx@xxx vit-vs-cnn]$ CUDA_VISIBLE_DEVICES=0 python imagenet_robust.py -a 'deit_small_patch16_224' --test-batch 256  --data data_path  --evaluate --imagenet-a --ckpt ckpt/deitsmall.pth --sing singln
Traceback (most recent call last):
  File "/home/research/domain_sim/vit-vs-cnn/imagenet_robust.py", line 394, in <module>
    main()
  File "/home/research/domain_sim/vit-vs-cnn/imagenet_robust.py", line 233, in main
    model = create_model(
  File "/home/anaconda3/envs/domain2/lib/python3.9/site-packages/timm/models/factory.py", line 59, in create_model
    raise RuntimeError('Unknown model (%s)' % model_name)
RuntimeError: Unknown model (deit_small_patch16_224)

When I use your code of autoatteck, the output I get is very different, and the program will be killed by them self. After output the APGD-CE accuracy, the terminal output Killed.
the part output of mine :

apgd-ce - 1/147 - 256 out of 256 successfully perturbed
0.74022
apgd-ce - 2/147 - 256 out of 256 successfully perturbed
0.7351
apgd-ce - 3/147 - 256 out of 256 successfully perturbed
0.72998
apgd-ce - 4/147 - 256 out of 256 successfully perturbed
0.72486
apgd-ce - 5/147 - 256 out of 256 successfully perturbed
0.71974
apgd-ce - 6/147 - 256 out of 256 successfully perturbed
0.71462
apgd-ce - 7/147 - 256 out of 256 successfully perturbed
0.7095
apgd-ce - 8/147 - 256 out of 256 successfully perturbed
0.70438
apgd-ce - 9/147 - 256 out of 256 successfully perturbed
0.69926
apgd-ce - 10/147 - 256 out of 256 successfully perturbed
0.69414
apgd-ce - 11/147 - 256 out of 256 successfully perturbed
0.68902
apgd-ce - 12/147 - 256 out of 256 successfully perturbed
0.6839
apgd-ce - 13/147 - 256 out of 256 successfully perturbed
0.67878
apgd-ce - 14/147 - 256 out of 256 successfully perturbed
0.67366
apgd-ce - 15/147 - 256 out of 256 successfully perturbed

the follow is the part output of yours(autoattack/log)

apgd-ce - 1/65 - 193 out of 512 successfully perturbed
0.65892
apgd-ce - 2/65 - 116 out of 512 successfully perturbed
0.65402
apgd-ce - 3/65 - 245 out of 512 successfully perturbed
0.64836
apgd-ce - 4/65 - 283 out of 512 successfully perturbed
0.64416
apgd-ce - 5/65 - 210 out of 512 successfully perturbed
0.6403
apgd-ce - 6/65 - 193 out of 512 successfully perturbed
0.63828
apgd-ce - 7/65 - 101 out of 512 successfully perturbed
0.6342
apgd-ce - 8/65 - 204 out of 512 successfully perturbed
0.62892
apgd-ce - 9/65 - 264 out of 512 successfully perturbed
0.62626

Thanks for your reply

Hi, Thanks for this excellent work!
You stated in your paper "with the constrain that maximum per-pixel change epsilon = 4/255".
However, in this implementation, the argument attack-epsilon (4) is multiplied by an Image Scale which is 2/255, and the real epsilon should be 8/255 instead of 4/255?

Hi, as you suggested in the earlier thread, we tried with the exact same settings and argument that you provided to do train on deit tiny adversarially. However, it is still not working, only 18% accuracy after 52 epochs.

{"train_lr": 9.999999999999953e-07, "train_loss": 6.91152723201459, "test_0_loss": 6.872483930142354, "test_0_acc1": 0.26, "test_0_acc5": 1.12, "test_5_loss": 6.9177000566849856, "test_5_acc1": 0.1805, "test_5_acc5": 0.6765, "epoch": 0, "n_parameters": 5717416}
{"train_lr": 9.999999999999953e-07, "train_loss": 6.900039605433039, "test_0_loss": 6.850005263940539, "test_0_acc1": 0.362, "test_0_acc5": 1.578, "test_5_loss": 6.9177000566849856, "test_5_acc1": 0.1805, "test_5_acc5": 0.6765, "epoch": 1, "n_parameters": 5717416}
{"train_lr": 0.00040089999999999305, "train_loss": 6.754413659242894, "test_0_loss": 6.145018349987379, "test_0_acc1": 3.726, "test_0_acc5": 10.93, "test_5_loss": 6.9177000566849856, "test_5_acc1": 0.1805, "test_5_acc5": 0.6765, "epoch": 2, "n_parameters": 5717416}
{"train_lr": 0.0008008000000000196, "train_loss": 6.625597798519379, "test_0_loss": 5.638628653967449, "test_0_acc1": 7.224, "test_0_acc5": 18.596, "test_5_loss": 6.9177000566849856, "test_5_acc1": 0.1805, "test_5_acc5": 0.6765, "epoch": 3, "n_parameters": 5717416}
{"train_lr": 0.001200700000000036, "train_loss": 6.560625480233337, "test_0_loss": 5.315861636678607, "test_0_acc1": 10.064, "test_0_acc5": 23.916, "test_5_loss": 6.9177000566849856, "test_5_acc1": 0.1805, "test_5_acc5": 0.6765, "epoch": 4, "n_parameters": 5717416}
{"train_lr": 0.0016005999999999657, "train_loss": 6.54946454628099, "test_0_loss": 5.134192888048774, "test_0_acc1": 11.984, "test_0_acc5": 27.634, "test_5_loss": 5.716198673022533, "test_5_acc1": 6.383, "test_5_acc5": 16.22425, "epoch": 5, "n_parameters": 5717416}
{"train_lr": 0.0020004999999999914, "train_loss": 6.542691466810225, "test_0_loss": 5.129832990186304, "test_0_acc1": 12.648, "test_0_acc5": 28.482, "test_5_loss": 5.716198673022533, "test_5_acc1": 6.383, "test_5_acc5": 16.22425, "epoch": 6, "n_parameters": 5717416}
{"train_lr": 0.002400399999999976, "train_loss": 6.536189370113406, "test_0_loss": 5.018598655821495, "test_0_acc1": 13.942, "test_0_acc5": 31.118, "test_5_loss": 5.716198673022533, "test_5_acc1": 6.383, "test_5_acc5": 16.22425, "epoch": 7, "n_parameters": 5717416}
{"train_lr": 0.002800300000000059, "train_loss": 6.503189501430777, "test_0_loss": 4.9134670785048495, "test_0_acc1": 15.256, "test_0_acc5": 33.352, "test_5_loss": 5.716198673022533, "test_5_acc1": 6.383, "test_5_acc5": 16.22425, "epoch": 8, "n_parameters": 5717416}
{"train_lr": 0.0032002000000000983, "train_loss": 6.5094980549373975, "test_0_loss": 4.851811613246408, "test_0_acc1": 15.578, "test_0_acc5": 34.102, "test_5_loss": 5.716198673022533, "test_5_acc1": 6.383, "test_5_acc5": 16.22425, "epoch": 9, "n_parameters": 5717416}
{"train_lr": 0.0036000999999999585, "train_loss": 6.511975479259384, "test_0_loss": 5.143745462633598, "test_0_acc1": 13.898, "test_0_acc5": 30.834, "test_5_loss": 5.7486809707954, "test_5_acc1": 7.313, "test_5_acc5": 17.87725, "epoch": 10, "n_parameters": 5717416}
{"train_lr": 0.0039023577500088853, "train_loss": 6.418486285981515, "test_0_loss": 5.218988336208, "test_0_acc1": 12.054, "test_0_acc5": 27.72, "test_5_loss": 5.7486809707954, "test_5_acc1": 7.313, "test_5_acc5": 17.87725, "epoch": 11, "n_parameters": 5717416}
{"train_lr": 0.003882057134063648, "train_loss": 6.32594983450038, "test_0_loss": 5.0386335921455325, "test_0_acc1": 14.276, "test_0_acc5": 31.454, "test_5_loss": 5.7486809707954, "test_5_acc1": 7.313, "test_5_acc5": 17.87725, "epoch": 12, "n_parameters": 5717416}
{"train_lr": 0.0038599040893470705, "train_loss": 6.353343641300567, "test_0_loss": 4.799374374913163, "test_0_acc1": 16.064, "test_0_acc5": 34.754, "test_5_loss": 5.7486809707954, "test_5_acc1": 7.313, "test_5_acc5": 17.87725, "epoch": 13, "n_parameters": 5717416}
{"train_lr": 0.0038359204782395374, "train_loss": 6.200804713199274, "test_0_loss": 5.33436276099656, "test_0_acc1": 11.77, "test_0_acc5": 26.914, "test_5_loss": 5.7486809707954, "test_5_acc1": 7.313, "test_5_acc5": 17.87725, "epoch": 14, "n_parameters": 5717416}
{"train_lr": 0.0038101299696696937, "train_loss": 6.372054211956134, "test_0_loss": 5.132588769103652, "test_0_acc1": 14.968, "test_0_acc5": 32.62, "test_5_loss": 5.900749441910766, "test_5_acc1": 6.776, "test_5_acc5": 16.361, "epoch": 15, "n_parameters": 5717416}
{"train_lr": 0.0037825580157558377, "train_loss": 6.417677939938699, "test_0_loss": 4.666486162294277, "test_0_acc1": 15.988, "test_0_acc5": 34.454, "test_5_loss": 5.900749441910766, "test_5_acc1": 6.776, "test_5_acc5": 16.361, "epoch": 16, "n_parameters": 5717416}
{"train_lr": 0.0037532318266873923, "train_loss": 6.26336412826221, "test_0_loss": 5.38978447795143, "test_0_acc1": 12.402, "test_0_acc5": 28.042, "test_5_loss": 5.900749441910766, "test_5_acc1": 6.776, "test_5_acc5": 16.361, "epoch": 17, "n_parameters": 5717416}
{"train_lr": 0.003722180343872929, "train_loss": 6.142464170686537, "test_0_loss": 4.77877281021782, "test_0_acc1": 15.232, "test_0_acc5": 33.184, "test_5_loss": 5.900749441910766, "test_5_acc1": 6.776, "test_5_acc5": 16.361, "epoch": 18, "n_parameters": 5717416}
{"train_lr": 0.0036894342113766073, "train_loss": 6.004803928301679, "test_0_loss": 5.506222593730944, "test_0_acc1": 10.49, "test_0_acc5": 24.774, "test_5_loss": 5.900749441910766, "test_5_acc1": 6.776, "test_5_acc5": 16.361, "epoch": 19, "n_parameters": 5717416}
{"train_lr": 0.0036550257456777735, "train_loss": 6.135914517725877, "test_0_loss": 5.406659782199775, "test_0_acc1": 11.604, "test_0_acc5": 27.232, "test_5_loss": 6.950556825081355, "test_5_acc1": 2.013, "test_5_acc5": 5.54425, "epoch": 20, "n_parameters": 5717416}
{"train_lr": 0.0036189889037779527, "train_loss": 6.093383449254086, "test_0_loss": 5.010170250158621, "test_0_acc1": 13.934, "test_0_acc5": 30.648, "test_5_loss": 6.950556825081355, "test_5_acc1": 2.013, "test_5_acc5": 5.54425, "epoch": 21, "n_parameters": 5717416}
{"train_lr": 0.0035813592496895356, "train_loss": 6.059879529152176, "test_0_loss": 5.371730933796497, "test_0_acc1": 10.762, "test_0_acc5": 25.292, "test_5_loss": 6.950556825081355, "test_5_acc1": 2.013, "test_5_acc5": 5.54425, "epoch": 22, "n_parameters": 5717416}
{"train_lr": 0.00354217391933767, "train_loss": 6.073066240544323, "test_0_loss": 5.048453500617107, "test_0_acc1": 13.242, "test_0_acc5": 29.126, "test_5_loss": 6.950556825081355, "test_5_acc1": 2.013, "test_5_acc5": 5.54425, "epoch": 23, "n_parameters": 5717416}
{"train_lr": 0.0035014715839127445, "train_loss": 6.08733743352951, "test_0_loss": 5.331494154414533, "test_0_acc1": 10.974, "test_0_acc5": 25.072, "test_5_loss": 6.950556825081355, "test_5_acc1": 2.013, "test_5_acc5": 5.54425, "epoch": 24, "n_parameters": 5717416}
{"train_lr": 0.003459292411705762, "train_loss": 6.1378554502646505, "test_0_loss": 5.264302222605172, "test_0_acc1": 10.784, "test_0_acc5": 25.304, "test_5_loss": 7.786235228686171, "test_5_acc1": 0.8985, "test_5_acc5": 2.83875, "epoch": 25, "n_parameters": 5717416}
{"train_lr": 0.0034156780284671424, "train_loss": 6.1356256470786965, "test_0_loss": 5.568371077798074, "test_0_acc1": 8.102, "test_0_acc5": 19.622, "test_5_loss": 7.786235228686171, "test_5_acc1": 0.8985, "test_5_acc5": 2.83875, "epoch": 26, "n_parameters": 5717416}
{"train_lr": 0.003370671476327724, "train_loss": 6.071214107396029, "test_0_loss": 5.589009375886435, "test_0_acc1": 9.484, "test_0_acc5": 22.578, "test_5_loss": 7.786235228686171, "test_5_acc1": 0.8985, "test_5_acc5": 2.83875, "epoch": 27, "n_parameters": 5717416}
{"train_lr": 0.00332431717132075, "train_loss": 5.972238908568732, "test_0_loss": 5.707102177010388, "test_0_acc1": 7.978, "test_0_acc5": 19.746, "test_5_loss": 7.786235228686171, "test_5_acc1": 0.8985, "test_5_acc5": 2.83875, "epoch": 28, "n_parameters": 5717416}
{"train_lr": 0.0032766608595485736, "train_loss": 6.1025936421539955, "test_0_loss": 5.42314580428013, "test_0_acc1": 11.556, "test_0_acc5": 26.76, "test_5_loss": 7.786235228686171, "test_5_acc1": 0.8985, "test_5_acc5": 2.83875, "epoch": 29, "n_parameters": 5717416}
{"train_lr": 0.0032277495720376523, "train_loss": 6.084920492580087, "test_0_loss": 5.923481827123914, "test_0_acc1": 6.536, "test_0_acc5": 16.768, "test_5_loss": 7.121632807848168, "test_5_acc1": 1.17125, "test_5_acc5": 3.44575, "epoch": 30, "n_parameters": 5717416}
{"train_lr": 0.003177631578323426, "train_loss": 6.115835655793298, "test_0_loss": 5.460594815469596, "test_0_acc1": 9.702, "test_0_acc5": 23.214, "test_5_loss": 7.121632807848168, "test_5_acc1": 1.17125, "test_5_acc5": 3.44575, "epoch": 31, "n_parameters": 5717416}
{"train_lr": 0.003126356338814952, "train_loss": 6.0647800623846475, "test_0_loss": 5.592425890023786, "test_0_acc1": 10.268, "test_0_acc5": 23.954, "test_5_loss": 7.121632807848168, "test_5_acc1": 1.17125, "test_5_acc5": 3.44575, "epoch": 32, "n_parameters": 5717416}
{"train_lr": 0.0030739744559831173, "train_loss": 6.054409027099609, "test_0_loss": 5.690968008889499, "test_0_acc1": 9.494, "test_0_acc5": 22.702, "test_5_loss": 7.121632807848168, "test_5_acc1": 1.17125, "test_5_acc5": 3.44575, "epoch": 33, "n_parameters": 5717416}
{"train_lr": 0.003020537624422026, "train_loss": 6.063710031606597, "test_0_loss": 4.698054779361473, "test_0_acc1": 15.238, "test_0_acc5": 32.862, "test_5_loss": 7.121632807848168, "test_5_acc1": 1.17125, "test_5_acc5": 3.44575, "epoch": 34, "n_parameters": 5717416}
{"train_lr": 0.0029660985798329645, "train_loss": 5.910670252202703, "test_0_loss": 5.064415029280474, "test_0_acc1": 13.038, "test_0_acc5": 29.468, "test_5_loss": 7.283367584701997, "test_5_acc1": 1.14525, "test_5_acc5": 3.321, "epoch": 35, "n_parameters": 5717416}
{"train_lr": 0.0029107110469803756, "train_loss": 5.995033393136794, "test_0_loss": 4.890588184388418, "test_0_acc1": 14.9, "test_0_acc5": 32.556, "test_5_loss": 7.283367584701997, "test_5_acc1": 1.14525, "test_5_acc5": 3.321, "epoch": 36, "n_parameters": 5717416}
{"train_lr": 0.002854429686672256, "train_loss": 5.9739233070044975, "test_0_loss": 5.277839526410142, "test_0_acc1": 12.19, "test_0_acc5": 28.048, "test_5_loss": 7.283367584701997, "test_5_acc1": 1.14525, "test_5_acc5": 3.321, "epoch": 37, "n_parameters": 5717416}
{"train_lr": 0.002797310041816381, "train_loss": 5.918417158744318, "test_0_loss": 5.211479980214925, "test_0_acc1": 13.658, "test_0_acc5": 30.232, "test_5_loss": 7.283367584701997, "test_5_acc1": 1.14525, "test_5_acc5": 3.321, "epoch": 38, "n_parameters": 5717416}
{"train_lr": 0.002739408482605983, "train_loss": 5.925514445745116, "test_0_loss": 5.419682004859031, "test_0_acc1": 10.596, "test_0_acc5": 24.802, "test_5_loss": 7.283367584701997, "test_5_acc1": 1.14525, "test_5_acc5": 3.321, "epoch": 39, "n_parameters": 5717416}
{"train_lr": 0.002680782150889308, "train_loss": 5.883729684505341, "test_0_loss": 5.13738645015431, "test_0_acc1": 13.256, "test_0_acc5": 29.476, "test_5_loss": 7.820061174479342, "test_5_acc1": 1.0445, "test_5_acc5": 3.129, "epoch": 40, "n_parameters": 5717416}
{"train_lr": 0.0026214889037779947, "train_loss": 5.8768603530623835, "test_0_loss": 5.099718636911188, "test_0_acc1": 14.214, "test_0_acc5": 32.066, "test_5_loss": 7.820061174479342, "test_5_acc1": 1.0445, "test_5_acc5": 3.129, "epoch": 41, "n_parameters": 5717416}
{"train_lr": 0.002561587256548306, "train_loss": 5.914836130887389, "test_0_loss": 5.2416254764783865, "test_0_acc1": 11.636, "test_0_acc5": 26.856, "test_5_loss": 7.820061174479342, "test_5_acc1": 1.0445, "test_5_acc5": 3.129, "epoch": 42, "n_parameters": 5717416}
{"train_lr": 0.0025011363248938225, "train_loss": 5.900150206830386, "test_0_loss": 5.101568935318627, "test_0_acc1": 13.276, "test_0_acc5": 30.104, "test_5_loss": 7.820061174479342, "test_5_acc1": 1.0445, "test_5_acc5": 3.129, "epoch": 43, "n_parameters": 5717416}
{"train_lr": 0.00244019576658606, "train_loss": 5.86289567250809, "test_0_loss": 4.907099856829994, "test_0_acc1": 14.74, "test_0_acc5": 32.012, "test_5_loss": 7.820061174479342, "test_5_acc1": 1.0445, "test_5_acc5": 3.129, "epoch": 44, "n_parameters": 5717416}
{"train_lr": 0.0023788257225985108, "train_loss": 5.835502566836721, "test_0_loss": 5.569610283303093, "test_0_acc1": 11.684, "test_0_acc5": 26.968, "test_5_loss": 7.334822424390113, "test_5_acc1": 1.49225, "test_5_acc5": 4.25875, "epoch": 45, "n_parameters": 5717416}
{"train_lr": 0.002317086757755297, "train_loss": 5.902843760119544, "test_0_loss": 5.3058019126750535, "test_0_acc1": 13.534, "test_0_acc5": 29.732, "test_5_loss": 7.334822424390113, "test_5_acc1": 1.49225, "test_5_acc5": 4.25875, "epoch": 46, "n_parameters": 5717416}
{"train_lr": 0.0022550398009607707, "train_loss": 5.798361852205248, "test_0_loss": 5.197937856175087, "test_0_acc1": 13.114, "test_0_acc5": 29.348, "test_5_loss": 7.334822424390113, "test_5_acc1": 1.49225, "test_5_acc5": 4.25875, "epoch": 47, "n_parameters": 5717416}
{"train_lr": 0.0021927460850704548, "train_loss": 5.73072993350353, "test_0_loss": 4.734242562521595, "test_0_acc1": 17.252, "test_0_acc5": 35.896, "test_5_loss": 7.334822424390113, "test_5_acc1": 1.49225, "test_5_acc5": 4.25875, "epoch": 48, "n_parameters": 5717416}
{"train_lr": 0.0021302670864610006, "train_loss": 5.69394142336125, "test_0_loss": 5.058369449217657, "test_0_acc1": 14.906, "test_0_acc5": 32.252, "test_5_loss": 7.334822424390113, "test_5_acc1": 1.49225, "test_5_acc5": 4.25875, "epoch": 49, "n_parameters": 5717416}
{"train_lr": 0.002067664464360847, "train_loss": 5.709019121363294, "test_0_loss": 4.3592056012351925, "test_0_acc1": 19.068, "test_0_acc5": 39.056, "test_5_loss": 10.509043875979218, "test_5_acc1": 0.72375, "test_5_acc5": 2.323, "epoch": 50, "n_parameters": 5717416}
{"train_lr": 0.002005000000000026, "train_loss": 5.700551097960972, "test_0_loss": 4.753117966484123, "test_0_acc1": 15.846, "test_0_acc5": 33.87, "test_5_loss": 10.509043875979218, "test_5_acc1": 0.72375, "test_5_acc5": 2.323, "epoch": 51, "n_parameters": 5717416}
{"train_lr": 0.0019423355356391193, "train_loss": 5.7724813230031975, "test_0_loss": 4.528603377741876, "test_0_acc1": 18.206, "test_0_acc5": 37.726, "test_5_loss": 10.509043875979218, "test_5_acc1": 0.72375, "test_5_acc5": 2.323, "epoch": 52, "n_parameters": 5717416}

Thanks very much

Thank you for sharing your awesome work.

I've tried to download the pre-trained model of DeiT-small (Adversarial Training).

The current link:
https://drive.google.com/file/d/1U5XmAUQkSlw5Q1ZhsVriBEOQEk-bPfFU/view?usp=sharing
does not have a pth file so I'm a little bit confused.

It would be great if you update the link.

Thank you

Hi The adversarial training script is showing strage trend, after certain epochs top-1 accuracy has fallen to 1.6% from around 21%. Is it normal?

I used the script for adv training as:
python -m torch.distributed.launch --nproc_per_node=4 --master_port=5672 --use_env main_adv_deit.py --model deit_small_patch16_224_adv --batch-size 128 --data-path /datasets/imagenet-ilsvrc2012 --attack-iter 1 --attack-epsilon 4 --attack-step-size 4 --epoch 100 --reprob 0 --no-repeated-aug --sing singln --drop 0 --drop-path 0 --start_epoch 0 --warmup-epochs 10 --cutmix 0 --output_dir save/deit_adv/deit_small_patch16_224

Here is the training log (till 40 epochs):
{"train_lr": 1.0000000000000031e-06, "train_loss": 6.885785259502969, "test_0_loss": 6.7725973782139715, "test_0_acc1": 0.806, "test_0_acc5": 2.804, "test_5_loss": 6.844994894602477, "test_5_acc1": 0.55, "test_5_acc5": 1.958, "epoch": 0, "n_parameters": 22050664}
{"train_lr": 1.0000000000000031e-06, "train_loss": 6.885785259502969, "test_0_loss": 6.7725973782139715, "test_0_acc1": 0.806, "test_0_acc5": 2.804, "test_5_loss": 6.844994894602477, "test_5_acc1": 0.55, "test_5_acc5": 1.958, "epoch": 0, "n_parameters": 22050664}
{"train_lr": 1.0000000000000031e-06, "train_loss": 6.846427675869634, "test_0_loss": 6.689390176393554, "test_0_acc1": 1.192, "test_0_acc5": 4.378, "test_5_loss": 6.844994894602477, "test_5_acc1": 0.55, "test_5_acc5": 1.958, "epoch": 1, "n_parameters": 22050664}
{"train_lr": 0.00020090000000000288, "train_loss": 6.701197089479981, "test_0_loss": 5.865309043488896, "test_0_acc1": 5.43, "test_0_acc5": 14.672, "test_5_loss": 6.844994894602477, "test_5_acc1": 0.55, "test_5_acc5": 1.958, "epoch": 2, "n_parameters": 22050664}
{"train_lr": 0.00040079999999998546, "train_loss": 6.543532955179588, "test_0_loss": 5.340847122768371, "test_0_acc1": 9.812, "test_0_acc5": 23.782, "test_5_loss": 6.844994894602477, "test_5_acc1": 0.55, "test_5_acc5": 1.958, "epoch": 3, "n_parameters": 22050664}
{"train_lr": 0.0006006999999999715, "train_loss": 6.4769038225916455, "test_0_loss": 5.03732673006796, "test_0_acc1": 13.248, "test_0_acc5": 29.602, "test_5_loss": 6.844994894602477, "test_5_acc1": 0.55, "test_5_acc5": 1.958, "epoch": 4, "n_parameters": 22050664}
{"train_lr": 0.0008006000000000287, "train_loss": 6.315360357340196, "test_0_loss": 5.300121459301969, "test_0_acc1": 10.944, "test_0_acc5": 25.546, "test_5_loss": 6.55244832365313, "test_5_acc1": 2.756, "test_5_acc5": 7.6525, "epoch": 5, "n_parameters": 22050664}
{"train_lr": 0.0010004999999999689, "train_loss": 6.190600837687318, "test_0_loss": 4.9149362563476755, "test_0_acc1": 14.35, "test_0_acc5": 31.418, "test_5_loss": 6.55244832365313, "test_5_acc1": 2.756, "test_5_acc5": 7.6525, "epoch": 6, "n_parameters": 22050664}
{"train_lr": 0.0012004000000000647, "train_loss": 6.088374964529566, "test_0_loss": 5.50498779843575, "test_0_acc1": 10.254, "test_0_acc5": 24.242, "test_5_loss": 6.55244832365313, "test_5_acc1": 2.756, "test_5_acc5": 7.6525, "epoch": 7, "n_parameters": 22050664}
{"train_lr": 0.0014002999999999238, "train_loss": 6.08913704293142, "test_0_loss": 4.774700349977363, "test_0_acc1": 14.72, "test_0_acc5": 32.384, "test_5_loss": 6.55244832365313, "test_5_acc1": 2.756, "test_5_acc5": 7.6525, "epoch": 8, "n_parameters": 22050664}
{"train_lr": 0.0016001999999999618, "train_loss": 6.150533516344121, "test_0_loss": 5.227625224198276, "test_0_acc1": 10.67, "test_0_acc5": 25.058, "test_5_loss": 6.55244832365313, "test_5_acc1": 2.756, "test_5_acc5": 7.6525, "epoch": 9, "n_parameters": 22050664}
{"train_lr": 0.0018001000000000126, "train_loss": 6.101692359891536, "test_0_loss": 5.141786843786161, "test_0_acc1": 11.414, "test_0_acc5": 26.346, "test_5_loss": 6.756372647642403, "test_5_acc1": 2.309, "test_5_acc5": 6.5675, "epoch": 10, "n_parameters": 22050664}
{"train_lr": 0.001951301233713633, "train_loss": 6.093319233182332, "test_0_loss": 4.774902591320924, "test_0_acc1": 14.368, "test_0_acc5": 31.786, "test_5_loss": 6.756372647642403, "test_5_acc1": 2.309, "test_5_acc5": 6.5675, "epoch": 11, "n_parameters": 22050664}
{"train_lr": 0.001941176365109525, "train_loss": 6.128870297345421, "test_0_loss": 5.251185640492503, "test_0_acc1": 11.492, "test_0_acc5": 26.726, "test_5_loss": 6.756372647642403, "test_5_acc1": 2.309, "test_5_acc5": 6.5675, "epoch": 12, "n_parameters": 22050664}
{"train_lr": 0.0019301276034588222, "train_loss": 6.053121808859752, "test_0_loss": 4.758080562108309, "test_0_acc1": 16.252, "test_0_acc5": 34.882, "test_5_loss": 6.756372647642403, "test_5_acc1": 2.309, "test_5_acc5": 6.5675, "epoch": 13, "n_parameters": 22050664}
{"train_lr": 0.0019181658525555538, "train_loss": 6.0439764577136055, "test_0_loss": 4.586510399862962, "test_0_acc1": 16.69, "test_0_acc5": 35.526, "test_5_loss": 6.756372647642403, "test_5_acc1": 2.309, "test_5_acc5": 6.5675, "epoch": 14, "n_parameters": 22050664}
{"train_lr": 0.0019053029172036828, "train_loss": 5.91496213320062, "test_0_loss": 4.488940908904268, "test_0_acc1": 17.398, "test_0_acc5": 36.698, "test_5_loss": 7.4814025707452325, "test_5_acc1": 1.2555, "test_5_acc5": 4.2435, "epoch": 15, "n_parameters": 22050664}
{"train_lr": 0.0018915514915675221, "train_loss": 6.002524321551898, "test_0_loss": 4.450921233922186, "test_0_acc1": 17.934, "test_0_acc5": 37.114, "test_5_loss": 7.4814025707452325, "test_5_acc1": 1.2555, "test_5_acc5": 4.2435, "epoch": 16, "n_parameters": 22050664}
{"train_lr": 0.0018769251466436458, "train_loss": 5.878266204508851, "test_0_loss": 4.308091710831062, "test_0_acc1": 20.404, "test_0_acc5": 41.2, "test_5_loss": 7.4814025707452325, "test_5_acc1": 1.2555, "test_5_acc5": 4.2435, "epoch": 17, "n_parameters": 22050664}
{"train_lr": 0.0018614383168689135, "train_loss": 5.789360093222343, "test_0_loss": 4.410817133793065, "test_0_acc1": 18.154, "test_0_acc5": 38.082, "test_5_loss": 7.4814025707452325, "test_5_acc1": 1.2555, "test_5_acc5": 4.2435, "epoch": 18, "n_parameters": 22050664}
{"train_lr": 0.0018451062858745686, "train_loss": 5.750880390286541, "test_0_loss": 4.467262921391278, "test_0_acc1": 19.266, "test_0_acc5": 39.462, "test_5_loss": 7.4814025707452325, "test_5_acc1": 1.2555, "test_5_acc5": 4.2435, "epoch": 19, "n_parameters": 22050664}
{"train_lr": 0.0018279451714032378, "train_loss": 5.764791792602562, "test_0_loss": 4.67896575738586, "test_0_acc1": 17.392, "test_0_acc5": 37.15, "test_5_loss": 7.557907587735987, "test_5_acc1": 0.9905, "test_5_acc5": 3.1575, "epoch": 20, "n_parameters": 22050664}
{"train_lr": 0.0018099719094030393, "train_loss": 5.759131700348416, "test_0_loss": 4.419680974762636, "test_0_acc1": 19.966, "test_0_acc5": 40.798, "test_5_loss": 7.557907587735987, "test_5_acc1": 0.9905, "test_5_acc5": 3.1575, "epoch": 21, "n_parameters": 22050664}
{"train_lr": 0.0017912042373137494, "train_loss": 5.710006896111605, "test_0_loss": 4.2751427415236405, "test_0_acc1": 20.356, "test_0_acc5": 41.114, "test_5_loss": 7.557907587735987, "test_5_acc1": 0.9905, "test_5_acc5": 3.1575, "epoch": 22, "n_parameters": 22050664}
{"train_lr": 0.0017716606765619972, "train_loss": 5.68051082098322, "test_0_loss": 4.154385426833091, "test_0_acc1": 21.638, "test_0_acc5": 43.102, "test_5_loss": 7.557907587735987, "test_5_acc1": 0.9905, "test_5_acc5": 3.1575, "epoch": 23, "n_parameters": 22050664}
{"train_lr": 0.0017513605142823508, "train_loss": 5.693617649811158, "test_0_loss": 4.25816687512535, "test_0_acc1": 20.994, "test_0_acc5": 41.96, "test_5_loss": 7.557907587735987, "test_5_acc1": 0.9905, "test_5_acc5": 3.1575, "epoch": 24, "n_parameters": 22050664}
{"train_lr": 0.0017303237842843694, "train_loss": 5.6821105527839695, "test_0_loss": 4.421267043301026, "test_0_acc1": 19.116, "test_0_acc5": 39.094, "test_5_loss": 8.897946262237587, "test_5_acc1": 0.514, "test_5_acc5": 1.892, "epoch": 25, "n_parameters": 22050664}
{"train_lr": 0.001708571247280513, "train_loss": 5.69677297047955, "test_0_loss": 4.398700253595852, "test_0_acc1": 19.178, "test_0_acc5": 39.536, "test_5_loss": 8.897946262237587, "test_5_acc1": 0.514, "test_5_acc5": 1.892, "epoch": 26, "n_parameters": 22050664}
{"train_lr": 0.0016861243703990647, "train_loss": 5.740358965097666, "test_0_loss": 4.446112109237348, "test_0_acc1": 19.972, "test_0_acc5": 40.84, "test_5_loss": 8.897946262237587, "test_5_acc1": 0.514, "test_5_acc5": 1.892, "epoch": 27, "n_parameters": 22050664}
{"train_lr": 0.0016630053059970855, "train_loss": 5.712303198760838, "test_0_loss": 4.1932648324234245, "test_0_acc1": 21.566, "test_0_acc5": 42.98, "test_5_loss": 8.897946262237587, "test_5_acc1": 0.514, "test_5_acc5": 1.892, "epoch": 28, "n_parameters": 22050664}
{"train_lr": 0.0016392368698000565, "train_loss": 5.74558376472631, "test_0_loss": 4.124165606513972, "test_0_acc1": 21.932, "test_0_acc5": 43.39, "test_5_loss": 8.897946262237587, "test_5_acc1": 0.514, "test_5_acc5": 1.892, "epoch": 29, "n_parameters": 22050664}
{"train_lr": 0.0016148425183847566, "train_loss": 5.3731044158518175, "test_0_loss": 4.680003530995935, "test_0_acc1": 15.374, "test_0_acc5": 33.588, "test_5_loss": 11.426024395688863, "test_5_acc1": 0.1075, "test_5_acc5": 0.45, "epoch": 30, "n_parameters": 22050664}
{"train_lr": 0.0015898463260310706, "train_loss": 4.259690835869474, "test_0_loss": 5.981102620495181, "test_0_acc1": 5.786, "test_0_acc5": 14.55, "test_5_loss": 11.426024395688863, "test_5_acc1": 0.1075, "test_5_acc5": 0.45, "epoch": 31, "n_parameters": 22050664}
{"train_lr": 0.0015642729609628443, "train_loss": 4.075305948785836, "test_0_loss": 5.933592574686403, "test_0_acc1": 4.598, "test_0_acc5": 13.066, "test_5_loss": 11.426024395688863, "test_5_acc1": 0.1075, "test_5_acc5": 0.45, "epoch": 32, "n_parameters": 22050664}
{"train_lr": 0.001538147661004018, "train_loss": 4.167220209940351, "test_0_loss": 6.295307500501207, "test_0_acc1": 3.228, "test_0_acc5": 9.566, "test_5_loss": 11.426024395688863, "test_5_acc1": 0.1075, "test_5_acc5": 0.45, "epoch": 33, "n_parameters": 22050664}
{"train_lr": 0.001511496208671658, "train_loss": 4.134730825523774, "test_0_loss": 5.972504679850104, "test_0_acc1": 3.806, "test_0_acc5": 11.758, "test_5_loss": 11.426024395688863, "test_5_acc1": 0.1075, "test_5_acc5": 0.45, "epoch": 34, "n_parameters": 22050664}
{"train_lr": 0.0014843449057311518, "train_loss": 4.365966309007885, "test_0_loss": 6.5958606600380065, "test_0_acc1": 2.156, "test_0_acc5": 7.3, "test_5_loss": 12.839466273136347, "test_5_acc1": 0.002, "test_5_acc5": 0.0035, "epoch": 35, "n_parameters": 22050664}
{"train_lr": 0.00145672054724078, "train_loss": 4.49492947772729, "test_0_loss": 6.905164708865429, "test_0_acc1": 1.588, "test_0_acc5": 5.264, "test_5_loss": 12.839466273136347, "test_5_acc1": 0.002, "test_5_acc5": 0.0035, "epoch": 36, "n_parameters": 22050664}
{"train_lr": 0.0014286503951072877, "train_loss": 4.562651729769558, "test_0_loss": 6.958603466617245, "test_0_acc1": 1.594, "test_0_acc5": 5.226, "test_5_loss": 12.839466273136347, "test_5_acc1": 0.002, "test_5_acc5": 0.0035, "epoch": 37, "n_parameters": 22050664}
{"train_lr": 0.0014001621511816529, "train_loss": 4.620032903101804, "test_0_loss": 6.883705623624269, "test_0_acc1": 1.946, "test_0_acc5": 5.582, "test_5_loss": 12.839466273136347, "test_5_acc1": 0.002, "test_5_acc5": 0.0035, "epoch": 38, "n_parameters": 22050664}
{"train_lr": 0.0013712839299212382, "train_loss": 4.635755813831715, "test_0_loss": 7.244386745887312, "test_0_acc1": 0.964, "test_0_acc5": 3.976, "test_5_loss": 12.839466273136347, "test_5_acc1": 0.002, "test_5_acc5": 0.0035, "epoch": 39, "n_parameters": 22050664}
{"train_lr": 0.0013420442306441068, "train_loss": 4.83734727265547, "test_0_loss": 7.3013705145603405, "test_0_acc1": 1.686, "test_0_acc5": 4.768, "test_5_loss": 14.802937962195847, "test_5_acc1": 0.0, "test_5_acc5": 0.0, "epoch": 40, "n_parameters": 22050664}