Evaluation File about generative_mlzsl

[akshitac8]

Hello @aliman80 ,
Thank you for the valuable suggestions. Can you please share your NUS-WIDE training result log file?

[aliman80]

Hi Thank you very much for your response. i have attached the recent most training results for your consideration please. regards Ali

…

________________________________ From: Akshita Gupta ***@***.***> Sent: Monday, January 17, 2022 2:07 PM To: akshitac8/Generative_MLZSL ***@***.***> Cc: aliman80 ***@***.***>; Mention ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Hello @aliman80<https://github.com/aliman80> , Thank you for the valuable suggestions. Can you please share your NUS-WIDE training result log file? — Reply to this email directly, view it on GitHub<#10 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANUKA7BSYAIKYOWRRQMKE7LUWPS4RANCNFSM5MATAPIQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[aliman80]

Training_File_Results.pdf

[akshitac8]

Hi @aliman80 I have attached my log file with the message. In the log file you shared the encoder loss is very high. Please check if all the inputs are passed correctly to the code and also the uploaded code was developed on Tesla v100 machine. So, if you are using a different machine make sure your environment is developed correctly and if you are still not able to get the results you might have to do certain parameter changes to achieve near by results.
pami_reproduce_git_nus_wide.log

[aliman80]

Hi, Thank you very much for your response. I have used the code directly from the repository without any change . Any last comments what inputs specifically should i check and if you have any reference how much those inputs be so that i can compare. Thanks again ali

…

________________________________ From: Akshita Gupta ***@***.***> Sent: Monday, January 17, 2022 3:15 PM To: akshitac8/Generative_MLZSL ***@***.***> Cc: aliman80 ***@***.***>; Mention ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Hi @aliman80<https://github.com/aliman80> I have attached my log file with the message. In the log file you shared the encoder loss is very high. Please check if all the inputs are passed correctly to the code and also the uploaded code was developed on Tesla v100 machine. So, if you are using a different machine make sure your environment is developed correctly and if you are still not able to get the results you might have to do certain parameter changes to achieve near by results. pami_reproduce_git_nus_wide.log<https://github.com/akshitac8/Generative_MLZSL/files/7881374/pami_reproduce_git_nus_wide.log> — Reply to this email directly, view it on GitHub<#10 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANUKA7D6JNYI7MYR2ZC4J23UWP24PANCNFSM5MATAPIQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[aliman80]

Another thing to understand the data loading do you have json files of NUS-WIDE , also it takes 30 hours of code run to complete training loop. If you can upload small dataset which can run early that will be great help. Thanks again Ali

…

________________________________ From: muhammad ali ***@***.***> Sent: Monday, January 17, 2022 3:55 PM To: akshitac8/Generative_MLZSL ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Hi, Thank you very much for your response. I have used the code directly from the repository without any change . Any last comments what inputs specifically should i check and if you have any reference how much those inputs be so that i can compare. Thanks again ali

________________________________ From: Akshita Gupta ***@***.***> Sent: Monday, January 17, 2022 3:15 PM To: akshitac8/Generative_MLZSL ***@***.***> Cc: aliman80 ***@***.***>; Mention ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Hi @aliman80<https://github.com/aliman80> I have attached my log file with the message. In the log file you shared the encoder loss is very high. Please check if all the inputs are passed correctly to the code and also the uploaded code was developed on Tesla v100 machine. So, if you are using a different machine make sure your environment is developed correctly and if you are still not able to get the results you might have to do certain parameter changes to achieve near by results. pami_reproduce_git_nus_wide.log<https://github.com/akshitac8/Generative_MLZSL/files/7881374/pami_reproduce_git_nus_wide.log> — Reply to this email directly, view it on GitHub<#10 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANUKA7D6JNYI7MYR2ZC4J23UWP24PANCNFSM5MATAPIQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[akshitac8]

Hi @aliman80 you can check the classifier batch size, learning rate. further you can also check the ratio of seen, unseen labels used for preparing the synthetic data. You can compose the json files of NUS-WIDE dataset using the uploaded features. You can manipulate the size of the dataset in the data loader.

[aliman80]

Thank you very much i will do it . Thanks again

…

________________________________ From: Akshita Gupta ***@***.***> Sent: Monday, January 17, 2022 8:06 PM To: akshitac8/Generative_MLZSL ***@***.***> Cc: aliman80 ***@***.***>; Mention ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Hi @aliman80<https://github.com/aliman80> you can check the classifier batch size, learning rate. further you can also check the ratio of seen, unseen labels used for preparing the synthetic data. You can compose the json files of NUS-WIDE dataset using the uploaded features. You can manipulate the size of the dataset in the data loader. — Reply to this email directly, view it on GitHub<#10 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANUKA7G27RZGFSKU65E5TEDUWQ5CBANCNFSM5MATAPIQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[aliman80]

One last clarification is that in the case of NUS-WIDE all the features and tags are given so I don't need any preprocessing/extraction . We only need that if we use custom datasets? as mentioned right?

…

________________________________ From: muhammad ali ***@***.***> Sent: Monday, January 17, 2022 8:09 PM To: akshitac8/Generative_MLZSL ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Thank you very much i will do it . Thanks again

________________________________ From: Akshita Gupta ***@***.***> Sent: Monday, January 17, 2022 8:06 PM To: akshitac8/Generative_MLZSL ***@***.***> Cc: aliman80 ***@***.***>; Mention ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Hi @aliman80<https://github.com/aliman80> you can check the classifier batch size, learning rate. further you can also check the ratio of seen, unseen labels used for preparing the synthetic data. You can compose the json files of NUS-WIDE dataset using the uploaded features. You can manipulate the size of the dataset in the data loader. — Reply to this email directly, view it on GitHub<#10 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANUKA7G27RZGFSKU65E5TEDUWQ5CBANCNFSM5MATAPIQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[aliman80]

As per your suggestion i checked the input features shape, train_loc = load_dict_from_hdf5(os.path.join(src, 'nus_wide_vgg_features','nus_seen_train_vgg19.h5')) test_unseen_loc = load_dict_from_hdf5(os.path.join(src, 'nus_wide_vgg_features', 'nus_zsl_test_vgg19.h5')) test_seen_unseen_loc = load_dict_from_hdf5(os.path.join(src, 'nus_wide_vgg_features', 'nus_gzsl_test_vgg19.h5')) Here the value of test_unseen_loc vs test_seen_unseen_loc features values are exactly similar in shape , In my understanding they need to be different . So my question is the extracted features which you have provided in google drive for zsl and gzsl need to be same? i think it should be different. If it makes sense to you and if somehow value of same feature sets is given twice then please update them. test_unseen_loc. (107859, 4096) test_seen_unseen_loc['features'].shape test_seen_unseen_loc (107859, 4096)

…

________________________________ From: muhammad ali ***@***.***> Sent: Monday, January 17, 2022 8:19 PM To: akshitac8/Generative_MLZSL ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) One last clarification is that in the case of NUS-WIDE all the features and tags are given so I don't need any preprocessing/extraction . We only need that if we use custom datasets? as mentioned right?

________________________________ From: muhammad ali ***@***.***> Sent: Monday, January 17, 2022 8:09 PM To: akshitac8/Generative_MLZSL ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Thank you very much i will do it . Thanks again

________________________________ From: Akshita Gupta ***@***.***> Sent: Monday, January 17, 2022 8:06 PM To: akshitac8/Generative_MLZSL ***@***.***> Cc: aliman80 ***@***.***>; Mention ***@***.***> Subject: Re: [akshitac8/Generative_MLZSL] Evaluation File (Issue #10) Hi @aliman80<https://github.com/aliman80> you can check the classifier batch size, learning rate. further you can also check the ratio of seen, unseen labels used for preparing the synthetic data. You can compose the json files of NUS-WIDE dataset using the uploaded features. You can manipulate the size of the dataset in the data loader. — Reply to this email directly, view it on GitHub<#10 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANUKA7G27RZGFSKU65E5TEDUWQ5CBANCNFSM5MATAPIQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[aliman80]

Hi, By using pre trained weights mAP of zsl in range of 3 and GZSL in range of 1 which is quite far away. Can you update the right weights in pre trained folder please.