Inquiry about the Hybrid design about poolformer

[yuweihao]

Hi @AbdelrahmanShakerYousef ,

Thanks for your attention.

Sorry that currently, I need to prepare for my final exams at university. I will release the code and trained models of the hybrid design before 1 May. We replace the whole pooling stage with the attention or SpatialFC stage. Remember to add position embedding before the first attention stage. For hybrid design, the norm is set as Layer Norm and the batch size as 1024.

[yuweihao]

Hi @AbdelrahmanShakerYousef ,

I have completed the release of code and trained models of the hybrid design before 1 May (Anywhere on Earth Time Zone haha).

Thanks for your attention :)