d-li14 / ghostnet.pytorch

73.6% GhostNet 1.0x pre-trained model on ImageNet

PyTorch Implementation of GhostNet

Reproduction of GhostNet architecture as described in GhostNet: More Features from Cheap Operations by Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, Chang Xu on ILSVRC2012 benchmark with PyTorch framework.

Pretrained Models

Architecture	# Parameters	MFLOPs	Top-1 / Top-5 Accuracy (%)
GhostNet 1.0x	5.181M	140.77	73.636 / 91.228

from ghostnet import ghostnet

net = ghostnet()
net.load_state_dict(torch.load('pretrained/ghostnet_1x-9c40f966.pth'))

Training Strategy

• batch size 1024 on 8 GPUs
• Initial learning rate 0.4
• weight decay 0.00004
• dropout rate 0.2
• no weight decay on BN

We keep the above settings as the same and conduct experiments with different training techniques below for ablation and reproduction. During the warmup phase, learning rate linearly ramps up from 0.1 to 0.4.

epoch	LR annealing	warmup	label smooth	Top-1 / Top-5 Accuracy (%)
240	linear	×	×	72.318 / 90.670
360	linear	×	×	72.458 / 90.780
240	cosine	√	×	72.772 / 90.902
240	cosine	√	√	73.636 / 91.228

Citation

@inproceedings{Han_2020_CVPR,
  title={GhostNet: More Features from Cheap Operations},
  author={Han, Kai and Wang, Yunhe and Tian, Qi and Guo, Jianyuan and Xu, Chunjing and Xu, Chang},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year={2020}
}

ghostnet.pytorch's Issues

Training Code for ImageNet

Hi @d-li14

Could you also share the training code for ghostNet?
I would like to train the model from scratch with some minor improvements of my own for my research.
Thanks in advance!

Custom Weight Initialization Effect

I noticed you use code for custom weight initialization:

ghostnet.pytorch/ghostnet.py

Lines 162 to 169 in 83274b7

	def _initialize_weights(self):
	for m in self.modules():
	if isinstance(m, nn.Conv2d):
	nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
	elif isinstance(m, nn.BatchNorm2d):
	m.weight.data.fill_(1)
	m.bias.data.zero_()

I've not seen this before anywhere. Is there a reason behind this specific strategy? Do you know the effect this has on the training, and have you compared this with the pytorch default weight initialization? Thank you!

Thank for your job!

请问怎么训练自己的数据呢

What's the specific parameters of learning rate annealing?

Hello, I set the learning rate annealing parameter like this: for every 5 epochs passed, the learning rate was multiplied by 0.5, from 0.4 to 0. I'm currently training up to 40 epochs, top-1 accuracy is 67.010%, top-5 accuracy is 86.918%, and now the increase is very small, do I need to continue training? Could you tell me your specific parameters of linear LR annealing, which's accuracy is 72.318%/90.670%?

Where is training code?

Hello, thank you for you work! I could not find your training code to reproduce your results. Where can I find this?

A question about size of model(ghostnet_1x-f97d70db.pth) and parameters.

Sorry to bother you again, I'm confused about the size of model. The file 'ghostnet_1x-f97d70db.pth' is about 20M, but the parameters is only about 5M as you mentioned. What's the difference between them and how to measure the size of parameters? What's more, how to compute the FLOPs in the model?