yitu-opensource / mobilenext

Updates

Oct 27, 2020

• Code for detection is available

rethinking_bottleneck_design

This repo contains the code for the paper Rethinking Bottleneck Structure for Efficient Mobile Network Design (ECCV 2020)

MNEXT is an light weight models cater for mobile devices. It combines the advantages of traditional ResNet bottleneck building block and the MBV2 inverted residual block. Besides, the newly proposed building block also takes the hardware implementation into consideration such that the memory consumption can be adjusted at algorithm level without minimum impacts on the model performance.

@article{zhou2020rethinking,
  title={Rethinking Bottleneck Structure for Efficient Mobile Network Design},
  author={Zhou, Daquan and Hou, Qibin and Chen, Yunpeng and Feng, Jiashi and Yan, Shuicheng},
  journal={ECCV, August},
  year={2020}
}

The training framework is modified based on an older version(upon release) of the repo pytorch-image-models

Performance

Model performance at different width multiplier:

Model	Param.(M)	Madd(M)	Top-1 Acc.(%)
MobileNeXt-1.40	6.1	590	76.1
MobileNeXt-1.00	3.5	300	74.02
MobileNeXt-0.75	2.5	210	72
MobileNeXt-0.50	2.1	110	67.7
MobileNeXt-0.35	1.8	80	64.7

Latency and accuracy with different tensor multiplier

Model	Tensor multiplier	Madd(M)	Top-1 Acc.(%)	Latency(Pytorch,ms)
MobileNeXt	6.1	300	74.02	211
MobileNeXt	3.5	300	74.09	196
MobileNeXt	2.5	300	73.91	195
MobileNeXt	2.1	300	73.68	188

Latency measurement with Pytorch and TF Lite:

Model	Pixel 4-CPU(ms)	Pixel 4-GPU(ms)	Platform
MBV2	190 - 220	-	Pytorch Mobile
Ours	191 - 220	-	Pytorch Mobile
MBV2	68 - 92	-	TF Mobile
Ours	66 - 91	-	TF Mobile
MBV2	44 - 47	15 - 17	TF Lite
Ours	48 - 51	16 - 17	TF Lite

To reproduce the results in the paper

run the batch script as below: bash mnext_efficient_l.sh # of process

The three scripts are used for the training from scratch.

1. mnext_efficient_l.sh: MobileNeXt large model based on EfficientNet backbone
2. mnext_efficient_s.sh: MobileNeXt small model based on EfficientNet backbone
3. mnext_mbv2.sh: MobileNeXt large model based on MobileNetV2 backbone

To reproduce the latency measurement

TF lite There are four steps to follow: change to mobile deployment folder:

1. Run the MobileNeXt model tf version to save a checkpoint

• bash run.sh

      DATASET_DIR= /home/e0357894/WSNet_improvement/imagenet_tfrecord/
      TRAIN_DIR=/home/e0357894/zhoudaquan/eccv20/tensorflow-train-to-mobile-tutorial/models/research/slim
      python train_image_classifier.py \
      --train_dir ./output/ \
      --max_number_of_steps 100 \
      --dataset_name=imagenet \
      --dataset_split_name=train \
      --dataset_dir /home/e0357894/WSNet_improvement/imagenet_tfrecord/ \
      --model_name=mobilenet_v2
  

2. convert the checkpoint to mobile graph

• bash mobile_graph.sh

      python /home/e0357894/anaconda3/lib/python3.7/site-packages/tensorflow/python/tools/freeze_graph.py \
      --input_graph=./graph_output/i2rnet_ori_v3.pb \
      --input_checkpoint=./output/model.ckpt-100 \
      --input_binary=true --output_graph=./graph_output/frozen_i2rnet_ori_v3.pb \
      --output_node_names=MobilenetV2/Predictions/Reshape_1
  

3. generate the freeze graph based on the mobile graph

• bash convert.sh

      python export_inference_graph.py \
          --alsologtostderr \
          --model_name=mobilenet_v2 \
          --output_file=graph_output/i2rnet_ori_v3.pb 
  

4. convert to tf lite model to be loaded by android studio

• python tflite_converter.py

    import tensorflow.lite as lite

    # Converting a SavedModel.
    saved_model_dir = "./output/model.ckpt-100"
    converter = lite.TFLiteConverter.from_saved_model(saved_model_dir)
    tflite_model = converter.convert()

    open("converted_model.tflite", "wb").write(tflite_model)

After generating the mobile model based on tf lite, copy the model to the assets folder under the android studio project

Code for detection on both Pascal VOC and COCO

Please refer to this repo. Our MobileNeXt improves MobileNetV2 by 1% (from 22.3% to 23.3%) in terms of mAP under the same settings.

To do :

1. Add in Android project for apk generation

mobilenext's Issues

Reproducing the results of using scripts

Hey there,

Unfortunately, I've found something confused when reproducing the results of using scripts mnext_mbv2.sh: MobileNeXt large model based on MobileNetV2 backbone.
I changed the ImageNet path written in the scripts:

~/WSNet_improvement/imagenet/ILSVRC/Data/CLS-LOC/ -> /my/path/to/imagenet/ILSVRC2012/

Then I ran the following commands as mentioned in the README.md (p.s. in a pytorch1.5.1 environment with 4 NVIDIA V100s):

bash mnext_mbv2.sh 4

And I found the codes loaded the checkpoint from 'output/74_09_i2rnetv3_half_id/model_best.pth.tar (epoch 197)' but got '*** Best metric: 69.21200000732422 (epoch 209)', I think this result is not corresponding to any results in papers.

Did I do something wrong or misunderstand something when executing the script used for the training from scratch?
Could you please show more details about the scripts used for the training from scratch?

Thanks in advance!

Can you release the training config for Mobilenext-0.75? fail to reproduce

Can you release the training config for Mobilenext-0.75? fail to reproduce. Here is my training settings:
1、SGDwM(m = 0.9), cosine lr, init lr = 0.1 and batchsize = 256 on a single V100 GPU, weight_decay = 1e-4, epochs = 240
2、label smooth(eps = 0.1)
3、Common data augmentation settings(RandomResizedCrop+ColorJitter+RandomFlip)
4、lr warmup is also used exactly following the settings you mentioned in the paper, and also cool-down epochs
I tried dozens of times, and the top-1 acc just hang around 69.5%，never break 70%, let alone the 72% reported in the paper.

Actually, I am currently research in light-weight architecture desgin, I don't know if I should report the performance of mobilenext-0.75 according to my own experiment or the statistics reported in the paper.
Many thanks

请问预训练模型哪里下

请问预训练模型哪里下

请问mobilenext转onnx可以吗？我转完onnx过程没有报错但是和pytorch输出对不上

我转onnx的代码如下：
onnx版本是1.6.0

import torch
import os
import numpy as np
from model.mobilenext import mnext

def main():
initial_checkpoint = 'mobiilenext_se.pth'
network = mnext(num_classes=2)
network.eval()
state = torch.load(initial_checkpoint, map_location=lambda storage, loc: storage)
network.load_state_dict({k[7:]: v for k, v in state.items()})
dummy_input1 = torch.randn(1, 3, 48, 48)
input_names = ["input"]
output_names = ["output"]
torch.onnx.export(network, dummy_input1, "mobilenext_se.onnx", verbose=True, input_names=input_names,
output_names=output_names)

if name == 'main':
main()

一些实验细节的疑问

和论文里的细节有些对不上
1、这里代码里用了label smooth，请问给其他对比的方法用了吗？
2、这里用了model-ema，论文里没说，请问给其他对比的方法用了吗？
3、提供的复现脚本，有个resume，这个resume的参数是个啥？

Could you share the MobileNeXt-0.75 model trained on ImageNet?

@yitutech-opensource Hi, thanks for your sharing. Could you share the MobileNeXt-0.75 model trained on ImageNet?Thank you very much!!

model_name should be one of

Then I ran the following commands:
bash mnext_efficient_s.sh 1

ValueError: model_name should be one of: efficientnet_b0, efficientnet_b1, efficientnet_b2, efficientnet_b3, efficientnet_b4, efficientnet_b5, efficientnet_b6, efficientnet_b7, i2rnet_b0, i2rnet_b1, i2rnet_b2, i2rnet_b3, i2rnet_b4, i2rnet_b5, i2rnet_b6, i2rnet_b7

Performance issue in mobile_deployment/tensorflow/slim/models/research/object_detection/model_lib_v2.py

Hello! Our static bug checker has found a performance issue in mobile_deployment/tensorflow/slim/models/research/object_detection/model_lib_v2.py: eager_eval_loop is repeatedly called in a for loop, but there is a tf.function decorated function compute_eval_dict defined and called in eager_eval_loop.

In that case, when eager_eval_loop is called in a loop, the function compute_eval_dict will create a new graph every time, and that can trigger tf.function retracing warning.

Here is the tensorflow document to support it.

Briefly, for better efficiency, it's better to use:

@tf.function
def inner():
    pass

def outer():
    inner()  

than:

def outer():
    @tf.function
    def inner():
        pass
    inner()

Looking forward to your reply.

MobileNeXt代码中注释掉的结构

尝试了一下mnext.py中注视掉的expansion ratio基本都是4的这个结构:
#self.cfgs = [
# # t, c, n, s
# [1, 16, 1, 1],
# [4, 24, 2, 2],
# [4, 32, 3, 2],
# [4, 64, 3, 2],
# [4, 96, 4, 1],
# [4, 160, 3, 2],
# [4, 320, 1, 1],
#]
发现在比较小的数据集上效果和默认mobilenext1.0的结果差不多，请问这个结构被注释但是没有删掉是有什么发现吗