microsoft / lq-nets

LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

LQ-Nets

By Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, Gang Hua.

Microsoft Research Asia (MSRA).

Introduction

This repository contains the training code of LQ-Nets introduced in our ECCV 2018 paper:

D. Zhang*, J. Yang*, D. Ye* and G. Hua. LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks. ECCV 2018 (*: Equal contribution) PDF

Dependencies

• Python 2.7 or 3.3+
• Python bindings for OpenCV
• TensorFlow >= 1.3.0
• TensorPack

Usage

Download the ImageNet dataset and decompress into the structure like

dir/
  train/
    n01440764/
      n01440764_10026.JPEG
      ...
    ...
  val/
    ILSVRC2012_val_00000001.JPEG
    ...

To train a quantized "pre-activation" ResNet-18, simply run

python imagenet.py --gpu 0,1,2,3 --data /PATH/TO/IMAGENET --mode preact --depth 18 --qw 1 --qa 2 --logdir_id w1a2 

After the training, the result model will be stored in ./train_log/w1a2.

For more options, please refer to python imagenet.py -h.

Results

ImageNet Experiments

Quantizing both weight and activation

Model	Bit-width(W/A)	Top-1(%)	Top-5(%)
ResNet-18	1/2	62.6	84.3
ResNet-18	2/2	64.9	85.9
ResNet-18	3/3	68.2	87.9
ResNet-18	4/4	69.3	88.8
ResNet-34	1/2	66.6	86.9
ResNet-34	2/2	69.8	89.1
ResNet-34	3/3	71.9	90.2
ResNet-50	1/2	68.7	88.4
ResNet-50	2/2	71.5	90.3
ResNet-50	3/3	74.2	91.6
ResNet-50	4/4	75.1	92.4
AlexNet	1/2	55.7	78.8
AlexNet	2/2	57.4	80.1
DenseNet-121	2/2	69.6	89.1
VGG-Variant	1/2	67.1	87.6
VGG-Variant	2/2	68.8	88.6
GoogLeNet-Variant	1/2	65.6	86.4
GoogLeNet-Variant	2/2	68.2	88.1

Quantizing weight only

Model	Bit-width(W/A)	Top-1(%)	Top-5(%)
ResNet-18	2/32	68.0	88.0
ResNet-18	3/32	69.3	88.8
ResNet-18	4/32	70.0	89.1
ResNet-50	2/32	75.1	92.3
ResNet-50	4/32	76.4	93.1
AlexNet	2/32	60.5	82.7

More results can be found in the paper.

Citation

If you use our code or models in your research, please cite our paper with

@inproceedings{ZhangYangYeECCV2018,
    author = {Zhang, Dongqing and Yang, Jiaolong and Ye, Dongqiangzi and Hua, Gang},
    title = {LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks},
    booktitle = {European Conference on Computer Vision (ECCV)},
    year = {2018}
}

lq-nets's Issues

Does it has pytorch version?

When I implement pytorch version, I met the matrix inverse(torch.inverse(BxB^T)) doesn't have, How Can I promise the probility,

what is the size needed for LQ-Nets vgg model to run

When I tried to run the VGG model ,I see the RAM quickly ran out and LQ-Nets be killed by system.
I am not sure my ram is enough for the Net to run.

Is it possible the BTxB is not invertible?

https://github.com/Microsoft/LQ-Nets/blob/ca764ae6a5ba9de0627aee2e7fe68c9cf3875305/learned_quantization.py#L92

resnet20 architecture

Hi,

Thanks for sharing the code. I noticed that the resnet20 was employed in the CIFAR10 quantization. However, I didn't find the architecture. I ever saw two versions of resnet20. One is for small dataset such as CIFAR, another is for bigger dataset, such as imagenet. The difference is the width of the channel count. I wonder if it is possible for you to share detail definition of resnet20? Thanks

Lq nets traning hardware setup.

Hi so i am trying to replicate training using your code for resnet50 . Would it be possible for to know the hardware setup you used ,nvidia driver , tensor flow version and cuda version as i keep running into various cuda related errors while trying to run it .

I am useing cuda 10.0 with tensorflow 1.14 and 2080 ti right now . I keep running into cudasolver insistence could not be created

How to quantize weights only on Imagenet?

I wanted to quantize weights only. Should 'qa' be set equal to 0/32?
Thanks

how this code works on a single GPU.

I want to ask how this code works on a single GPU and what needs to be modified.

Lower quantized values set to 0 instead of levels[0]

https://github.com/Microsoft/LQ-Nets/blob/ca764ae6a5ba9de0627aee2e7fe68c9cf3875305/learned_quantization.py#L77-L79

It seems to me that for real input values below than thrs[0], that are those in (t₁, t₂] in paper notations, the corresponding y values are set to 0 instead of levels[0] (i.e. q₁ in paper).

Am I overlooking something ?

about tf version

hi! Thanks for your code.
Can you help provide the cuda/cudnn/trensorflow/tensorpack version that your guys used for your latest code？I got OOM with alxenet/batchsize=8/4 1080ti, which I think is caused by the wrong software version.

Version of tensorpack and tensorflow

Thanks for sharing the code. Would you mind sharing the version fo tensorpack and tensorflow? For my case, there are some bugs related with the version. Thanks!

questions about your paper and code

in the paper you mentioned "Our goal is to find an optimal
quantizer basis v ∈ RK as well as an encoding B = [b1, ..., bN] ∈ {−1, 1}K×N
that minimize the quantization error". So take the activation quantization for example, did you mean that

1. each layer has its own v and its own B?
2. v is learnable (included in the computional graph) and B is not learnable but just calculated with v( both in the training and testing stage)?
3. what is the variable for? "init_thrs_multiplier" here:
   https://github.com/Microsoft/LQ-Nets/blob/master/learned_quantization.py#L50

NotImplementedError

When I only have one GPU training, the following error occurs：

Traceback (most recent call last):
File "cifar10-vgg-small.py", line 156, in
launch_train_with_config(config,trainer)
File "/usr/local/lib/python3.5/dist-packages/tensorpack/train/interface.py", line 90, in launch_train_with_config
model.get_input_signature(), input,
File "/usr/local/lib/python3.5/dist-packages/tensorpack/utils/argtools.py", line 200, in wrapper
value = func(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tensorpack/graph_builder/model_desc.py", line 92, in get_input_signature
return [TensorSpec(shape=p.shape, dtype=p.dtype, name=get_op_tensor_name(p.name)[0]) for p in inputs]
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 4339, in get_controller
yield default
File "/usr/local/lib/python3.5/dist-packages/tensorpack/graph_builder/model_desc.py", line 86, in get_input_signature
inputs = self.inputs()
File "/usr/local/lib/python3.5/dist-packages/tensorpack/graph_builder/model_desc.py", line 117, in inputs
raise NotImplementedError()
NotImplementedError

Question about implementation (not bug)

Hi,

Can you help to explain more about the code line 111-112 and 253?
As I understand, these codes seem to work as the Straight-Through Estimator (STE)? However, I cannot understand how the output y is assigned to a new value but it is only affected in the backward pass? In the forward pass, y should still be quantized, right?
Additionally, why do you use tf.stop_gradient(-x_clip) instead of tf.stop_gradient(x_clip) (the minus sign)? What is the difference here?

Thank you very much.
Cheers,

Frequent `Segmentation Fault (Core dumped)`

I am trying to run the code in the usage part of the README file.
python imagenet.py --gpu 0,1,2,3 --data /home/bcrc/Datasets/imagenet --mode pre .......
However, I encountered 'core dump' error frequently during quantizing.

I am not familiar with debugging the dumped core file with python.
I can give the core file if someone can help me (It is too large to upload). Or anyone can give me some instructions?

why naive way wouldn`t get bitwise

nothing...

Extream Case (a=1bit/w=1bit)

Is it possible to train a net with 1-bit activations and 1-bit weights with LQ-Nets?

update deprecated APIs

Thanks for your great jobs.
Could you help update the deprecated APIs in the project.
For example, [0225 20:33:30 @develop.py:99] WRN [Deprecated] ModelDescBase._build_graph() interface will be deprecated after 30 Mar. Use build_graph() instead!

could you release the code using BP instead of QEM

Basis vectors tend to depend on initialized value

Tried implementing this on Pytorch.
The basis values tend to be fixed to the initialized value and do not update much.
Also network accuracy increases to 66% around and saturates after that.

How to train the full-precision model?

How to train the full-precision model? To train the full precision model, set the qw and qa to 32?

loss=0

Hello,
When i use your method, the value of classify loss sometimes is 0.0000. Why? Does it cause by quantization?