This is the open-sourced implementation of multi-precision training connected to the paper "Multi-Precision Policy Enforced Training (MuPPET): A precision-switching strategy for quantised fixed-point training of CNNs" that was published in ICML 2020. If you reference this work in a publication, we would appreciate you using the following citation:
@misc{rajagopal2020multiprecision,
title={Multi-Precision Policy Enforced Training (MuPPET): A precision-switching strategy for quantised fixed-point training of CNNs},
author={Aditya Rajagopal and Diederik Adriaan Vink and Stylianos I. Venieris and Christos-Savvas Bouganis},
year={2020},
eprint={2006.09049},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Checkout https://www.imperial.ac.uk/intelligent-digital-systems to see other publications by the Intelligent Digital Systems Lab at Imperial College London.
git clone https://github.com/ICIdsl/pytorch_training.git
cd pytorch_training
git submodule update --init src/muppet
MuPPET_Hyperparameters
- Run_Muppet : If False, all following parameters are ignored and regular training is performed
- Bit_Width : Bitwidth at which MuPPET training begins. If FP32, set to -1
- Data_Type : One of "DFixed" or "Float". Has to match Bit_Width specified
- Round_Meth : One of "Simple" or "Stochastic"
- Policy_Resolution : Refer to resolution hyperparameter in associated paper
- Policy_Patience : Refer to patience hyperparameter in associated paper
- Fp32_Epochs_Per_Lr : Number of epochs run at each learning rate once in FP32 training
- Prec_Schedule : Precisions to change into at each switch. First precision must match with the value for Bit_Width