liuguoyou / gradient-centralization

A New Optimization Technique for Deep Neural Networks

Gradient-Centralization

Gradient-Centralization: A New Optimization Technique for Deep Neural Networks

The optimizers are provided in the files: SGD.py, Adam.py and Adagrad.py, including SGD_GC, SGD_GCC, SGDW_GCC, Adam_GC, Adam_GCC, AdamW_GCC and Adagrad_GCC. The optimizers with "_GC" use GC for both Conv layers and FC layers, and the optimizers with "_GCC" use GC only for Conv layers. We can use the following codes to import SGD_GC:

    from SGD import SGD_GC  

Experiments

Mini-ImageNet

The codes is in GC_code/Mini_ImageNet. The split dataset can be downloaded from here.

CIFAR100

The codes is in GC_code/CIFAR100.

ImageNet

The codes is in GC_code/ImageNet.

Fine-grained Classification

The codes is in GC_code/Fine-grained_classification. The preprocessed dataset can be downloaded from here

Objection Detection and Segmentation

The codes is in GC_code/MMdetection.