bethgelab / imagecorruptions

Python package to corrupt arbitrary images.

imagecorruptions

This package provides a set of corruptions that can be applied to images in order to benchmark the robustness of neural networks. These corruptions are not meant to be used as training data augmentation but rather to test the networks against unseen perturbations. For more information have a look at the paper on the original corruption package by Hendrycks and Dietterich: Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations.

Installation and Usage

This package is pip installable via pip3 install imagecorruptions. An example of how to use the corruption function is given below:

from imagecorruptions import corrupt
...
corrupted_image = corrupt(image, corruption_name='gaussian_blur', severity=1)
...

Looping over all available corruptions can be done either by name or by index:

# via name
from imagecorruptions import get_corruption_names
for corruption in get_corruption_names():
    for severity in range(5):
        corrupted = corrupt(image, corruption_name=corruption, severity=severity+1)
        ...

# via number:
for i in range(15):
    for severity in range(5):
        corrupted = corrupt(image, corruption_number=i, severity=severity+1)
        ...

Note that the first 15 image corruptions are the common corruptions (the ones you get via get_corruption_names()). If you really wish to use these as data augmentation, there exist four additional validation corruptions which can be accessed via get_corruption_names('validation') which should then be used to test the corruption robustness of the trained model.

Citation

If you use our code or the imagecorruptions package, please consider citing:

@article{michaelis2019dragon,
  title={Benchmarking Robustness in Object Detection: 
    Autonomous Driving when Winter is Coming},
  author={Michaelis, Claudio and Mitzkus, Benjamin and 
    Geirhos, Robert and Rusak, Evgenia and 
    Bringmann, Oliver and Ecker, Alexander S. and 
    Bethge, Matthias and Brendel, Wieland},
  journal={arXiv preprint arXiv:1907.07484},
  year={2019}
}

Credit and Changelog

This package is an extension of the image corruption functions provided by Dan Hendrycks in the repository corruptions. The image corruptions implemented by Hendrycks are generalized to work on images with arbitrary image dimensions and aspect ratios aswell as on grayscale images. We furthermore removed the dependency to libmagickwand and the python api Wand and reimplemented the motion_blur in python.

imagecorruptions's Issues

motion blur fails on grayscale inputs

Your motion_blur implementation fails with grayscale inputs, e.g. 128x128 and any severity.
Minimal example:

img = np.zeros((128, 128))
motion_blur(img)

How can I use your library function to achieve the snow effect that the first image in your paper simulated？

The first picture is about the game of Thrones, and the snow effect it simulates is very real. However, I cannot achieve this effect by using the function you provided. I would like to ask whether you have used additional rendering technology to achieve this effect.Because I want to simulate snow falling on Cityscapes data sets, maybe you guys have any ideas that might be useful?

glass_blur is very slow, it cost 46 seconds

the demo code is

from imagecorruptions import corrupt, get_corruption_names
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import time
# image = np.asarray(Image.open('demo.jpg'))
image = np.ones((427, 640, 3), dtype=np.uint8)

# corrupted_image = corrupt(img, corruption_name='gaussian_blur', severity=1)

for corruption in get_corruption_names('all'):
    tic = time.time()
    for severity in range(5):
        corrupted = corrupt(image, corruption_name=corruption, severity=severity+1)
        plt.imshow(corrupted)
        plt.show()
    print(corruption, time.time() - tic)

and the result is

gaussian_noise 1.0818257331848145
shot_noise 1.0653698444366455
impulse_noise 0.9121599197387695
defocus_blur 0.8352060317993164
glass_blur 46.11975622177124
motion_blur 1.0764939785003662
zoom_blur 3.1540017127990723
snow 0.6903097629547119
frost 0.4696829319000244
fog 0.6326844692230225
brightness 1.0597143173217773
contrast 0.44191575050354004
elastic_transform 0.877924919128418
pixelate 0.4538998603820801
jpeg_compression 0.47989392280578613
speckle_noise 0.6127912998199463
gaussian_blur 1.788027048110962
spatter 1.2991032600402832
saturate 1.558685064315796

acknowledge Hendrycks

needs to be acknowledged; also: license probably needs to mention his contribution, right?

severity 0 has more than no effect

When applying image corruption with severity 0 (calling corruption()) (for example in an iterative loop), the resulting image is actually corrupted quite strongly. Maybe, either do not allow zero (as its call might be questionable), map zero to zero effect or document the meaning of zero severity.

Corruption as data augmentation?

Hi, thanks very much for your awesome work!

I am just wondering why don't you suggest using this corruption as data augmentation when training detection deep neural network? Is it only because the process time is kind of long?

Thanks again.

COCO-C

How exactly do you apply the library to get exactly the same COCO-C dataset as was used in https://arxiv.org/abs/1907.07484 ?

motion blur fails on small input sizes

Your motion_blur implementation fails with smaller inputs sizes like 32x32x3 and severity=5
Minimal example:

img = np.zeros((32, 32, 3))
motion_blur(img, severity=5)

motion_blur still fails on input of size 32x32x3

when applied to SVHN or CIFAR images of shape 32x32x3, motion_blur fails with ValueError: could not broadcast input array from shape (32,0,3) into shape (32,32,3)

documentation of corruption function

When using the package I relied on the docstrings to the functions corruption. They do not seem to be up to date. 1) The function allows for arbitrary image width and height. 2) The function only accepts numpy arrays of dtype 'uint8'. Updated docstringt would be VERY helpful ;)

Most corruptions fail on Cifar dataset with size 32 x 32 x 3

I'm trying to create corrupted version of Cifar-10 train set and I noticed that this library fails for most of the corruptions. I have attached example images of some noises at severity level 5 where the object in the image is barely visible anymore.

And here's the code I'm using to create them:

for corruption_name in ['motion_blur', 'gaussian_noise', 'snow',
                        'pixelate', 'shot_noise', 'impulse_noise', 'defocus_blur',
                        'glass_blur', 'zoom_blur', 'frost', 'fog',
                        'brightness', 'contrast', 'elastic_transform', 'jpeg_compression',
                        'speckle_noise', 'gaussian_blur', 'spatter', 'saturate']:
    print(corruption_name)
    img = img.astype('uint8')
    corrupted_img = corrupt(img, corruption_name=corruption_name, severity=5)
    plt.figure(figsize=(2,2))
    plt.imshow(corrupted_img)
    plt.show()

I remember reading that this library supports images as small as 32x32. Am I missing something?

Global warnings filter should be restricted to a single module

This warnings filter silently kills all warnings in all packages (link)

Can it be narrowed down to warnings.filterwarnings('ignore', module='imagecorruptions.corruptions')?

Feature request: Seed value concept

For usage in science, reproducibility of the corruption types would be very helpful. Thus, some sort of seed value concept would be a great feature.

pip-installable package?

Quick question: would it be possible to provide the code as a pip-installable package, similar to what Dan did with the ImageNet-C corruptions?

From a usability perspective, this would be the best possible option I'd say.

write readme

stuff that needs to be contained:

• purpose of the software, mention origin (Hendrycks)
• usage: pip install, import, call corrupt, utility functions

Note: This readme will also appear on pypi as the readme file, so it should be self contained for everyone who does not look at the code (pypi will link to github, but only host the compiled package)

Cityscapes 2D object detection

Where can I find the annotations for Cityscapes 2D object detection?

Thanks in Advance!