adobe / antialiased-cnns

pip install antialiased-cnns to improve stability and accuracy

Antialiased CNNs [Project Page] [Paper] [Talk]

Making Convolutional Networks Shift-Invariant Again
Richard Zhang. In ICML, 2019.

Quick & easy start

Run pip install antialiased-cnns

import antialiased_cnns
model = antialiased_cnns.resnet50(pretrained=True) 

If you have a model already and want to antialias and continue training, copy your old weights over:

import torchvision.models as models
old_model = models.resnet50(pretrained=True) # old (aliased) model
antialiased_cnns.copy_params_buffers(old_model, model) # copy the weights over

If you want to modify your own model, use the BlurPool layer. More information about our provided models and how to use BlurPool is below.

C = 10 # example feature channel size
blurpool = antialiased_cnns.BlurPool(C, stride=2) # BlurPool layer; use to downsample a feature map
ex_tens = torch.Tensor(1,C,128,128)
print(blurpool(ex_tens).shape) # 1xCx64x64 tensor

Updates

• (Oct 2020) Finetune I initialize the antialiased model with weights from baseline model, and finetune. Before, I was training from scratch. The results are better.
• (Oct 2020) Additional models We now have 23 total model variants. I added variants of vgg, densenet, resnext, wide resnet varieties! The same conclusions hold.
• (Sept 2020) Pip install You can also now pip install antialiased-cnns and load models with the pretrained=True flag.
• (Sept 2020) Kernel 4 I have added kernel size 4 experiments. When downsampling an even sized feature map (e.g., a 128x128-->64x64), this is actually the correct size to use to keep the indices from drifting.

Table of contents

1. More information about antialiased models
   
2. Instructions for antialiasing your own model, using the BlurPool layer
   
3. ImageNet training and evaluation code. Achieving better consistency, while maintaining or improving accuracy, is an open problem. Help improve the results!

(0) Preliminaries

Pip install this package

• pip install antialiased-cnns

Or clone this repository and install requirements (notably, PyTorch)

https://github.com/adobe/antialiased-cnns.git
cd antialiased-cnns
pip install -r requirements.txt

(1) Loading an antialiased model

The following loads a pretrained antialiased model, perhaps as a backbone for your application.

import antialiased_cnns
model = antialiased_cnns.resnet50(pretrained=True, filter_size=4)

We also provide weights for antialiased AlexNet, VGG16(bn), Resnet18,34,50,101, Densenet121, and MobileNetv2 (see example_usage.py).

(2) How to antialias your own architecture

The antialiased_cnns module contains the BlurPool class, which does blur+subsampling. Run pip install antialiased-cnns or copy the antialiased_cnns subdirectory.

Methodology The methodology is simple -- first evaluate with stride 1, and then use our BlurPool layer to do antialiased downsampling. Make the following architectural changes.

import antialiased_cnns

# MaxPool --> MaxBlurPool
baseline = nn.MaxPool2d(kernel_size=2, stride=2)
antialiased = [nn.MaxPool2d(kernel_size=2, stride=1), 
    antialiased_cnns.BlurPool(C, stride=2)]
    
# Conv --> ConvBlurPool
baseline = [nn.Conv2d(Cin, C, kernel_size=3, stride=2, padding=1), 
    nn.ReLU(inplace=True)]
antialiased = [nn.Conv2d(Cin, C, kernel_size=3, stride=1, padding=1),
    nn.ReLU(inplace=True),
    antialiased_cnns.BlurPool(C, stride=2)]

# AvgPool --> BlurPool
baseline = nn.AvgPool2d(kernel_size=2, stride=2)
antialiased = antialiased_cnns.BlurPool(C, stride=2)

We assume incoming tensor has C channels. Computing a layer at stride 1 instead of stride 2 adds memory and run-time. As such, we typically skip antialiasing at the highest-resolution (early in the network), to prevent large increases.

Add antialiasing and then continue training If you already trained a model, and then add antialiasing, you can fine-tune from that old model:

antialiased_cnns.copy_params_buffers(old_model, antialiased_model)

If this doesn't work, you can just copy the parameters (and not buffers). Adding antialiasing doesn't add any parameters, so the parameter lists are identical. (It does add buffers, so some heuristic is used to match the buffers, which may throw an error.)

antialiased_cnns.copy_params(old_model, antialiased_model)

(3) ImageNet Evaluation, Results, and Training code

We observe improvements in both accuracy (how often the image is classified correctly) and consistency (how often two shifts of the same image are classified the same).

ACCURACY	Baseline	Antialiased	Delta
alexnet	56.55	56.94	+0.39
vgg11	69.02	70.51	+1.49
vgg13	69.93	71.52	+1.59
vgg16	71.59	72.96	+1.37
vgg19	72.38	73.54	+1.16
vgg11_bn	70.38	72.63	+2.25
vgg13_bn	71.55	73.61	+2.06
vgg16_bn	73.36	75.13	+1.77
vgg19_bn	74.24	75.68	+1.44
resnet18	69.74	71.67	+1.93
resnet34	73.30	74.60	+1.30
resnet50	76.16	77.41	+1.25
resnet101	77.37	78.38	+1.01
resnet152	78.31	79.07	+0.76
resnext50_32x4d	77.62	77.93	+0.31
resnext101_32x8d	79.31	79.33	+0.02
wide_resnet50_2	78.47	78.70	+0.23
wide_resnet101_2	78.85	78.99	+0.14
densenet121	74.43	75.79	+1.36
densenet169	75.60	76.73	+1.13
densenet201	76.90	77.31	+0.41
densenet161	77.14	77.88	+0.74
mobilenet_v2	71.88	72.72	+0.84

CONSISTENCY	Baseline	Antialiased	Delta
alexnet	78.18	83.31	+5.13
vgg11	86.58	90.09	+3.51
vgg13	86.92	90.31	+3.39
vgg16	88.52	90.91	+2.39
vgg19	89.17	91.08	+1.91
vgg11_bn	87.16	90.67	+3.51
vgg13_bn	88.03	91.09	+3.06
vgg16_bn	89.24	91.58	+2.34
vgg19_bn	89.59	91.60	+2.01
resnet18	85.11	88.36	+3.25
resnet34	87.56	89.77	+2.21
resnet50	89.20	91.32	+2.12
resnet101	89.81	91.97	+2.16
resnet152	90.92	92.42	+1.50
resnext50_32x4d	90.17	91.48	+1.31
resnext101_32x8d	91.33	92.67	+1.34
wide_resnet50_2	90.77	92.46	+1.69
wide_resnet101_2	90.93	92.10	+1.17
densenet121	88.81	90.35	+1.54
densenet169	89.68	90.61	+0.93
densenet201	90.36	91.32	+0.96
densenet161	90.82	91.66	+0.84
mobilenet_v2	86.50	87.73	+1.23

To reduce clutter, extended results (different filter sizes) are here. Help improve the results!

Licenses

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

All material is made available under Creative Commons BY-NC-SA 4.0 license by Adobe Inc. You can use, redistribute, and adapt the material for non-commercial purposes, as long as you give appropriate credit by citing our paper and indicating any changes that you've made.

The repository builds off the PyTorch examples repository and torchvision models repository. These are BSD-style licensed.

Citation, contact

If you find this useful for your research, please consider citing this bibtex. Please contact Richard Zhang <rizhang at adobe dot com> with any comments or feedback.

antialiased-cnns's Issues

What about upsampling?

Thank you for your awesome work. I have a question:
What if I want to perform upsampling instead of downsampling? As I understand, aliasing is a problem only when downsampling. But I came across this paper A Style-Based Generator Architecture for Generative Adversarial Networks where they also blur during upsampling, citing your work. Here the blur was applied after upsampling (instead of before as in downsampling). Could you comment on that?

I intend to apply your technique in a VAE or GAN, and I would like to know whether I should include the blur in the decoder/generator.

Utility for own model require a retrained Imagenet Pretrained ?

Thanks for your great work.
Models applied in other task usually require a Imagenet Pretrained backbones.
If I want to use this Module into own backbone , is it necessary to train a ImageNet Pretrain ... ?
Or just replace MaxPool with MaxBlurPool then load the original Pretrain weight, then to train other task ?
For example you don't have Res101 in weights/download_antialiased_models.sh , but I hope to farely compare Res101 with the same Pretrained weights.

RuntimeError when training resnext50_32x4d

Hi,
I have trained a resnet50 model successfully, but when I train resnext50_32x4d, there is an error:
models_lpf/resnet.py", line 147, in forward
out += identity
RuntimeError: The size of tensor a (20) must match the size of tensor b (80) at non-singleton dimension 3.

In addition, in models_lpf/resnet.py, "groups=4, width_per_group=32" in resnext32x4d is differenet from "groups=32, width_per_group=4" in pytorch offical code "torchvision/models/resnet.py"
Do you have any advices?

Max-blur-pool used in text recognition model (CRNN)

Hi,

we tried max-blur-pool for a text-reading model (CRNN).
This model consists of convolutional layers followed by recurrent layers (LSTM).
We also have pooling strides of e.g. (1, 2), in this case we use a filter of size 1 in the one direction and of 5 in the other direction, and compute the outer product of them.

I evaluate max-pool and max-blur-pool by shifting the input images by -4, -3, ..., +3 pixels in horizontal direction.

1st experiment

• Recognize text in shift=0 image
• Shift image, and compute probability of the shift=0 text in the current image
• Compute amplitude of "wobble" by taking max(probability) - min(probability) (see plot below for illustration)
• Compare histograms of amplitudes to see which model has less "wobble"

In this experiment, when using max-blur-pool, we get a median amplitude of 0.107 for max-pool and 0.063 for max-blur-pool.
So here max-blur-pool performs better (the same applies when computing the mean of the amplitude values)

2nd experiment

• Evaluate model by comparing ground truth texts and recognized texts for different shifts
• Compute character error rate
• Compute difference between best and worst character error rate as a measure for how robust the model is w.r.t. shifts

In this experiment, max-pool performs much better, by having only a max. abs. difference of 0.04% for character error rate, while max-blur-pool has up to 0.55%.

3rd experiment

• Recognize text in shift=0 image
• Recognize texts in all other shifted images
• Sum up edit distances between text in original image and all shifted versions

In the 2nd experiment, blur-pool performed worse than max-pool, however, we were comparing to the ground-truth, so the difference could be caused by overall model performance.
We also want to know how consistent the model is, even though the recognized text is wrong, we want it to be the same for all shifts.

Again, max-pool (slightly) performed better than max-blur-pool, by having a summed edit distance of 51 instead of 53.

Questions

• Do you have any idea why max-pool outperforms max-blur-pool? (I hoped that the recognized text in the shift=0 image does not change that much in the shifted versions of the image)
• Did you ever try and analyze non-square blur filters (e.g. 1x3, 1x5, 3x5, ...)?
• Did you ever conduct any experiments with models having recurrent layers after the convolutional layers?
• Is there any benefit in using even-sized filter kernels? (the original code uses 4 as the default)

P.S.: experiments were repeated for a blur filter size of 3 instead of 5, however, the results were worse, and I only wanted to use the best max-blur-pool model for comparison.

Implementation of original ResNet

Hi!
I am confused about the code In the L170-171 of resnet.py.
In the original implementation of resnet, It should be max-pooling 2D, which is

nn.MaxPool2d(kernel_size=2, stride=2)

Why do you use nn.MaxPool2d(kernel_size=2, stride=1) and Downsample layer (with Gaussian blur) instead?

Smooth convolutional filters instead of feature maps?

From the paper:

How do the learned convolutional filters change? Our proposed change smooths the internal feature maps for purposes of downsampling. How does training with this layer affect the learned convolutional layers? ... the anti-aliased networks (red-purple) actually learn smoother filters throughout the network, relative to the baseline (black). Adding in more aggressive low-pass filtering further decreases the TV (increasing smoothness). This indicates that our method actually induces a smoother feature extractor overall.

I wonder is it possible to smooth convolutional filters itself instead of feature maps?

No strided pooling

Hi, as the theory, strided downsample layer should be revised, and I wonder that pooling with size=3, stride=1 should be replaced as your version?
Thx!

About the implementation

Thanks for your great work! I have two questions regarding the implementation details:
(1) In the situation of strided-convolution, why the BlurPool layer is placed after the ReLU rather than right next to the convolution?
It would be much more flexible if the conv and blurpool can be coupled.
I was considering the implementation in the pre-activation resnet.
(2) This question might be silly, but why not apply bilinear interpolation layers to downsample the feature map? I haven't seen any work use it.

IPython missing from requirements.txt

Expected Behaviour

In a new python virtualenv, after pip installing the requirements, I ran python main.py --data /home/datasets/ImageNet -e -f 3 -a alexnet_lpf --weights ./weights/alexnet_lpf3.pth.tar and expected evaluation output.

Actual Behaviour

Traceback (most recent call last):
  File "main.py", line 62, in <module>
    import models_lpf
  File "/home/aswanberg/antialiased-cnns/models_lpf/__init__.py", line 12, in <module>
    from IPython import embed
ModuleNotFoundError: No module named 'IPython'

Reproduce Scenario (including but not limited to)

Seems like this would happen any time someone uses this repo with a fresh python install.

Steps to Reproduce

1. Create a new python virtualenv (or create a new python install some other way).
2. Run the above command.

Platform and Version

Ubuntu

Some Problems on Reproducing the Results on cub200

Hi, I want to train antialias-cnn on cub200 dataset. I use ResNet-50 backbone.

However, I do not see increase of accuracy with BlurPool after several experiments. Do you have any suggestions for antialias-cnn on fine-grained datasets (e.g. cub200)?

Thank you.

Expected Behaviour

Actual Behaviour

Reproduce Scenario (including but not limited to)

Steps to Reproduce

Platform and Version

Sample Code that illustrates the problem

Logs taken while reproducing problem

Larger strides/downsampling factors

First, thanks for the very nice work!

In your implementation as well as in the paper, it seems that the proposed filters (which are the binomial coefficients) are only valid for strides/downsampling factors of 2. Extrapolating from this, does it mean that I need to use the trinomial coefficients for stride 3, quadrinomial coefficients for stride 4, and so on?

By the way, you could simplify your code in downsample.py using scipy.special.binom instead of hard-coding each filter. Something like a = np.asarray([binom(filt_size-1, i) for i in range(filt_size)]) which will take care of arbitrary filt_size

Confuse abount "pool_only" in resnet.py and alexnet.py

Hi

I am confused about the meaning of args "pool_only" inside the resnet.py and alexnet.py
Default is False for Alexnet and True for ResNet, I can't find where this argument change in main.py. I think pool_only=True is the original version of the model. Is that correct?

ResNet parameter "pool_only=True"

In experiment , ResNet use parameter "pool_only=True" ?

This parameter effect accuracy ?

I think "pool_only=False" is more make sense ,but "pool_only=True" is default.

In main.py , I can't find any variable to setting this parameter

When I read the code , I am very confuse

3D implementation ?

Hello

very nice woks !
I do also struggle with spatial shift sensitivity of my 3D CNN and I would like to test your solution.
Is someone aware of a 3D implementation of those filter ? or can you guide us on the implementation ?

Many thanks

Feature Req: Making the channel argument optional

If the channel argument is set to None, keep the filter kernel with only one channel and then in the forward pass use PyTorch's .expand() function to match the input's channel count. I'm uncertain of the performance impact so only having this as an optional behavior seems safest.

This helps with testing before finalizing a design since you don't have to change the channel argument each time the channel count changes.

can i cite the figure of the paper directly?

hi, author,
i want to cite your figure 4 in my paper.
however, i can't find the information about right/permission.
can i cite your figure if i give clear indication in my paper?
expecting your answer, many thanks!

Unable to establish SSL connection,when use command: bash weights/download_antialiased_models.sh

Bilinear Upsampler Implementation

Hi, it's really refreshing to see signal processing principles used in deep networks. I have a question about the upsampling mechanism.

After going through the original code and a related issue #28, following is my attempt to implement a bilinear-upsampler:

class BilinearUpsample(nn.Module):
    def __init__(self, channels=None, interpolation_factor=2):
        super(BilinearUpsample, self).__init__()
        self.channels = channels
        self.stride = interpolation_factor

        lpf = torch.tensor([[1., 2., 1.],
                            [2., 4., 2.],
                            [1., 2., 1.]])
        lpf = lpf/torch.sum(lpf)*4
        lpf = lpf.unsqueeze(dim=0).unsqueeze(dim=0)
        lpf = lpf.repeat([self.channels, 1, 1, 1])
        self.register_buffer('lpf', lpf)

    def forward(self, x):
            return F.conv_transpose2d(input = x, weight = self.lpf, stride=self.stride, padding = 1, output_padding=1, groups=self.channels)

Is this implementation correct ?
The result seems very different from PyTorch bilinear interpolation.

Using

BilinearUpsample(channels=num_channels)(a)

vs

F.interpolate(a, scale_factor=2, mode='bilinear', align_corners=True) 

returns very different output.

HTTP Error 403: Forbidden when loading weights

Expected Behaviour

Model weights are downloaded from aws.

Actual Behaviour

urllib.error.HTTPError: HTTP Error 403: Forbidden

Reproduce Scenario (including but not limited to)

Same issue as #45

Steps to Reproduce

Platform and Version

Ubuntu 16.04 and 22.04

Sample Code that illustrates the problem

Logs taken while reproducing problem

Depthwise convolutions

How would one use this for depthwise convolutions, can groups be taken in as an input and passed through to F.conv2d instead of input.shape[1] ?

What about tf.image.resize?

@richzhang Hi. Thank you for work.
Since in TensorFlow 2.0, tf.image.resize supports gradients. Will it be more efficient or superior in performance to downsample with aforementioned function(various interpolation methods are provided)?

Adaptive version?

Hi folks,

A lot of implementations use adaptive versions of poolings to support images with different resolutions (e.g. most of the models from this repo use it).

Is it possible to modify downsampling layer to support adaptive behaviour?

Is there any particular reason for puting Blurpool before the skip-connection layers?

Hi. Thanks for your brilliant work on solving the antialiasing problem. I find in the implementation of Antialiased-ResNet50, the order to apply the antialiased layers in the skip connections is different from the one in the main pathway.
The original skip connection is:
...->conv(stride=2)->...
Now it is:
...blurpool(stride=2)->conv(stride=1)->...
But from my understanding, it should be:
...->conv(stride=1)->blurpool(stride=2)->...
And in this way, it is identical to how you deal with the Bottleneck.conv2 and Bottleneck.conv3.
Is there any reason the calculation order is inverse in the skip connections?

How is this approach related to Fourier Domain Convnets?

Hi,

interesting approach! How is this related to networks doing pooling in Fourier Domain, as discussed in http://ecmlpkdd2017.ijs.si/papers/paperID11.pdf (and the references therein)?

If stride=1, is there a difference between BlurPool and maxpool

If stride=1, is there a difference between BlurPool and maxpool

Visualizing shift-equivariance

Thanks very much for the great work!

I am very curious about the visualizing method in this paper. Do you have any plans about releasing the code about that part?

Thanks!

Any plans to explore using sinc filter for downsampling?

Awesome work.

Take a look at alias-free gan by Nvidia if you haven't already.
https://nvlabs.github.io/alias-free-gan/

The filters in this research only reduce alias by smoothing but using a sinc kaiser filter like alias-free gan can almost completely remove aliasing. It will be very interesting to see how the network performs.

Padding size issue for small images

Hello all,

I would like to train the model with CIFAR10.

But it gives error with blurring kernel size larger than 3:

RuntimeError: Padding size should be less than the corresponding input dimension, but got: padding (2, 2) at dimension 3 of input [64, 512, 2, 2]

What do you suggest? Is there a way to apply blurpool for small images?

can Downsample use tensorflow?

do you have the code of downsample with tensorflow? I really need it, thankyou for your help

Asking for Anti-antialiased U-Net codes

Hi, could you provide the anti-antialiased U-Net codes which you used in pix2pix? I wonder how you apply blurring after upsampling. Thank you in advance!

Use buffers instead of parameters in the Downsample layer

Thank you for sharing this great work and for open-sourcing your code!

I think that the things to watch out for could be avoided by defining the weights in the Downsample layer as buffers instead of parameters. See torch.nn.Module.register_buffer.

is it good for autoencoder?

is it good for autoencoder?

Could you please provide a 3D implementation in pytorch?

Expected Behaviour

Actual Behaviour

Reproduce Scenario (including but not limited to)

Steps to Reproduce

Platform and Version

Sample Code that illustrates the problem

Logs taken while reproducing problem

FLOPS increase?

Hi. Have you measured how does BlurPool affect the total number of multiply-adds needed for the model? I suppose it would make them a little bit more computationally heavy and wonder how huge is the increase.

weights connot download can you offer other links,thanks

Expected Behaviour

Actual Behaviour

Reproduce Scenario (including but not limited to)

Steps to Reproduce

Platform and Version

Sample Code that illustrates the problem

Logs taken while reproducing problem

is it possible not to use the pad layer?

hi @richzhang
is is possible not to use pad layer in Downsample layer? or just use dw-conv to replace it?
thanks

Why Conv->Relu->BlurPool ?

Why Conv->Relu->BlurPool and not Conv->BlurPool->Relu?

Just as a sidenote for maxpooling there is no difference https://stackoverflow.com/questions/35543428/activation-function-after-pooling-layer-or-convolutional-layer

Is the blur_kernel trainable or parametric?

Thank you for your great work.

I have a question. Is the blur_kernel trainable or fixed？I think it's fixed gussian parameter to blur?

Is AvrPool already an antialiased downsampling method?

I wonder is AvrPool already an antialiased downsample? and it's just a special case - filter size 2?

antialiased-cnns/models_lpf/__init__.py

Line 29 in 3f3f960

a = np.array([1., 1.])

HTTP Error 403: Forbidden when loading weights

Expected Behaviour

File from AWS loads properly

Actual Behaviour

AWS returns Access Denied for any file
urllib.error.HTTPError: HTTP Error 403: Forbidden

Reproduce Scenario (including but not limited to)

pip install antialiased_cnns
Then

import antialiased_cnns
model = antialiased_cnns.resnet50(pretrained=True)

Platform and Version

Reproduced this both on my Mac and 2 Linux servers

Increased memory usage vs. torchvision equivalent

Hello,

First of all: fantastic paper and contribution -- and the pypi package is the cherry on top :D

I decided to try switching one of my model trainings to use antialiased_cnns.resnet34 as a drop-in replacement for torchvision.models.resnet34. It seems however that the memory needs are almost 1.5x higher with the anti-aliased CNN. This is based on the fact that with the torchvision version, my model trains with a batch size of 16 per GPU (it's a sequence model, so the actual number of images going through the CNN per batch is actually much higher). With the anti-aliased CNN, I get CUDA out of memory errors for any batch size above 11.

Were you aware of this? I'm not really expecting you to post a fix, just wondering if it makes sense to you and if you were already aware of it.

Thanks again!

my keras impl

class BlurPool(KL.Layer):
    """
    https://arxiv.org/abs/1904.11486 https://github.com/adobe/antialiased-cnns
    """
    def __init__(self, filt_size=5, stride=2, **kwargs):
        self.strides = (stride,stride)
        self.filt_size = filt_size
        self.padding = ( (int(1.*(filt_size-1)/2), int(np.ceil(1.*(filt_size-1)/2)) ), (int(1.*(filt_size-1)/2), int(np.ceil(1.*(filt_size-1)/2)) ) )
        if(self.filt_size==1):
            self.a = np.array([1.,])
        elif(self.filt_size==2):
            self.a = np.array([1., 1.])
        elif(self.filt_size==3):
            self.a = np.array([1., 2., 1.])
        elif(self.filt_size==4):    
            self.a = np.array([1., 3., 3., 1.])
        elif(self.filt_size==5):    
            self.a = np.array([1., 4., 6., 4., 1.])
        elif(self.filt_size==6):    
            self.a = np.array([1., 5., 10., 10., 5., 1.])
        elif(self.filt_size==7):    
            self.a = np.array([1., 6., 15., 20., 15., 6., 1.])
        super(BlurPool, self).__init__(**kwargs)
    def compute_output_shape(self, input_shape):
        height = input_shape[1] // self.strides[0]
        width = input_shape[2] // self.strides[1] 
        channels = input_shape[3]
        return (input_shape[0], height, width, channels)
        
    def call(self, x):
        k = self.a
        k = k[:,None]*k[None,:]
        k = k / np.sum(k)
        k = np.tile (k[:,:,None,None], (1,1,K.int_shape(x)[-1],1) )                
        k = K.constant (k, dtype=K.floatx() )
        
        x = K.spatial_2d_padding(x, padding=self.padding)
        x = K.depthwise_conv2d(x, k, strides=self.strides, padding='valid')
        return x

About Internal feature distance for shift Equivariance visualization

Dear Author:
Thank you for sharing your insight and code. I have a question as below
For the Shift Equivariance visualization in Figure 5, it looks like the feature map always has same resolution with input image(32x32), how could you achieve that? As I know, the resolution of feature map should reduce as downsampling is executed.
If the bilinear interpolation is used, is that a fair feature distance metrics?
A more general question is how could one measure shift-equivariance between pixel level shift on input image and corresponding sub-pixel shift on feature map
Thank you in advance!

Feature Req: Separable Convolution

The filters are made by multiplying against its flipped copy, so it should work fine if it was kept as, for example, 1x7 instead of 7x7. Then conv2d twice with the second conv2d using the weights after swapping the width and height dimensions.

I'm uncertain if this provides much of an improvement for a size of 3, but as the filter size grows it should be faster due to the number of reads increasing linearly as the width multiplied by two instead of exponentially as the width squared.

Edit: Sorry for the closed / reopen notifications, I thought I did something wrong when trying this again recently.

Why do deeper CNNs have better shift consistency?

With baseline CNNs with no anti-aliasing, we see better shift consistency if we increase the CNN's depth, e.g. VGG11 -> VGG19, Resnet18 -> Resnet152. Why is that so?

About kernel_size of initial max pooing for Resnet

Thank you for sharing this repo.

I remember that kernel_size of initial max pooing for original ResNet is 3.

Why did you change the kernel_size of initial max pooing for ResNet from 3 to 2?

https://github.com/adobe/antialiased-cnns/blob/master/models_lpf/resnet.py#L170

Is there a special reason?

Are conv2d_transpose layers shift variant, too?

Hi, thanks for sharing the code!
I really enjoyed reading your paper.

As I understand, you have mainly considered downsampling layers (pool, strided convolution)
does the same shift variance issue exist for conv2d_transpose layers as well?
If it does, could you share your insights on how to replace the layer?

Thanks very much!

Can't find Low Pass Filter Usage in Code

Hi,

As mentioned in paper, low pass filter is used for convolution after blur pool. But I can find its implementation in code. Conv2d filters are seemed to learn by backpropagation.

Am I missing something?

Thanks,