alessiogalluccio / fastflow

an implementation of the architecture of FastFlow (Jiawei Yu et al.)

Hi there 👋

I am a DevOps and Test Automation Engineer with a background in Machine Learning.

• 🔭 I completed a grad thesis on Generative Computer Vision ML (Normalizing Flows) for Anomaly Detection.
• 🌱 I am currently learning Kubernetes, Helm, Terraform, and MLOPS

fastflow's Issues

License type

Is the license type MIT?

RuntimeError: Node 'conv_high_res_0' resp. TypeError: conv2d(): argument 'padding'

Thanks for providing your code for FastFlow.
I try to train it but I got the following error:

Traceback (most recent call last):
  File "./FastFlow/FrEIA/framework/graph_inn.py", line 300, in forward
    mod_out = node.module(mod_in, rev=rev, jac=jac)
  File "~/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./FastFlow/FrEIA/modules/all_in_one_block.py", line 248, in forward
    a1 = self.subnet(x1c)
  File "~/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "~/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward
    input = module(input)
  File "~/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "~/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 399, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "~/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 395, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
TypeError: conv2d(): argument 'padding' (position 5) must be tuple of ints, not str

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "main.py", line 29, in <module>
    model = train(train_loader, test_loader)
  File "./FastFlow/train.py", line 86, in train
    z, log_jac_det = model(inputs)
  File "~/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./FastFlow/model.py", line 133, in forward
    z, log_jac_det = self.nf(feat_s)
  File "~/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./FastFlow/FrEIA/framework/graph_inn.py", line 302, in forward
    raise RuntimeError(f"{node} encountered an error.") from e
RuntimeError: Node 'conv_high_res_0': [(24, 24, 768)] -> AllInOneBlock -> [(24, 24, 768)] encountered an error.

I'm not sure what's the error here. I assumed it was the padding-problem (which was a problem between version of torch before 1.8)
I'm using torch 1.8 but I cannot find any padding-problems.

Did someone else got this error?

AttributeError: 'GraphINN' object has no attribute 'jacobian'

when I set grad_map_viz = True ， There is an error ：AttributeError: 'GraphINN' object has no attribute 'jacobian'.
It seems to have appeared in: localization.py", line 44, in export_gradient_maps
loss = get_loss(emb, model.nf.jacobian(run_forward=False))

Any advice? Thanks

RuntimeError: shape '[-1, 32, 262144]' is invalid for input of size 792723456

Traceback (most recent call last):
File "/snap/pycharm-community/261/plugins/python-ce/helpers/pydev/pydevd.py", line 1483, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/snap/pycharm-community/261/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/hzf/QLING/code/FastFlow/main.py", line 31, in
model = train(train_loader, test_loader)
File "/home/hzf/QLING/code/FastFlow/train.py", line 128, in train
z_grouped = torch.cat(test_z, dim=0).view(-1, c.n_transforms_test, c.n_feat)

The error occured in this code:
test_labels = np.concatenate(test_labels) is_anomaly = np.array([0 if l == 0 else 1 for l in test_labels]) z_grouped = torch.cat(test_z, dim=0).view(-1, c.n_transforms_test, c.n_feat)
I set the dataloder params as follows:
n_transforms = 4 # number of transformations per sample in training n_transforms_test = 32 # number of transformations per sample in testing batch_size = 8 #
test_z shape is (42, 32, 64, 96, 96)

AttributeError: Can't pickle local object '_BaseCouplingBlock.__init__.<locals>.<lambda>'

Error in saving model
AttributeError: Can't pickle local object '_BaseCouplingBlock.init..'

in code
train.py
if c.save_model:
model.to('cpu')
save_model(model, c.modelname)
save_weights(model, c.modelname)

OOM when calculating anomaly_score in testing phase

Duplicate of #9

As title mentioned, I get OOM error when my model reach the first meta epoch test set phase in calculating anomaly score.
I encounter this error whether using resnet18, DeiT, CaiT.
Seem likes @AlessioGalluccio had try to modify calculate anomaly score into the for loop, but still have the problem.
I'm using GTX1080 8GB.
Is there have any solutions? except decrease the batch size

can't import neptuneparams in train.py

neptune-client has been installled

calculate anomaly score waste lots of GPU memory?

z_grouped = torch.cat(test_z, dim=0).view(-1, c.n_transforms_test, c.n_feat)
anomaly_score = t2np(torch.mean(z_grouped ** 2, dim=(-2, -1)))
train.py line130, 131, when calculate the anomaly_score, much GPU memory needed. And this is can be solved by calculating along with every loop at line 109-112, and then merage the results when the loop is over.

The problem when I launch the project

When I launch the project via "python main.py", I got the error message as fellowing:

Traceback (most recent call last):
  File "main.py", line 29, in <module>
    model = train(train_loader, test_loader)
  File "/home/temporary/simu/object-detection/FastFlow/train.py", line 116, in train
    if test_labels[i] == 0: # label value of good TODO eliminate magic numbers
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The error occured in this code

        # evaluate

                test_loss.append(t2np(loss))
                test_labels.append(t2np(labels))

        test_loss_good = list()
        test_loss_defective = list()
        '''
                 Error occured Here
        '''
        for i in range(len(test_labels)): 
            if test_labels[i] == 0: # label value of good TODO eliminate magic numbers
                test_loss_good.append(test_loss[i])
            else:
                test_loss_defective.append(-test_loss[i])

As I print the test_labels[i], I got an array like [0, 2]

So, what should I do to launch this project? Thanks!

ImportError: cannot import name 'DtypeArg' from 'pandas._typing'

which version of python is used？

Traceback (most recent call last):
File "main.py", line 7, in
from train import train
File "/home/robin/cuiyajie/FastFlow-master/train.py", line 12, in
import neptune.new as neptune
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/init.py", line 27, in
from neptune.internal.api_clients import backend_factory
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/internal/api_clients/init.py", line 17, in
from neptune.internal.api_clients.hosted_api_clients.hosted_backend_api_client import (
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/internal/api_clients/hosted_api_clients/hosted_backend_api_client.py", line 40, in
from neptune.internal.api_clients.hosted_api_clients.hosted_alpha_leaderboard_api_client import (
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/internal/api_clients/hosted_api_clients/hosted_alpha_leaderboard_api_client.py", line 37, in
from neptune.internal.api_clients.hosted_api_clients.mixins import HostedNeptuneMixin
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/internal/api_clients/hosted_api_clients/mixins.py", line 35, in
from neptune.utils import with_api_exceptions_handler
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/neptune/utils.py", line 30, in
import pandas as pd
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/init.py", line 142, in
from pandas.io.api import (
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/api.py", line 8, in
from pandas.io.excel import ExcelFile, ExcelWriter, read_excel
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/excel/init.py", line 1, in
from pandas.io.excel._base import ExcelFile, ExcelWriter, read_excel
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/excel/_base.py", line 32, in
from pandas.io.parsers import TextParser
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/parsers/init.py", line 1, in
from pandas.io.parsers.readers import (
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 17, in
from pandas._typing import (
ImportError: cannot import name 'DtypeArg' from 'pandas._typing' (/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/pandas/_typing.py)

ROC-AUC on Mvtec

Hello Alessio,
I'm Emilie, a PhD student in deep learning. I'm really interested in the anomaly detection model, and FastFlow seems really promising. I'm trying to develop my own version to make it work on medical images inspired by the following repo:

• yours of course ;)
• https://github.com/openvinotoolkit/anomalib/tree/feature/fastflow/anomalib/models/fastflow
• https://github.com/openai/glow
  Using your implementation, I can't reproduce the results of the article. So I would like to know if I can't get it to work properly or if you are still working on it and haven't managed to reproduce them yet.
  Thank you for your great work!

WIP status

Hi,
In the documentation you mention

Attention!: This repo is not ready!

I was able to run the code and get some results. Can you give more details why you think the repo is not ready? Is it because some of the unknowns you mentioned (reg computing anomaly localization score etc.) or is it something else?

What more might be required to say that the repo is complete?

About MVTec dataset accuracy

Hello, have you ever tested the mvtec dataset accuracy?
I tested the auroc of the carpet category with only 0.49，It is very different from the paper.
Is there any guidance and suggestions ? Thanks!

RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

Traceback (most recent call last):
File "main.py", line 29, in
model = train(train_loader, test_loader)
File "/home/robin/cuiyajie/FastFlow-master/train.py", line 65, in train
model = FastFlow()
File "/home/robin/cuiyajie/FastFlow-master/model.py", line 104, in init
print(summary(self.feature_extractor, (3,384,384)))
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/torchsummary/torchsummary.py", line 72, in summary
model(*x)
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/torch/nn/modules/container.py", line 139, in forward
input = module(input)
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1071, in _call_impl
result = forward_call(*input, **kwargs)
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/timm/models/layers/patch_embed.py", line 35, in forward
x = self.proj(x)
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1071, in _call_impl
result = forward_call(*input, **kwargs)
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 443, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/home/robin/anaconda3/envs/patchcore/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 440, in _conv_forward
self.padding, self.dilation, self.groups)
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

Q&A

Hello.

I would like to open this issue to talk about this project. I am also interested in developing this project and would be great to share information as the paper doesn't give deeply information about the implementations and offical code is no available.

If you are agree with this iniciative, firstly we could simplify the project to use Wide-ResNet50 in order to get comparative results with the previous researching. I would like to start from the begining of the paper when says:

For ResNet, we directly use the features of the last layer in the first three blocks, and put these features into three corresponding FastFlow model.

This make me thing that in the implementation we need to use the features after the input layer, layer 1 and layer 2. In this way this table 6 makes sense

But can not to imagine how to concatenate this information for make it sense with the next

In the forward process, it takes the feature map from the backbone network as input

Depending of what part you read, it seems that just one feature map or 3 are taken