sharif-apu / bjdd_cvpr21 Goto Github PK

1. When I use the inference, I need to change "binningFactor" in config.json to "binnigFactor" otherwise it will throw error.
2. line 83 in processDataset.py should be changed to "elif self.gridSze == 2:"

Hi, I downloaded the pretrain QuadBayer parameters and found most value of self.attentionNet.psUpsampling1.upSample[1].running_var is inf and nan.

In eval() model, the nan value will cause the final image to be a nan image. I'm not sure whether there is some problem here.

Hey, I believe that the color loss might be a bit off. The input is a normalized image, and the ciede_2000 color difference is in the range 0 to 100 i think? Something like:

imageNP = (image * 0.5 + 0.5).permute(1,2,0).detach().cpu().numpy() * 255

or use the UnNormalize in dataTools... and to normalize the color difference:

deltaE /= 100.0

Normally I would do a PR, but my fork diverges quite a bit at the moment. Side note, your model also works great on 12 channel mosaicked images (with some trivial alterations) !

[edit: reshape becomes permute, going from C,H,W to H,W,C]

I'm really impressed with your work. Also, how well does this compare with other algorithms for traditional bayer sensors?

Dear author, I'm confused about the function of the color loss function. I find you detach the generated tensor from the foward graph and convert it to numpy array. However, after these operations, the grads cannot backpropagate to the network. So why the color loss can optimize the parameters of the network???

Hi, this work is fantastic and interesting.
I have some questions when I using your code to test Bayer mode on McM.
I would like to reproduce the results of Table.2 in your paper.
I used your code and pertained bayer model and got the resulting images are in size of 512x512.
However, the ground truth of McM is smaller (500x500).
How do you evaluate the metrics (psnr/ssim)?
There are two main questions:

1. Line 26 of inferenceUtils.py,
   sigma = self.noiseLevel/100.
   maybe should be
   sigma = self.noiseLevel/255.?
2. Line 64-75 of inferenceUtils.py,
   resizeDimension = (512, 512)
img = img.resize(resizeDimension)
   Why the image should be resized to 512, 512 before testing?

BR,
Wenzhu

In dataTools/customTransform.py, line 13, you have the line

noisyTensor = tensor + torch.randn(tensor.size()).uniform_(0, 1.) * sigma  + self.mean

This looks to be adding uniform random noise (U(0,sigma)) instead of Gaussian random noise (N(0,sigma^2)), as described in the paper (see PyTorch docs). This function is used in dataTools/customDataloader.py (customDatasetReader) which is then used in the main training code (mainModule/BJDD.py, lines 124, 91). Are the numbers reported in the paper derived from this code?

Another question: are the numbers reported in the paper a single model PIPNet model trained over a range of noise-levels, or multiple models trained at each level? If it's a single model, what was the training noise-level range?

Thanks,
Nikola

I have read your interesting paper and appreciate that you made your code available! Unfortunately, the results using the pretrained weights (bayer or quad bayer) do not give satisfactory results. Maybe this is related to #4.

Using the provided weights gives the following result:
Noise: 5 sigma

Noise: 10 sigma

Any input what the reason could be and what I do wrong would be highly appreciated.

Hi! In the paper, you said that you extracted 741,968 non-overlapping image patches of dimension 128128 from DIV2K and Flickr2K datasets. While in the code, when doing data sampling in the code ("python main.py -ds"), the 128128 images are directly resized from original images of DIV2K dataset. I am wondering when you trained your model (which led to the released pretrained weights), which way did you use to generate the training samples. Thanks a lot!