yzxing87 / invertible-isp Goto Github PK
View Code? Open in Web Editor NEW[CVPR2021] Invertible Image Signal Processing
License: MIT License
[CVPR2021] Invertible Image Signal Processing
License: MIT License
Hi! How can i add the 'constant' parameter in convolve method
The preprocessing code contains the following lines:
if camera_name == 'Canon EOD 5D':
raw_img = np.maximum(raw_img - 127.0, 0)
The string literal is incorrect, it should be Canon_EOS_5D
(with an 'S' and underscores). As a result, the Canon data has not been correctly shifted.
The network has most likely learned to correct for this on its own, but I still thought I'd let you know.
Keep in mind that fixing the typo without releasing a new pretrained model will probably result in broken outputs.
From my understand, the trained model provides an invertible function that can convert between Raw and RGB image. The GT RGBs used in training are not the JPEGs straight out of camera but generated by Rawpy. So the model simulates the process of Rawpy, which is relatively simple comapred to the ISP inside the real camera(e.g. most likely no local tone mapping is used, neither the camera manufacturer's proprietary color profile). Therefore, the trained model only applies to a specific RAW to Jpeg process, with fixed ISP parameters. White balance, which is most likely different between photos, is conducted in preprocess, but other ISP tunable parameters like tone curve, 3D LUT, color temperature-related CCM, lens shading are left for the network to simulate, which leads to my first question:
I tried the pretrained model with NIKON data, and found the simulated Raw to Jpeg process cause visible hightlight color shifting, as shown in the top left corner:
unable to activate the conda environment.
unable to import the packages that are installed in conda environment .
conda activate myenv not working. It says the shell is not configured. Someone plz help.
How about the training cost for this project, e.g., the number of GPUs, and the total training time?
我英语不太好,请允许我用中文提问。
在发布的test_raw.py 和 cal_metrics.py中,RAW图像没有经过反转 白平衡(white balance) 和 去马赛克(demosaicing)直接计算了PSNR指标,这样算出来的指标可能存在问题(不反转白平衡,RAW图像的像素值范围可能会超出[0, 1])。请问论文中的数值是如何计算的?
我对RGB图像指标的计算也存在疑惑。在
Invertible-ISP/data/data_preprocess.py
Line 54 in 344dd33
Line 97 in 344dd33
在我理解中,InvISP的目的是
为什么需要在
Invertible-ISP/data/data_preprocess.py
Line 54 in 344dd33
是我哪里理解错误?期待您的回答。
I'm trying to train the model on another dataset. But I have encountered the following problem:
Parsed arguments: Namespace(aug=True, batch_size=1, camera='Canon1DsMkIII', data_path='/data/lly/inv_isp_data/', debug_mode=False, gamma=True, loss='L1', lr=0.0001, out_path='/data/lly/inv_isp_data/Canon1DsMkIII/', resume=False, rgb_weight=1, task='debug')
[INFO] Start data loading and preprocessing
[INFO] Start to train
task: debug Epoch: 0 Step: 0 || loss: 0.46242 raw_loss: 0.10383 rgb_loss: 0.35858 || lr: 0.000100 time: 0.316538
task: debug Epoch: 0 Step: 1 || loss: 0.24662 raw_loss: 0.01957 rgb_loss: 0.22705 || lr: 0.000100 time: 0.270781
task: debug Epoch: 0 Step: 2 || loss: 0.05458 raw_loss: 0.00540 rgb_loss: 0.04919 || lr: 0.000100 time: 0.269678
task: debug Epoch: 0 Step: 3 || loss: 0.12149 raw_loss: 0.00757 rgb_loss: 0.11392 || lr: 0.000100 time: 0.269641
task: debug Epoch: 0 Step: 4 || loss: 0.17164 raw_loss: 0.00870 rgb_loss: 0.16295 || lr: 0.000100 time: 0.282781
task: debug Epoch: 0 Step: 5 || loss: 0.09719 raw_loss: 0.00595 rgb_loss: 0.09124 || lr: 0.000100 time: 0.277356
task: debug Epoch: 0 Step: 6 || loss: 0.08278 raw_loss: 0.00824 rgb_loss: 0.07454 || lr: 0.000100 time: 0.276587
task: debug Epoch: 0 Step: 7 || loss: 0.08254 raw_loss: 0.00801 rgb_loss: 0.07453 || lr: 0.000100 time: 0.279638
task: debug Epoch: 0 Step: 8 || loss: 0.11994 raw_loss: 0.01274 rgb_loss: 0.10720 || lr: 0.000100 time: 0.270859
task: debug Epoch: 0 Step: 9 || loss: 0.07166 raw_loss: 0.00605 rgb_loss: 0.06562 || lr: 0.000100 time: 0.287317
task: debug Epoch: 0 Step: 10 || loss: 0.19911 raw_loss: 0.00554 rgb_loss: 0.19357 || lr: 0.000100 time: 0.272710
task: debug Epoch: 0 Step: 11 || loss: 0.14320 raw_loss: 0.00622 rgb_loss: 0.13698 || lr: 0.000100 time: 0.279719
task: debug Epoch: 0 Step: 12 || loss: 0.05994 raw_loss: 0.00999 rgb_loss: 0.04996 || lr: 0.000100 time: 0.282813
task: debug Epoch: 0 Step: 13 || loss: 0.04691 raw_loss: 0.00428 rgb_loss: 0.04263 || lr: 0.000100 time: 0.269908
task: debug Epoch: 0 Step: 14 || loss: 0.09645 raw_loss: 0.00515 rgb_loss: 0.09129 || lr: 0.000100 time: 0.287600
task: debug Epoch: 0 Step: 15 || loss: 0.08834 raw_loss: 0.00427 rgb_loss: 0.08407 || lr: 0.000100 time: 0.288736
train.py:69: UserWarning: Using a target size (torch.Size([1, 3, 0, 256])) that is different to the input size (torch.Size([1, 3, 256, 256])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
rgb_loss = F.l1_loss(reconstruct_rgb, target_rgb)
Traceback (most recent call last):
File "train.py", line 98, in <module>
main(args)
File "train.py", line 69, in main
rgb_loss = F.l1_loss(reconstruct_rgb, target_rgb)
File "/home/amax/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py", line 2633, in l1_loss
expanded_input, expanded_target = torch.broadcast_tensors(input, target)
File "/home/amax/anaconda3/lib/python3.8/site-packages/torch/functional.py", line 71, in broadcast_tensors
return _VF.broadcast_tensors(tensors) # type: ignore
RuntimeError: The size of tensor a (256) must match the size of tensor b (0) at non-singleton dimension 2
I have searched for this problem on Google and stackoverflow, but the answers only mention that it maybe the wrong output dimension of certain layers.
So are there any fixed parameters of image size in this code? Would you mind having a look at it and pointing out the problem? Thanks!
hi, we tried to run the test.sh
script on test_raw.py
, at the line input_RGBs = sorted(glob(out_path+"pred*jpg"))
input_RGBs is an empty list, we looked on out_path and wanted to know if we need to put the images in this folder or this happened in the data_process.
we ran the test on your pretrained weights.
We are trying to convert RGB image to RAW with your model.
Can you please give us some guidelines or tips to do so ?
Thanks
I read your impressive paper and am interested in your work.
I ask you how to save the predicted images in a raw format.
In your implementation, you save it in a jpg format.
https://github.com/yzxing87/Invertible-ISP/blob/main/test_raw.py#L104
Please share the code of saving in raw format.
Hi! Thanks for the nice work! I notice that in your paper you visualize the RAW image through bilinear demosaicing, but I don't know how I can visualize the RAW image after bilinear demosaicing. in the data/data_preprocess.py
, the RAW image after bilinear demosaicing ia simply saved in the format of '.npz', and I can't find any code to visualize it. Could you please tell me how I can visualize it? Thank you very much!
Hi,
I am currently facing this issue below, when running train.py. Could you plz give me a hand?
My pc env is under:
/home/anaconda3/bin/python /home/Documents/Invertible-ISP-main/train_cuda.py --task=debug --data_path=./data/ --gamma --aug --camera=NIKON_D700 --out_path=./exps/ --debug_mode
Parsed arguments: Namespace(aug=True, batch_size=1, camera='NIKON_D700', data_path='./data/', debug_mode=True, gamma=True, loss='L1', lr=0.0001, out_path='./exps/', resume=False, rgb_weight=1, task='debug')
[INFO] Start data loading and preprocessing
[INFO] Start to train
Traceback (most recent call last):
File "/home/Documents/Invertible-ISP-main/train_cuda.py", line 99, in
main(args)
File "/home/Documents/Invertible-ISP-main/train_cuda.py", line 72, in main
reconstruct_raw = net(reconstruct_rgb, rev=True)
File "/home/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/Documents/Invertible-ISP-main/model/model.py", line 176, in forward
out = op.forward(out, rev)
File "/home/Documents/Invertible-ISP-main/model/model.py", line 124, in forward
self.s = self.clamp * (torch.sigmoid(self.H(x1)) * 2 - 1)
RuntimeError: CUDA error: an illegal memory access was encountered
Process finished with exit code 1
If switched to invertible-isp as your environment.yml said, the code somehow ghost stopped at
line 22: DiffJPEG = DiffJPEG(differentiable=True, quality=90).cuda()
without showing any errors nor printing "start to train"
A great work!
In your works, the predicted raws are results of demosaicing with 3 channels, right?
How can I get raw before demosaicing?
Hi Yazhou,
An excellent work!
I notice that you use bilinear demosaicing by Python library colour_demosaicing
, and I guess it is aiming to reverse this step. However, I wonder would if bilinear demosaicing would be enough for an ISP? It seems to have some disadvantages, such as colour error and blurring. Did you notice this problem?
Best,
Kenneth
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.