aiff22 / dped Goto Github PK

Hi there,

I am looking at your code and trying to reproduce your WESPE paper. I found in the function _conv_layer() in models.py file, the BN layer is after leaky_relu layer, which is different from what other people usually do (BN first, then relu layer). Is there any special reason you do so?

BTW, will the code for WESPE paper will be released?

Thanks a lot.

Best,
Yvonne

Is it posible to resume learning from latest iteration?

Google Drive returns:

Access to doc-04-7c-docs.googleusercontent.com was denied. You don't have authorization to view this page.

When I test images with resolution=orig，the size of enhanced images becomes larger than original images.
e.g. original size is 295KB while enhanced size is 1.1MB, why becoming so large?

Another question: The speed of inference is 60s per image on my macbookpro, so slow?

Please, tell me HOW TO stitch enhanced patches to single image? What do you use to stitch enhanced patches to single image? Do you meet some problem with combining or stitching? Or May be some artifacts?

hello, thank you for sharing your work. I downloaded the dataset from the link you gave, but found that the given dataset was not aligned. The test set full_size_test_images only has sony, iphone, blackberry, but no canon, can provide full_size aligned data sets for training and testing. If it is convenient, it can be sent to my mailbox [email protected]. Thank you.

the texture loss is defined as following:

loss_discrim = -tf.reduce_sum(discrim_target * tf.log(tf.clip_by_value(discrim_predictions, 1e-10, 1.0)))
loss_texture = -loss_discrim

in this case loss_texure is a negtive value, and I wonder if this is an bug. So I think the correct texture loss is :

loss_discrim = -tf.reduce_sum(discrim_target * tf.log(tf.clip_by_value(discrim_predictions, 1e-10, 1.0)))
loss_texture = loss_discrim

Please share some insight on how to port resnet(image) from model.py onto android devices.
I have trained and saved a model, but the model requires the output from resnet(image) as input. Not sure how to do that on android.

Thanks in advance.

Hi @aiff22,

I am trying to port your code to MxNet. But there is an issue about loss function definition.
For example I can not find the corresponding ops like tf.image.rgb_to_grayscale in MxNet.

Maybe you can guide me. Thanks

When i run the test code with GPU,I got the note as below:

Testing original sony model, processing image 17.jpg
2019-07-11 01:37:25.413701: E tensorflow/stream_executor/cuda/cuda_dnn.cc:353] Could not create cudnn handle: CUDNN_STATUS_NOT_INITIALIZED
2019-07-11 01:37:25.413830: E tensorflow/stream_executor/cuda/cuda_dnn.cc:361] Possibly insufficient driver version: 384.183.0
Segmentation fault (core dumped)

I use tensorflow-gpu 1.11.0, cuda9.0
I had change my cudnn version from 7.4.2 to 7.0.5. Both of these two version didn't work.
I'll thank for any suggestion!

@aiff22 , I ran the code, and I found that the loss curve about the texture loss has obvious concussion. Is that normal？

Hi, @aiff22
I am interesting about the work in DPED and WESPE. Could I get your code about WESPE?
Thanks!

Hi @aiff22,

I port your TF's version to MXNet Gluon https://github.com/pribadihcr/DPED_Gluon. Still the enhanced image is not good as yours. Please help to check the code.

BEST

Has anyone tried to change the DPED network to a single channel network？

I have captured some photos and hope to create self dataset, can you upload the scrptes which can generate training dataset?

thank you very much

Hello , i was wondering if u can elaborate about the discriminator predictions variable . it wasn't clear to me what the code line
discrim_predictions = models.adversarial(adversarial_image) returns . is it a vector with predictions over a batch of patches? or a single patch prediction?
thanks !

Hi @aiff22,

Why in your code enhanced = tf.nn.tanh(conv2d(c11, W12) + b12) * 0.58 + 0.5,
there is * 0.58 + 0.5. there is an explanation?

Thanks

I have some questions about extracting patches bases on cross-correlation metrics @aiff22

Hi @aiff22
Thank you for sharing your work.
I want to transfer your models, but the pre-training models lack of the '.meta' file, do you have the file of '.meta'?

When training the model, i.e, executing the command python train_model.py model=iphone, I find the training model seems to use the CPU instead of GPU, but my machine has a 2080Ti nvidia.
I have tried to use CUDA_VISIBLE_DECIVES=0 python train_model.py model=iphone. And I also have tried to add "os.environ['CUDA_VISIBLE_DEVICES']='0' in the code. Besides, I tried to add config=tf.compat.v1.ConfigProto(device_count={'GPU':0}) in the code. But none of them worked.
Could anyone help me, please?

cv2.normalize(img,dst=None,alpha=350,beta=10,norm_type=cv2.NORM_MINMAX)
各位可以测一下，我这边是测了发现正常环境opencv和DPED没差多少
但是在极限弱光环境下，opencv的对比度拉升要好于DPED

左边原图，中间opencv，右边DPED

I have trained the model as per your training code and dataset. I have selected 100 as batch size and 12000 number of iterations. But after training the model, the results are not good. The output image has glitch and there is no significant difference in the images (input and output). Kindly reply.
I have attached one output image for reference.

Thank you for sharing great work.
I'd like to ask a question about the comparison with APE for evaluation.

APE is a general purpose enhancement tool that does not know from what camera source the image was taken.
In my opinion, the better quality of DPED partly comes from the fact that it was trained only for specific camera sources.

Would you give me your thoughts on my opinion?
And would you share another evaluation result if you conducted training a network with all the camera sources together?

Hi,

I wish to know the hyper-parameter settings of the pre-trained model provided.
I used the default parameters and trained the network, the output quality does not match with the output generated using pre-trained model.

I've tried the following coefficients of loss function and trained the network:
Content:10.0 , tv:2000 , texture: 1.0, color:0.5
as well as
content: 1.0, tv : 400 , texture: 0.4 , color: 0.1
However these settings produce output that is degraded compared to that of pre-trained model.

Moreover, the paper suggests to pre-train the discriminator. But, the code provided does not use pre-trained discriminator.

Please help with the above 2 queries.

Awaiting your help,

I cannot download dped.gz anymore

Hi @aiff22, thanks for you release your great work. Could you release your data preprocessing code?

Hello, @aiff22 do you think it is necessary for different cameras to collect images at the same location when collecting data?

DPED dataset
I can't download DPED dataset, is it still available for download?

Dear @aiff22,
In the generator, I am confused that why not use * 0.5 +0.5 to map the values to the range of [0, 1]. 'tanh(x)*0.58 + 0.5' will map the values to the range [-0.08, 1.08]. Thank you!

Hi,

I ran the iphone,sony,blackberry pre-trained models on some sunset and visually attractive images.

The iphone model produces decent outputs but the other two remove the sharpness and almost the output is smooth

Can I know few hyper parameter settings I could try in training the model so that the output has good texture and the input image is not degraded in case of them images taken in decent light conditions ?

Hi:
I download the training data and vgg net, run the train_model.py using the given command.
But I encountered the following error:

kernels, bias = weights[i][0][0][0][0]
ValueError: too many values to unpack

So, am I wrong somewhere with the code ?

Thank you very much !

I am using the command mentioned.
python test_model.py model=iphone_orig test_subset=full resolution=orig use_gpu=false

But for a single image, it is taking around ~24 seconds. Is there any way to improve the time?

Is the tensorboard file saved in an existing folder? I am studying tensorflow and graph and statistics in tensorboard really helps me a lot. thanks very much!

Super cool. Thank you for creating this, I am testing it now, will let you know if I have any questions 👍

Can I convert to Core ML for use on iPhone?

I run the cmd, but error.
python test_model.py model=iphone_orig test_subset=full resolution=orig use_gpu=true
the error below, how should I do?
Testing original iphone model, processing image 1.jpg
Traceback (most recent call last):
File "test_model.py", line 50, in
image = np.float16(misc.imresize(misc.imread(test_dir + photo), res_sizes[phone])) / 255
AttributeError: 'module' object has no attribute 'imresize'

How to run the pre-trained model? It seems that there is no tutorial command.

The link given in the README file redirects to a Google Drive folder that does not have access.

I don't understand the appearence of 0.58 and 0.5. Also I can't find them in the paper。May @aiff22 can explain it? Thanks!

2020-03-13 00:28:09.773043: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-03-13 00:28:09.786026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-03-13 00:28:09.794261: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]
WARNING:tensorflow:From C:\Python3\lib\site-packages\tensorflow_core\python\ops\resource_variable_ops.py:1635: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Traceback (most recent call last):
File "train_model.py", line 57, in
enhanced = models.resnet(phone_image)
File "C:\Users\rkarnat\Python\DPED-master\DPED-master\models.py", line 13, in resnet
c2 = tf.nn.relu(_instance_norm(conv2d(c1, W2) + b2))
File "C:\Users\rkarnat\Python\DPED-master\DPED-master\models.py", line 114, in _instance_norm
batch, rows, cols, channels = [i.value for i in net.get_shape()]
File "C:\Users\rkarnat\Python\DPED-master\DPED-master\models.py", line 114, in
batch, rows, cols, channels = [i.value for i in net.get_shape()]
AttributeError: 'NoneType' object has no attribute 'value'

Hi, just a little doubt that what do the coefficients of the model definition of the generator (that is, 0.58 and 0.5, separately) represent for. Thanks.

enhanced = tf.nn.tanh(conv2d(c11, W12) + b12) * 0.58 + 0.5

totalMemory: 11.00GiB freeMemory: 9.09GiB
2018-06-26 11:36:14.357681: I C:\Users\User\Source\Repos\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1312] Adding visible gpu devices: 0
2018-06-26 11:36:15.811224: I C:\Users\User\Source\Repos\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8806 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
Initializing variables
Training network
2018-06-26 11:36:33.432391: E C:\Users\User\Source\Repos\tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-06-26 11:36:33.590026: E C:\Users\User\Source\Repos\tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-06-26 11:36:33.747158: E C:\Users\User\Source\Repos\tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-06-26 11:36:33.907955: E C:\Users\User\Source\Repos\tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 2.92G (3131030784 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-06-26 11:36:34.064080: E C:\Users\User\Source\Repos\tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 2.62G (2817927680 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-06-26 11:36:34.223302: E C:\Users\User\Source\Repos\tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 2.36G (2536134912 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY

How can I solve this?thanks for a lot.