[JON 2022] Lightweight Deep Network-Enabled Real-Time Low-Visibility Enhancement for Promoting Vessel Detection in Maritime Video Surveillance
Lightweight Deep Network-Enabled Real-Time Low-Visibility Enhancement for Promoting Vessel Detection in Maritime Video Surveillance [paper]
Yu Guo, Yuxu Lu, Ryan Wen Liu* (* indicates corresponding author)
Journal of Navigation
- Python == 3.7
- Pytorch == 1.1.0
Maritime video surveillance has become an essential part of the vessel traffic services system, which can guarantee vessel traffic safety and security in maritime applications. To make maritime surveillance more feasible and practicable, many intelligent vision-empowered technologies have been developed to automatically detect moving vessels from maritime visual sensing data (i.e., maritime surveillance videos). However, the visual data collected in low-visibility environment easily makes the essential optical information hidden in the dark, potentially resulting in decreased accuracy of vessel detection. To guarantee reliable vessel detection under lowvisibility conditions, we propose a low-visibility enhancement network (termed LVENet) based on Retinex theory to enhance imaging quality in maritime video surveillance. To be specific, LVENet is essentially a lightweight deep neural network by incorporating a depthwise separable convolution. The synthetically-degraded image generation and hybrid loss function are further presented to enhance the robustness and generalization capacities of our LVENet. Both full-reference and no-reference evaluation experiments have demonstrated that LVENet could yield comparable or even better visual qualities than other state-of-the-art methods. In addition, LVENet only needs 0.0045 seconds to restore degraded images of size 1920×1080 on an NVIDIA 2080Ti GPU, which can adequately meet real-time requirements. The vessel detection performance can also be tremendously promoted with the enhanced visibility under low-light imaging conditions.
In this work, we design a lightweight convolutional neural network for learning the feature of maritime low-visibility scenes. In particular, we use depthwise separable convolution instead of traditional convolution to reduce model parameters and improve calculation speed. Give the current advances, no research has been conducted on the depthwise separable convolution adopted to low-light image enhancement thus far. Furthermore, a hybrid loss function is constructed to supervise the network training and enhance the network generalization.
Figure 2. The flowchart of our proposed low-visibility enhancement network (LVENet). The DS-Conv and DS-DConv represent depthwise separable convolution and depthwise separable deconvolution, respectively.
Figure 3. Usage case of traditional convolution and depthwise separable convolution.
- Put the Train images in the "data/train/syn" folder. [l6jq]
- Put the Test image in the "data/input/syn" folder. [tqok]
- Run "data/img/prepare_patches.py" to generate the "train_syn.h5".
- Modify the 145 lines of code in "main.py" to "parser.add_argument("--train", type=str, default = True, help = 'train or test')".
- Run "main.py".
- The trained model parameters will be saved in "checkpoint/".
This code contains two modes, i.e., synthetic and real-world enhancement.
- Put the low-light image in the "data/input/syn" folder.
- Modify the 145 lines of code in "main.py" to "parser.add_argument("--train", type=str, default = False, help = 'train or test')".
- Modify the 154 lines of code in "main.py" to "parser.add_argument("--syn", type=str, default = True, help = 'syn or real')".
- Run "main.py".
- The enhancement result will be saved in the "Result/output" folder.
- Put the low-light image in the "data/input/real" folder.
- Modify the 145 lines of code in "main.py" to "parser.add_argument("--train", type=str, default = False, help = 'train or test')".
- Modify the 154 lines of code in "main.py" to "parser.add_argument("--syn", type=str, default = False, help = 'syn or real')".
- Run "main.py".
- The enhancement result will be saved in the "Result/output" folder.
@article{guo2022lightweight,
title={Lightweight deep network-enabled real-time low-visibility enhancement for promoting vessel detection in maritime video surveillance},
author={Guo, Yu and Lu, Yuxu and Liu, Ryan Wen},
journal={J. Navig.},
volume={75},
number={1},
pages={230--250},
month={Oct.},
year={2022},
publisher={Cambridge University Press}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent