Resulting pictures looks clear from noise, with normal colors but very blurred.
May the problem be in a lot of warnings about parameters that will not be optimized?
If not, what should be changed?
# python3 ./scripts/cutomized_denoiser.py -t "v0" -p pretrained/LED_Pretrain_None_None_CVPR20_Setting_Ratio1-200.pth --dataroot /input/Calibration/ --data_pair_list /input/Calibration/list
Disable distributed.
Path already exists. Rename it to /ICCV23-LED/tb_logger/v0_archived_20240122_130426
2024-01-22 13:04:26,952 INFO:
______ __ __ __ __
/ ____/____ ____ ____/ / / / __ __ _____ / /__ / /
/ / __ / __ \ / __ \ / __ / / / / / / // ___// //_/ / /
/ /_/ // /_/ // /_/ // /_/ / / /___/ /_/ // /__ / /< /_/
\____/ \____/ \____/ \____/ /_____/\____/ \___//_/|_| (_)
Version Information:
LED: 0.1.1
PyTorch: 1.13.1+cu117
TorchVision: 0.14.1+cu117
2024-01-22 13:04:26,952 INFO:
val:[
val_freq: 99999.0
save_img: False
suffix: None
calculate_metric_in_batch: True
illumination_correct: True
metric_in_srgb: False
metrics:[
psnr:[
type: calculate_psnr
crop_border: 2
test_y_channel: False
]
ssim:[
type: calculate_ssim
crop_border: 2
test_y_channel: False
]
]
]
logger:[
print_freq: 200
save_checkpoint_freq: 99999.0
use_tb_logger: True
wandb: None
]
datasets:[
train:[
num_worker_per_gpu: 8
batch_size_per_gpu: 1
dataset_enlarge_ratio: 99999
prefetch_mode: None
name: DemoFinetuneDataset
type: FewshotPairedRAWDataset
dataroot: /input/Calibration/
which_meta: gt
data_pair_list: /input/Calibration/list
zero_clip: False
use_hflip: True
use_rot: True
crop_size: 1024
phase: train
scale: 1
]
]
train:[
generalize_first: True
pixel_opt:[
type: L1Loss
loss_weight: 1.0
reduction: mean
]
total_iter: 1500
align_iter: 1000
oomn_iter: 500
warmup_iter: -1
align_opt:[
optim_g:[
type: Adam
lr: 0.0001
weight_decay: 0
betas: [0.9, 0.999]
]
scheduler:[
type: HandieLR
milestones: [999999]
lrs: [0]
]
]
oomn_opt:[
optim_g:[
type: Adam
lr: 1e-05
weight_decay: 0
betas: [0.9, 0.999]
]
scheduler:[
type: HandieLR
milestones: [999999]
lrs: [0]
]
]
]
repnr_opt:[
dont_convert_module: ['conv10_1']
branch_num: 5
align_opts:[
init_weight: 1.0
init_bias: 0.0
]
aux_conv_opts:[
bias: True
init: zero_init_
]
]
network_g:[
type: UNetArch
inchannels: 4
outchannels: 4
channels: 32
]
base: ['options/base/network_g/repnr_unet.yaml', 'options/base/finetune/CVPR20_ELD.yaml', 'options/base/val_and_logger.yaml']
name: v0
model_type: LEDFinetuneModel
scale: 1
num_gpu: 1
manual_seed: 2022
path:[
pretrain_network_g: pretrained/LED_Pretrain_None_None_CVPR20_Setting_Ratio1-200.pth
strict_load_g: False
resume_state: None
experiments_root: /ICCV23-LED/experiments/v0
models: /ICCV23-LED/experiments/v0/models
training_states: /ICCV23-LED/experiments/v0/training_states
log: /ICCV23-LED/experiments/v0
visualization: /ICCV23-LED/experiments/v0/visualization
]
dist: False
rank: 0
world_size: 1
auto_resume: False
is_train: True
root_path: /ICCV23-LED
load lq metas in mem...: 100%|███████████████████████████████████████████████████| 2/2 [00:00<00:00, 2.29it/s]
load gt metas in mem...: 100%|███████████████████████████████████████████████████| 2/2 [00:00<00:00, 2.47it/s]
2024-01-22 13:04:28,777 INFO: Dataset [FewshotPairedRAWDataset] - DemoFinetuneDataset is built.
/opt/conda/envs/LED-ICCV23/lib/python3.8/site-packages/torch/utils/data/dataloader.py:554: UserWarning: This DataLoader will create 8 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
2024-01-22 13:04:28,777 INFO: Training statistics:
Number of train images: 2
Dataset enlarge ratio: 99999
Batch size per gpu: 1
World size (gpu number): 1
Require iter number per epoch: 199998
Total epochs: 1; iters: 1500.
2024-01-22 13:04:28,836 INFO: Network [UNetArch] is created.
2024-01-22 13:04:28,836 INFO: Convert UNetArch into RepNRBase using kwargs:
OrderedDict([('dont_convert_module', ['conv10_1']), ('branch_num', 5), ('align_opts', OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])), ('aux_conv_opts', OrderedDict([('bias', True), ('init', 'zero_init_')]))])
2024-01-22 13:04:29,626 INFO: Network: RepNRBase, with parameters: 22,619,512
2024-01-22 13:04:29,626 INFO: RepNRBase: UNetArch(
(conv1_1): RepNRConv2d(
4, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv1_2): RepNRConv2d(
32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv2_1): RepNRConv2d(
32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv2_2): RepNRConv2d(
64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(pool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv3_1): RepNRConv2d(
64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv3_2): RepNRConv2d(
128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(pool3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv4_1): RepNRConv2d(
128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv4_2): RepNRConv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(pool4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv5_1): RepNRConv2d(
256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv5_2): RepNRConv2d(
512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(upv6): ConvTranspose2d(512, 256, kernel_size=(2, 2), stride=(2, 2))
(conv6_1): RepNRConv2d(
512, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv6_2): RepNRConv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(upv7): ConvTranspose2d(256, 128, kernel_size=(2, 2), stride=(2, 2))
(conv7_1): RepNRConv2d(
256, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv7_2): RepNRConv2d(
128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(upv8): ConvTranspose2d(128, 64, kernel_size=(2, 2), stride=(2, 2))
(conv8_1): RepNRConv2d(
128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv8_2): RepNRConv2d(
64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(upv9): ConvTranspose2d(64, 32, kernel_size=(2, 2), stride=(2, 2))
(conv9_1): RepNRConv2d(
64, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv9_2): RepNRConv2d(
32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1, 1, 1), padding_mode=constant,
branch_num=5, align_opts=OrderedDict([('init_weight', 1.0), ('init_bias', 0.0)])
forward_type=reparameterize, aux_conv_opts=OrderedDict([('bias', True), ('init', 'zero_init_')])
)
(conv10_1): Conv2d(32, 4, kernel_size=(1, 1), stride=(1, 1))
)
2024-01-22 13:04:29,642 INFO: Loading RepNRBase model from pretrained/LED_Pretrain_None_None_CVPR20_Setting_Ratio1-200.pth, with param key: [params].
2024-01-22 13:04:29,660 WARNING: Current net - loaded net:
2024-01-22 13:04:29,660 WARNING: conv1_1.aux_bias
2024-01-22 13:04:29,660 WARNING: conv1_1.aux_weight
2024-01-22 13:04:29,660 WARNING: conv1_2.aux_bias
2024-01-22 13:04:29,660 WARNING: conv1_2.aux_weight
2024-01-22 13:04:29,661 WARNING: conv2_1.aux_bias
2024-01-22 13:04:29,661 WARNING: conv2_1.aux_weight
2024-01-22 13:04:29,661 WARNING: conv2_2.aux_bias
2024-01-22 13:04:29,661 WARNING: conv2_2.aux_weight
2024-01-22 13:04:29,661 WARNING: conv3_1.aux_bias
2024-01-22 13:04:29,661 WARNING: conv3_1.aux_weight
2024-01-22 13:04:29,661 WARNING: conv3_2.aux_bias
2024-01-22 13:04:29,661 WARNING: conv3_2.aux_weight
2024-01-22 13:04:29,661 WARNING: conv4_1.aux_bias
2024-01-22 13:04:29,661 WARNING: conv4_1.aux_weight
2024-01-22 13:04:29,661 WARNING: conv4_2.aux_bias
2024-01-22 13:04:29,661 WARNING: conv4_2.aux_weight
2024-01-22 13:04:29,661 WARNING: conv5_1.aux_bias
2024-01-22 13:04:29,661 WARNING: conv5_1.aux_weight
2024-01-22 13:04:29,661 WARNING: conv5_2.aux_bias
2024-01-22 13:04:29,661 WARNING: conv5_2.aux_weight
2024-01-22 13:04:29,661 WARNING: conv6_1.aux_bias
2024-01-22 13:04:29,661 WARNING: conv6_1.aux_weight
2024-01-22 13:04:29,661 WARNING: conv6_2.aux_bias
2024-01-22 13:04:29,661 WARNING: conv6_2.aux_weight
2024-01-22 13:04:29,661 WARNING: conv7_1.aux_bias
2024-01-22 13:04:29,661 WARNING: conv7_1.aux_weight
2024-01-22 13:04:29,661 WARNING: conv7_2.aux_bias
2024-01-22 13:04:29,661 WARNING: conv7_2.aux_weight
2024-01-22 13:04:29,661 WARNING: conv8_1.aux_bias
2024-01-22 13:04:29,661 WARNING: conv8_1.aux_weight
2024-01-22 13:04:29,661 WARNING: conv8_2.aux_bias
2024-01-22 13:04:29,661 WARNING: conv8_2.aux_weight
2024-01-22 13:04:29,661 WARNING: conv9_1.aux_bias
2024-01-22 13:04:29,661 WARNING: conv9_1.aux_weight
2024-01-22 13:04:29,661 WARNING: conv9_2.aux_bias
2024-01-22 13:04:29,662 WARNING: conv9_2.aux_weight
2024-01-22 13:04:29,662 WARNING: Loaded net - current net:
2024-01-22 13:04:29,683 INFO: Loss [L1Loss] is created.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv1_1.weight will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv1_1.bias will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv1_2.weight will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv1_2.bias will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv2_1.weight will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv2_1.bias will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv2_2.weight will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv2_2.bias will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv3_1.weight will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv3_1.bias will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv3_2.weight will not be optimized.
2024-01-22 13:04:29,683 WARNING: Params base_module.conv3_2.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv4_1.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv4_1.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv4_2.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv4_2.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv5_1.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv5_1.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv5_2.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv5_2.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.upv6.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.upv6.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv6_1.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv6_1.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv6_2.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv6_2.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.upv7.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.upv7.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv7_1.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv7_1.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv7_2.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv7_2.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.upv8.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.upv8.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv8_1.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv8_1.bias will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv8_2.weight will not be optimized.
2024-01-22 13:04:29,684 WARNING: Params base_module.conv8_2.bias will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params base_module.upv9.weight will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params base_module.upv9.bias will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params base_module.conv9_1.weight will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params base_module.conv9_1.bias will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params base_module.conv9_2.weight will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params base_module.conv9_2.bias will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params base_module.conv10_1.weight will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params base_module.conv10_1.bias will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.main_weight will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.main_bias will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_2.main_weight will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_2.main_bias will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,685 WARNING: Params repnr_module.conv1_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv1_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv1_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv1_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv1_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv1_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv1_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.main_weight will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.main_bias will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.main_weight will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.main_bias will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,686 WARNING: Params repnr_module.conv2_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv2_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv2_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv2_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.main_weight will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.main_bias will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.main_weight will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.main_bias will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,687 WARNING: Params repnr_module.conv3_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.main_weight will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.main_bias will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.main_weight will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.main_bias will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv4_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv5_1.main_weight will not be optimized.
2024-01-22 13:04:29,688 WARNING: Params repnr_module.conv5_1.main_bias will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.main_weight will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.main_bias will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv5_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.upv6.weight will not be optimized, though it requires grad!
2024-01-22 13:04:29,689 WARNING: Params repnr_module.upv6.bias will not be optimized, though it requires grad!
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv6_1.main_weight will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv6_1.main_bias will not be optimized.
2024-01-22 13:04:29,689 WARNING: Params repnr_module.conv6_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.main_weight will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.main_bias will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv6_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.upv7.weight will not be optimized, though it requires grad!
2024-01-22 13:04:29,690 WARNING: Params repnr_module.upv7.bias will not be optimized, though it requires grad!
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv7_1.main_weight will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv7_1.main_bias will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv7_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,690 WARNING: Params repnr_module.conv7_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.main_weight will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.main_bias will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv7_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.upv8.weight will not be optimized, though it requires grad!
2024-01-22 13:04:29,691 WARNING: Params repnr_module.upv8.bias will not be optimized, though it requires grad!
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv8_1.main_weight will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv8_1.main_bias will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv8_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv8_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,691 WARNING: Params repnr_module.conv8_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.main_weight will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.main_bias will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv8_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.upv9.weight will not be optimized, though it requires grad!
2024-01-22 13:04:29,692 WARNING: Params repnr_module.upv9.bias will not be optimized, though it requires grad!
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv9_1.main_weight will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv9_1.main_bias will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv9_1.align_weights.0 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv9_1.align_weights.1 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv9_1.align_weights.2 will not be optimized.
2024-01-22 13:04:29,692 WARNING: Params repnr_module.conv9_1.align_weights.3 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_1.align_weights.4 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_1.align_biases.0 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_1.align_biases.1 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_1.align_biases.2 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_1.align_biases.3 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_1.align_biases.4 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.main_weight will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.main_bias will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_weights.0 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_weights.1 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_weights.2 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_weights.3 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_weights.4 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_biases.0 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_biases.1 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_biases.2 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_biases.3 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv9_2.align_biases.4 will not be optimized.
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv10_1.weight will not be optimized, though it requires grad!
2024-01-22 13:04:29,693 WARNING: Params repnr_module.conv10_1.bias will not be optimized, though it requires grad!
2024-01-22 13:04:29,694 INFO: Model [LEDFinetuneModel] is created.
2024-01-22 13:04:29,879 INFO: Start training from epoch: 0, iter: 0
2024-01-22 13:05:38,652 INFO: [v0..][epoch: 0, iter: 200, lr:(1.000e-04,)] [eta: 0:06:33, time (data): 0.344 (0.010)] l_pix: 1.1457e-02
2024-01-22 13:06:40,049 INFO: [v0..][epoch: 0, iter: 400, lr:(1.000e-04,)] [eta: 0:05:34, time (data): 0.307 (0.007)] l_pix: 1.3481e-02
2024-01-22 13:07:41,510 INFO: [v0..][epoch: 0, iter: 600, lr:(1.000e-04,)] [eta: 0:04:34, time (data): 0.307 (0.007)] l_pix: 9.3888e-03
2024-01-22 13:08:42,970 INFO: [v0..][epoch: 0, iter: 800, lr:(1.000e-04,)] [eta: 0:03:33, time (data): 0.307 (0.006)] l_pix: 1.1264e-02
2024-01-22 13:09:44,459 INFO: [v0..][epoch: 0, iter: 1,000, lr:(1.000e-04,)] [eta: 0:02:32, time (data): 0.307 (0.006)] l_pix: 1.2119e-02
2024-01-22 13:09:44,475 INFO: Switch to optimize oomn branch....
2024-01-22 13:10:46,266 INFO: [v0..][epoch: 0, iter: 1,200, lr:(1.000e-05,)] [eta: 0:01:31, time (data): 0.309 (0.006)] l_pix: 1.0544e-02
2024-01-22 13:11:48,080 INFO: [v0..][epoch: 0, iter: 1,400, lr:(1.000e-05,)] [eta: 0:00:30, time (data): 0.309 (0.006)] l_pix: 7.9759e-03
2024-01-22 13:12:19,472 INFO: End of training. Time consumed: 0:07:49
2024-01-22 13:12:19,473 INFO: Save the latest model.