zhanglichao / generatedtir_tracking Goto Github PK

original image:

chip:

ir:

code:

    opt = """
    aspect_ratio: 1.0
    batchSize: 1
    checkpoints_dir: ./checkpoints
    dataroot: /home/lichao/tracking/datasets/GOT-10k/train
    dataset_mode: single
    display_id: 1
    display_port: 8097
    display_winsize: 256
    fineSize: 256
    gpu_ids: []
    how_many: 10000000
    init_type: normal
    init_gain: 0.02
    input_nc: 3
    isTrain: False
    loadSize: 256
    max_dataset_size: inf
    model: test
    nThreads: 2
    n_layers_D: 3
    name: thermal_pix2pix_1ch_all_batch4
    ndf: 64
    ngf: 64
    no_dropout: True
    no_flip: False
    norm: batch
    ntest: inf
    output_nc: 1
    phase: test
    preprocess: 
    resize_or_crop: resize_and_crop
    results_dir: /home/lichao/tracking/datasets/GOT-10k_i_/train
    serial_batches: False
    which_direction: BtoA
    which_epoch: 25
    netD: basic
    netG: unet_256
    """
    opt = objectview(yaml.load(opt))
    p = Pix2PixModel(opt)
    load_model_state(p.netG, "/home/maksim/Downloads/net_G_pix2pix.pth")
    image = Image.open("/home/maksim/data/FLIR_ADAS_1_3/train/RGB/FLIR_00006.jpg")
    d = 1024
    x, y = 500, 516

    image = image.crop((x, y, x+d, y+d))
    image_np = np.asarray(image)
    plt.imshow(image_np)
    plt.show()
    image_tensor = ToTensor()(image_np).float().unsqueeze(0)
    ir = p.netG.forward(image_tensor)
    plt.imshow(ir.data.numpy()[0,0], cmap="gray")
    plt.show()

Hello, I did experiments on my own dataset. However, the Generated thermal infrared images are blurred.The command I entered is as follows：python test_pix2pix_got.py --dataroot ./datasets/facades --name thermal_pix2pix_1ch_all_batch4 --model test --dataset_mode single --which_model_netG unet_256 --which_direction AtoB --norm batch --which_epoch 25 --results_dir ./result --output_nc 1 --no_dropout --loadSize 256 --gpu_ids 0

Hi zhang,
I generated the TIR iamges by using pix2pix weight you have provided, but I got bad result. Could you give me the best model weight

I tried to use your way to generate thermal images,but I cannot get the results like the results in the paper. So do you have a plan to open the generated dataset,thanks for your answers.

And do you use this code to generate images:https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

We will follow your work, thanks for sharing:-)

Hi,

Thank you for your very nice work!

I am now trying to test with your trained model using my RGB data to generate infrared data. But I got the following error:

"size mismatch for model.model.3.weight: copying a param with shape torch.Size([128, 1, 4, 4]) from checkpoint, the shape in current model is torch.Size([128, 3, 4, 4])."

do you have any idea why this happen and how to solve this?

Thanks a lot!

Hi, thank you for sharing. I downloaded your weight file. But I encountered a problem using https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix. CycleGAN can generate thermal infrared images, but pix2pix reports an error. I enter the command in the terminal as: python test.py --dataroot ./datasets/visible/ --direction AtoB --model test --name net_G_pix2pix. But the error message is AttributeError: 'Sequential' object has no attribute 'model'.

Hello.
I'm very impressed by your study results so I want to run your tracker.
However, while 'ECO_tir' tracker is running, I got an error, "Reference to nonexistent field'net'".

This is my procedure :

1. download vot tool kit and create VOT_workspace. ( experiment=vottir2016, identifier=ECO_tir, language=matlab,...)
2. download  matconvnet and compiled it. set it's path on setup_paths.m with my local path
3. set 'ECO_repo_path' on VOT_integration/tracker_ECO_tir.m  with my local  repo. path.
4. copy tracker_EOC_tir.m  into my VOT workspace folder.
5. run run_experiments.

This is errors printed command lines :

Initializing workspace ...
Verifying native components ...
Experiment baseline
  Tracker ECO_tir
    Sequence quadrocopter2
      Repetition 1
      Tracker execution interrupted: Unable to establish connection.
An error occurred while using: traxclient
Unable to establish connection.

ERROR: tracker_run (line 77)
    data = traxclient(tracker.command, callback, ...

ERROR: experiment_supervised (line 71)
    data = tracker_run(tracker, @callback, data);

ERROR: tracker_evaluate (line 57)
	[files, metadata] = experiment_function(tracker, sequence, directory, parameters, scan);

ERROR: workspace_evaluate>execute_iterator (line 128)
            tracker_evaluate(event.tracker, event.sequence, event.experiment);

ERROR: iterate (line 65)
            context = iterator(event, context);

ERROR: workspace_evaluate (line 93)
context = iterate(experiments, trackers, sequences, 'iterator', iterator, 'context', context);

ERROR: run_experiments (line 10)
workspace_evaluate(tracker, sequences, experiments);

This is error information from VOT_integration\error_reports\ECO_OTB_DEEP_settings_tircnn_200829_1327.mat

1. error
   error.message means "Reference to nonexistent field'net'".
   

2. error. stack
   

According to stack, this error occurs at the 93th line in 'init features.m'.
net = features{k}.fparams.net;
=>This error means 'fparams.net' field was not set before .
Could you please give any advice?
Thanks in advance.

I downloaded "net_G_pix2pix.pth" this model . Then rename it to "25_net_G.pth" and put it under checkpoints / thermal_pix2pix_1ch_all_batch4 directory.
Use it to generate thermal infrared pictures.
But result was very bad. Did I have a problem somewhere?

Thank you for your outstanding contribution to infrared image generation.I am currently working on multi-modality research.
I would like to ask you about the issue of infrared data. I have tested your public code,It doesn't seem to have worked out as well as it could.Can you give me the best model?

Hi,

Can this model be used to general images with original size? I have tested the model, but all images are resized to 256*256.

Thanks!
Xingchen

Hello,

This ins't an issue, I was more wondering if you plan on releasing a version of your trained pix2pix or/and CycleGAN from the paper: Synthetic data generation for end-to-end thermal infrared tracking ?
Did you use the original implementation of pix2pix and CycleGAN or a Tensorflow/Pytorch implementation?
And can we expect the TIR images made from pix2pix to be available as a dataset?

In the meantime, thanks for your time and efforts.

Just wondering if you can add a license. I am hoping it is MIT.