amazingren / pi-trans

The code of PI-Trans for cross-view image translation (ICASSP2023)

PI-TRANS (Accepted by ICASSP 2023)

PI-Trans: Parallel-ConvMLP and Implicit-Transformation Based GAN for Cross-View Image Translation
Bin Ren^1,2, Hao Tang³, Yiming Wang⁴, Xia Li³, Wei Wang⁵, Nicu Sebe².

¹University of Pisa, Italy.
²University of Trento, Italy.
³ETH, Switzerland.
⁴Fondazione Bruno Kessler (FBK), Italy
⁵Beijing Jiaotong University, China

The repository offers the official implementation of our paper in PyTorch.

🦖News(Feb 26, 2023)! We are organizaing our code, the code will be released soon!

Installation (Environment)

Dataset Preparation

• For Dayton and CVUSA, the datasets must be downloaded beforehand. Please download them on the respective webpages. In addition, we put a few sample images in this code repo data samples. Please cite their papers if you use the data.

• Preparing Ablation Dataset. We conduct ablation study in a2g (aerialto-ground) direction on Dayton dataset. To reduce the training time, we randomly select 1/3 samples from the whole 55,000/21,048 samples i.e. around 18,334 samples for training and 7,017 samples for testing. The trianing and testing splits can be downloaded here.

• Preparing Dayton Dataset. The dataset can be downloaded here. In particular, you will need to download dayton.zip. Ground Truth semantic maps are not available for this datasets. We adopt RefineNet trained on CityScapes dataset for generating semantic maps and use them as training data in our experiments. Please cite their papers if you use this dataset. Train/Test splits for Dayton dataset can be downloaded from here.

• Preparing CVUSA Dataset. The dataset can be downloaded here. After unzipping the dataset, prepare the training and testing data as discussed in our CrossMLP. We also convert semantic maps to the color ones by using this script. Since there is no semantic maps for the aerial images on this dataset, we use black images as aerial semantic maps for placehold purposes.

🌲 Note that for your convenience we also provide download scripts:

bash ./datasets/download_selectiongan_dataset.sh [dataset_name]

[dataset_name] can be:

• dayton_ablation : 5.7 GB
• dayton: 17.0 GB
• cvusa: 1.3 GB

Training

Testing（Generating Images）

Evaluation (Numerical Results)

Contributions

If you have any questions/comments/bug reports, feel free to open a github issue or pull a request or e-mail to the author Bin Ren ([email protected]).

Acknowledgments

This source code borrows heavily from Pix2pix and SelectionGAN. We also thank the authors X-Fork & X-Seq for providing the evaluation codes. This work was supported by...