flowerduo / ges-gsp-stitching

The repository contains our dataset and C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching.

Geometric Structure Preserving Warp for Natural Image Stitching

This repository contains our dataset and C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching. If you use any code or data from our work, please cite our paper.

Figure 1. An example of stitching 10 images. (a) The AutoStitch's [8] result is severely distorted. (b) The person on the right side is distorted in the APAP's [7] result. (c) Several misalignments (red and green closeup) in the ELA’s [9] result. (d) The SPW's [10] result exhibits significant wrong scale at the right end. (e) There are distortions in the red box, e.g., the floor and carpet are curved in the result obtained by GSP [2]. (f) Our result preserves the salient geometric structures in scene.

Download

1. Paper+Supplementary
2. Code
   • GitHub
   • Baidu NetDisk
3. DataSet (GES-50)
   • GitHub
   • Google NetDisk
   • Baidu NetDisk
4. Android Appication(Harmony)
   • Server download
   • Baidu NetDisk

Code

1. Usage

(1). Download code and comile.
	You need Opencv 4.4.0, VLFEAT, Eigen.
(2). Download dataset to "input-data" folder.
(3). Run project.

Or

(4). We provide scripts that make it easier to test data. The following are the steps:
(5). Edit "RUN_EXE.bat". 
	Change "file=\RUN_FILE.txt" and "\GES_Stitching.exe" to corresponding path.
(6). List dataset names you want to test in "RUN_FILE.txt".
(7). Click "RUN_EXE.bat".

Notice:

• If you make changes to the code, you can copy .exe from the "x64" to the root directory and rename it to "GES_Stitching.exe" after running project.
• If the .exe output errors, try to run the project to get a new .exe.

You can find results in folder "input-data".

Dataset

1. Introduction

There are 50 diversified and challenging dataset (26 from [1–7] and 24 collected by ourselves). The numbers of images range from 2 to 35.

2. Usage

(1). Copy dataset to folder "input-data" in project.
(2). Make sure the file "xxx-STITCH-GRAPH.txt" in each dataset correspond to the name of this dataset.
(3). You can change the relation between the images by modifying the file "xxx-STITCH-GRAPH.txt".

Android(Harmony) Application

1. Introduction

Based on the C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching, we have developed an Android(Harmony) application.

With our Android(Harmony) application, you can easily perform image stitching and obtain large-scale images in various fields such as cultural tourism, smart agriculture, and security monitoring. You can effortlessly complete the stitching process with astonishing speed while ensuring high-quality results

We feel sorry, but currently this application only supports Chinese. However, you can follow our instructions to use it.

2. Guide

(1). Download and install the package on an Android(Harmony) phone.
(2). Apply for a trial account and log in.
(3). Select to import from the gallery or capture images for stitching.
(4). Select 'Speed Priority'(Left) or 'Quality Priority'(Right) and then click on the top-right corner to start the stitching process.
(5). After obtaining the stitching result, you can choose to perform operations such as cropping, saving, sharing, and more.

3. Part of the software screenshot

Figure 2 Part of the software screenshot

4. Account application

Welcome to download the Android application. If you need a trial account, please contact us via email([email protected])

Contact

Feel free to contact me if there is any question ([email protected]).

Reference

1. Che-Han Chang, Yoichi Sato, and Yung-Yu Chuang. Shapepreserving half-projective warps for image stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3254–3261, 2014.
2. Yu-Sheng Chen and Yung-Yu Chuang. Natural image stitching with the global similarity prior. In European conference on computer vision, pages 186–201. Springer, 2016.
3. Junhong Gao, Seon Joo Kim, and Michael S Brown. Constructing image panoramas using dual-homography warping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 49–56. IEEE, 2011.
4. Qi Jia, ZhengJun Li, Xin Fan, Haotian Zhao, Shiyu Teng,Xinchen Ye, and Longin Jan Latecki. Leveraging line-point consistence to preserve structures for wide parallax image stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 12186–12195,2021.
5. Chung-Ching Lin, Sharathchandra U Pankanti, Karthikeyan Natesan Ramamurthy, and Aleksandr Y Aravkin. Adaptive as-natural-as-possible image stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1155–1163, 2015.
6. Yoshikuni Nomura, Li Zhang, and Shree K Nayar. Scene collages and flexible camera arrays. In Proceedings of the 18th Eurographics conference on Rendering Techniques, pages 127–138, 2007.
7. Julio Zaragoza, Tat-Jun Chin, Michael S Brown, and David Suter. As-projective-as-possible image stitching with moving dlt. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2339–2346, 2013.
8. Matthew Brown and David G Lowe. Automatic panoramic image stitching using invariant features. International journal of computer vision, 74(1):59–73, 2007.
9. Jing Li, Zhengming Wang, Shiming Lai, Yongping Zhai, and Maojun Zhang. Parallax-tolerant image stitching based on robust elastic warping. IEEE Transactions on multimedia, 20(7):1672–1687, 2017.
10. Tianli Liao and Nan Li. Single-perspective warps in natural image stitching. IEEE Transactions on Image Processing, 29:724–735, 2019.

ges-gsp-stitching's Issues

How to build this repo with CMake in ubuntu machine ?

Kindly help with all the steps required for building this repo in ubuntu machine with CMake by providing all the required libraries and their versions and the CMakeList.txt file .

shit

this project is the worst project i have ever seen. nobody can run if from zero , because it needs so many libraries, and you did not say it clearly, this project is really cost time.

错误 MSB4019 找不到导入的项目“D:\Projects\Opencv_Release.props”。

错误 MSB4019 找不到导入的项目“D:\Projects\Opencv_Release.props”。请确认 Import 声明“....\Opencv_Release.props”中的表达式正确，且文件位于磁盘上。 GES_Stitching D:\Projects\GES-GSP-Stitching\Code\GES_Stitching.vcxproj 130

程序编译正确，但是用来做拼接测试报错

您好，我已经把环境配好，生成exe成功了，但是用bat文件测试拼接图片还是报错，如下：

Start NISwGSP-01_SantaMaria ---------------------------------
nThreads = 20
[#Images : 1]
i = 1, [Images : NISwGSP-01_SantaMaria]
[ERROR] F(getImageFileFullNamesInDir) could not open directory
[ERROR] F(getImagesMatchGraph) image match graph verification [2] didn't be implemented yet
[ERROR] F(getImagesMatchGraph) image match graph verification [2] didn't be implemented yet
[ERROR] F(getImagesMatchGraph) image match graph verification [2] didn't be implemented yet
[ERROR] F(getImagesMatchGraph) image match graph verification [2] didn't be implemented yet
A = [2, 0]
Assertion failed: it->row()>=0 && it->row()<mat.rows() && it->col()>=0 && it->col()<mat.cols(), file D:\ProgramFiles64\OpenCV\eigen-3.3.9\Eigen\src\SparseCore\SparseMatrix.h, line 935

我是把bat、dll依赖、图片、txt文本和exe程序放在一个文件夹，然后运行bat，请问是我的测试方式不对吗？
您发布的exe我测试也是报错，不是完全相同，但是类似：

Start test_library ---------------------------------
nThreads = 1
[#Images : 1]
i = 1, [Images : test_library]
[ERROR] F(getImageFileFullNamesInDir) could not open directory
[ERROR] F(getImagesMatchGraph) image match graph verification [2] didn't be implemented yet
[ERROR] F(getImagesMatchGraph) image match graph verification [2] didn't be implemented yet
[ERROR] F(getImagesMatchGraph) image match graph verification [2] didn't be implemented yet
[ERROR] F(getImagesMatchGraph) image match graph verification [2] didn't be implemented yet
A = [2, 0]
Finish test_library ---------------------------------
All finish ---------------------------------
请按任意键继续. . .

希望您能抽取宝贵的时间为我解惑，谢谢！

Missing vl/sift.h file can be added

Missing vl/sift.h file can be added

Stitched images are not created

After compiling the project, I ran ges_stitching.exe using run_exe.bat.
The message is displayed as below, but nothing exists in the 0_result and 1_debugs folders of the input-data folder.

 `Start AANAP-01_skyline ---------------------------------
nThreads = 12
[#Images : 1]
i = 1, [Images : AANAP-01_skyline]
center_image_index = 3
center_image_rotation_angle = 0
images_count = 4
Finish AANAP-01_skyline ---------------------------------
All finish ---------------------------------
Press any key to continue. .`

Same thing happens with other datasets. For confirmation, when a photo or AANAP-01_skyline-STITCH-GRAPH.txt is removed from the AANAP-01_skyline folder, an error occurs normally.

🔥 Refer to OBJ-GSP if you have difficulties in running this repo.

You may want to refer to OBJ-GSP repo, an improved version of GES-GSP, to see instructions in running GES-GSP and OBJ-GSP.

It takes some time to configure c++ codes and run projects on own pcs. GES-GSP is a great and state-of-the-art work in image stitching. However, some files GES-GSP repo are missing, and the provided .sln may not work well on your pcs.

Some quick notes:

1. Run on windows and Visual Studio 2019
2. Be sure to compile OpenCV 4.4.0 correctly before running. We are not responsible to answer any questions regarding OpenCV compilation.
3. .sln and Visual Studio projects need to be configured right, some frequently encountered problems include: unresolved external symbol, missing .dl. Google them!
4. set const int RUN_TYPE = 1; in Configure.h to get GES-GSP results.

Feel free to open issues in OBJ-GSP or drop an email to me if you have any problems: [email protected].

does it really work?

if it does not work, please no confuse me.

缺少这个库，能加一下吗？

i have run successfully.

i have solved all problem,you can contact me if you have any questions