linrl3 / image-stitching-opencv

Simple image stitching algorithm based on SIFT, homography, KNN and Ransac in Python and OpenCV

Image-Stitching-OpenCV

Simple image stitching algorithm using SIFT, homography, KNN and Ransac in Python. For full details and explanations, you're welcome to read image_stitching.pdf.

The project is to implement a featured based automatic image stitching algorithm. When we input two images with overlapped fields, we expect to obtain a wide seamless panorama.

We use scale invariant features transform(SIFT) to extract local features of the input images, K nearest neighbors algorithms to match these features and Random sample consensus(Ransac) to calculate the homograph matrix, which will be used for image warping. Finally we apply a weighted matrix as a mask for image blending.

Dependency

• Python 2 or 3
• OpenCV 3

Usage

python Image_Stitching [/PATH/img1] [/PATH/img2]

Sample

Input images

Matching

Output image

other examples

Room:

Another building: This is a stitching of three related images. It doesn't work so well. Trying to improve.

image-stitching-opencv's Issues

Value error

Hi;
My images can not make mask on this line can you help me please.
`---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
in
7 if name == 'main':
8 try:
----> 9 main()
10 except IndexError:
11 print ("Please input two source images: ")

in main()
2 img1 = cv2.imread('data/Rainier1.png')
3 img2 = cv2.imread('data/Rainier2.png')
----> 4 final=Image_Stitching().blending(img1,img2)
5 cv2.imwrite('panorama.jpg', final)
6

in blending(self, img1, img2)
53
54 panorama1 = np.zeros((height_panorama, width_panorama, 3))
---> 55 mask1 = self.create_mask(img1,img2,version='left_image')
56 panorama1[0:img1.shape[0], 0:img1.shape[1], :] = img1
57 panorama1 *= mask1

in create_mask(self, img1, img2, version)
37 mask = np.zeros((height_panorama, width_panorama))
38 if version== 'left_image':
---> 39 mask[:, barrier - offset:barrier + offset ] = np.tile(np.linspace(1, 0, 2 * offset ).T, (height_panorama, 1))
40 mask[:, :barrier - offset] = 1
41 else:

ValueError: could not broadcast input array from shape (388,800) into shape (388,0)`
Images link:

https://drive.google.com/drive/folders/1nyWK45IjSofDFvkWRkJqtYx3Gn1n6QSp?usp=sharing

Thank you then.

module 'cv2.cv2' has no attribute 'xfeatures2d'

python Image_Stitching.py [image1.jpg] [image2.jpg]
Traceback (most recent call last):
File "Image_Stitching.py", line 78, in
main(sys.argv[1],sys.argv[2])
File "Image_Stitching.py", line 74, in main
final=Image_Stitching().blending(img1,img2)
File "Image_Stitching.py", line 9, in init
self.sift=cv2.xfeatures2d.SIFT_create()
AttributeError: module 'cv2.cv2' has no attribute 'xfeatures2d'

UnboundLocalError: local variable 'H' referenced before assignment

Traceback (most recent call last):
  File "d:\Image-Stitching-OpenCV\Image_Stitching.py", line 92, in <module>
    main(imageA,imageB)
  File "d:\Image-Stitching-OpenCV\Image_Stitching.py", line 74, in main
    final=Image_Stitching().blending(imageA,imageB)
  File "d:\Image-Stitching-OpenCV\Image_Stitching.py", line 51, in blending
    H = self.registration(img1,img2)
  File "d:\Image-Stitching-OpenCV\Image_Stitching.py", line 31, in registration
    return H
UnboundLocalError: local variable 'H' referenced before assignment

Cannot imshow final result

I tried to cv2.imshow the final result, but I only got a black & white image.
can you imshow your result?

you're welcome to read image_stitching.pdf

“you're welcome to read image_stitching.pdf” where i can get the pdf

License

Please, kindly provide a license to give us a legal statement to use your code, Sir.

How to three pictures

import cv2
import numpy as np
import sys

class Image_Stitching():
    def __init__(self) :
        self.ratio=0.85
        self.min_match=10
        self.sift=cv2.SIFT_create()
        self.smoothing_window_size=800

    def registration(self,img1,img2,img3):
        kp1, des1 = self.sift.detectAndCompute(img1, None)
        kp2, des2 = self.sift.detectAndCompute(img2, None)
        kp3, des3 = self.sift.detectAndCompute(img3, None)

        matcher = cv2.BFMatcher()
        raw_matches = matcher.knnMatch(des1, des2, des3, k=3)
        good_points = []
        good_matches=[]
        for m1, m2 in raw_matches:
            if m1.distance < self.ratio * m2.distance:
                good_points.append((m1.trainIdx, m1.queryIdx))
                good_matches.append([m1])
        img3 = cv2.drawMatchesKnn(img1, kp1, img2, kp2, img3, kp3, good_matches, None, flags=3)
        cv2.imwrite('matching.jpg', img3)
        if len(good_points) > self.min_match:
            image1_kp = np.float32(
                [kp1[i].pt for (_, i) in good_points])
            image2_kp = np.float32(
                [kp2[i].pt for (i, _) in good_points])
            image3_kp = np.float32(
                [kp3[i].pt for (i, _) in good_points])
            H, status = cv2.findHomography(image2_kp, image1_kp,image3_kp, cv2.RANSAC,5.0)
        return H

    def create_mask(self,img1,img2,img3,version):
        height_img1 = img1.shape[0]
        width_img1 = img1.shape[1]
        width_img2 = img2.shape[1]
        width_img3 = img3.shape[1]
        height_panorama = height_img1
        width_panorama = width_img1 +width_img2 +width_img3
        offset = int(self.smoothing_window_size / 2)
        barrier = img1.shape[1] - int(self.smoothing_window_size / 2)
        mask = np.zeros((height_panorama, width_panorama))
        if version== 'left_image':
            mask[:, barrier - offset:barrier + offset ] = np.tile(np.linspace(1, 0, 2 * offset ).T, (height_panorama, 1))
            mask[:, :barrier - offset] = 1
        else:
            mask[:, barrier - offset :barrier + offset ] = np.tile(np.linspace(0, 1, 2 * offset ).T, (height_panorama, 1))
            mask[:, barrier + offset:] = 1
        return cv2.merge([mask, mask, mask])

    def blending(self,img1,img2,img3):
        H = self.registration(img1,img2,img3)
        height_img1 = img1.shape[0]
        width_img1 = img1.shape[1]
        width_img2 = img2.shape[1]
        width_img3 = img3.shape[1]
        height_panorama = height_img1
        width_panorama = width_img1 +width_img2 +width_img3

        panorama1 = np.zeros((height_panorama, width_panorama, 3))
        mask1 = self.create_mask(img1,img2,img3,version='left_image')
        panorama1[0:img1.shape[0], 0:img1.shape[1], :] = img1
        panorama1 *= mask1
        mask2 = self.create_mask(img1,img2,img3,version='right_image')
        panorama2 = cv2.warpPerspective(img2, H, (width_panorama, height_panorama))*mask2

        panorama1 *= mask2
        mask3 = self.create_mask(img1,img2,img3,version='right_image')
        panorama3 = cv2.warpPerspective(img3, H, (width_panorama, height_panorama))*mask3

        result=panorama1+panorama2+panorama3

        rows, cols = np.where(result[:, :, 0] != 0)
        min_row, max_row = min(rows), max(rows) + 1
        min_col, max_col = min(cols), max(cols) + 1
        final_result = result[min_row:max_row, min_col:max_col, :]
        return final_result
def main(argv1,argv2,argv3):
    img1 = cv2.imread(argv1)
    img2 = cv2.imread(argv2)
    img3 = cv2.imread(argv3)

    final=Image_Stitching().blending(img1,img2,img3)
    cv2.imwrite('panorama.jpg', final)
if __name__ == '__main__':
    try: 
        main(sys.argv[1],sys.argv[2],sys.argv[3])
    except IndexError:
        print ("Please input two source images: ")
        print ("For example: python Image_Stitching.py '/Users/linrl3/Desktop/picture/p1.jpg' '/Users/linrl3/Desktop/picture/p2.jpg'")
    

My method is wrong

Runtime improvement

I know that is not really the point of the script, but since I used it modified for a real-time application I wanted to share this:
In lines 67 and 68, you find the minimum/maximum using python's build-in version. using np.min() and np.max significantly improves runtime.

OpeCV already has implement named Stitcher

Sorry for saying in a rude way, as i know, OpenCV already has implement named Stitcher,which can realize image stitching quicker and better,maybe u can know this?