Coder Social home page Coder Social logo

feature matching? about tf-lift HOT 28 OPEN

cvlab-epfl avatar cvlab-epfl commented on July 29, 2024
feature matching?

from tf-lift.

Comments (28)

hjf579068 avatar hjf579068 commented on July 29, 2024 2

@kmyid Hi, is this the right feature matching code?

import h5py
import cv2
import numpy as np
from utils.kp_tools import kp_list_2_opencv_kp_list

img1 = cv2.imread('input/1.png')
img2 = cv2.imread('input/2.png')

f1 = h5py.File('output/1_desc.h5')
f2 = h5py.File('output/2_desc.h5')

kp1 = np.array(f1['keypoints'].value)
kp2 = np.array(f2['keypoints'].value)

opencv_kp1 = kp_list_2_opencv_kp_list(kp1)
opencv_kp2 = kp_list_2_opencv_kp_list(kp2)

des1 = np.array(f1['descriptors'].value)
des2 = np.array(f2['descriptors'].value)

bf = cv2.BFMatcher()
matches = bf.knnMatch(des1, des2, k=2)
good = []
for m, n in matches:
if m.distance < 0.4*n.distance:
good.append([m])

img5 = cv2.drawMatchesKnn(img1,opencv_kp1,img2,opencv_kp2,good,None,flags=2)
cv2.imwrite('output/test12.jpg', img5)
cv2.imshow("BFmatch", img5)
cv2.waitKey(0)
cv2.destroyAllWindows()

from tf-lift.

Shu-HowTing avatar Shu-HowTing commented on July 29, 2024 1

@abeermohamed1 Here is my code, hope to help you.

import h5py
import cv2
import numpy as np


img1 = cv2.imread('./1_left.jpg')
img2 = cv2.imread('./1_right.jpg')


f1 = h5py.File('./out/image_left_desc.h5')
f2 = h5py.File('./out/image_right_desc.h5')

kp1 = np.array(f1['keypoints'].value)
kp2 = np.array(f2['keypoints'].value)

des1 = np.array(f1['descriptors'].value)
des2 = np.array(f2['descriptors'].value)

# matcher = cv2.BFMatcher_create(cv2.NORM_L2)
# matches = matcher.match(des1, des2)
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks = 50)
flann = cv2.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(des1,des2,k=2)
good = []
for m,n in matches:
    if m.distance < 0.7*n.distance:
        good.append(m)

height = max(img1.shape[0], img2.shape[0])
width = img1.shape[1] + img2.shape[1]
output = np.zeros((height, width, 3), dtype=np.uint8)
output[0:img1.shape[0], 0:img2.shape[1]] = img1
output[0:img2.shape[0], img1.shape[1]:] = img2[:]
for i in range(len(good)):
    left = kp1[good[i].queryIdx][:2]
    right = tuple(sum(x) for x in zip(kp2[good[i].trainIdx][:2], (img1.shape[1], 0)))
    cv2.line(output, tuple(map(int, left)), tuple(map(int, right)), (0, 255, 0), lineType= cv2.LINE_AA)


cv2.imwrite("./Lift.jpg", output, [int(cv2.IMWRITE_PNG_COMPRESSION), 0])
cv2.imshow('out', output)
cv2.waitKey()

from tf-lift.

luczeng avatar luczeng commented on July 29, 2024

OpenCV has some good toolbox for feature matching : https://docs.opencv.org/3.3.0/dc/dc3/tutorial_py_matcher.html

from tf-lift.

ytongbai avatar ytongbai commented on July 29, 2024

@TheToadAlly Hi, I tried the toolbox of feature matching that you provided, but I am in trouble with the data format transition.
I use this method to transit the value read from h5 file:
des1 = f1['descriptors'][()]
kp1 = np.float32(f1['keypoints'][:]).reshape(-1,1,2)

And after I ran the code, I got this error:

img3 = cv2.drawMatches(img1, kp1, img2, kp2, good, None, flags=2)
TypeError: Expected cv::KeyPoint for argument 'keypoints1'

Could you help me with this question? Or can you send me the matching part code? It bothers me for a really long time.

Thanks!

from tf-lift.

chuanzhidong avatar chuanzhidong commented on July 29, 2024

@TheToadAlly Hi, I tried the toolbox of feature matching that you provided, but I am in trouble with the data format transition.
I use this method to transit the value read from h5 file:
des1 = f1['descriptors'][()]
kp1 = np.float32(f1['keypoints'][:]).reshape(-1,1,2)

And after I ran the code, I got this error:

img3 = cv2.drawMatches(img1, kp1, img2, kp2, good, None, flags=2)
TypeError: Expected cv::KeyPoint for argument 'keypoints1'

Could you help me with this question? Or can you send me the matching part code? It bothers me for a really long time.

Thanks!

@ytongbai Hi, I also face this problem. Did you solve that?

from tf-lift.

Shu-HowTing avatar Shu-HowTing commented on July 29, 2024

@wisemaker Hi,I face the problem same with you,Did you solve that?

from tf-lift.

hudsonmartins avatar hudsonmartins commented on July 29, 2024

Hi guys, maybe its late but you can solve it by using the kp_list_2_opencv_kp_list function available on utils/kp_tools.py

from tf-lift.

Shu-HowTing avatar Shu-HowTing commented on July 29, 2024

@hudsonmartins Hi,i have solved it in my way ,Anyway, thank you for your reply!

from tf-lift.

abeermohamed1 avatar abeermohamed1 commented on July 29, 2024

@hudsonmartins Hi,i have solved it in my way ,Anyway, thank you for your reply!

Hi, how did you solve it? please help

from tf-lift.

kmyi avatar kmyi commented on July 29, 2024

Just as an heads up, it is harmful to do the NN ratio testing for LIFT. That test is only valid for SIFT

from tf-lift.

kmyi avatar kmyi commented on July 29, 2024

No. You are still doing the nn-ratio test here.

from tf-lift.

hjf579068 avatar hjf579068 commented on July 29, 2024

@kmyid OK, thank you, and can you provide your matching code?

from tf-lift.

abeermohamed1 avatar abeermohamed1 commented on July 29, 2024

@abeermohamed1 Here is my code, hope to help you.

import h5py
import cv2
import numpy as np


img1 = cv2.imread('./1_left.jpg')
img2 = cv2.imread('./1_right.jpg')


f1 = h5py.File('./out/image_left_desc.h5')
f2 = h5py.File('./out/image_right_desc.h5')

kp1 = np.array(f1['keypoints'].value)
kp2 = np.array(f2['keypoints'].value)

des1 = np.array(f1['descriptors'].value)
des2 = np.array(f2['descriptors'].value)

# matcher = cv2.BFMatcher_create(cv2.NORM_L2)
# matches = matcher.match(des1, des2)
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks = 50)
flann = cv2.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(des1,des2,k=2)
good = []
for m,n in matches:
    if m.distance < 0.7*n.distance:
        good.append(m)

height = max(img1.shape[0], img2.shape[0])
width = img1.shape[1] + img2.shape[1]
output = np.zeros((height, width, 3), dtype=np.uint8)
output[0:img1.shape[0], 0:img2.shape[1]] = img1
output[0:img2.shape[0], img1.shape[1]:] = img2[:]
for i in range(len(good)):
    left = kp1[good[i].queryIdx][:2]
    right = tuple(sum(x) for x in zip(kp2[good[i].trainIdx][:2], (img1.shape[1], 0)))
    cv2.line(output, tuple(map(int, left)), tuple(map(int, right)), (0, 255, 0), lineType= cv2.LINE_AA)


cv2.imwrite("./Lift.jpg", output, [int(cv2.IMWRITE_PNG_COMPRESSION), 0])
cv2.imshow('out', output)
cv2.waitKey()

Thank you

from tf-lift.

abeermohamed1 avatar abeermohamed1 commented on July 29, 2024

@kmyid Hi, is this the right feature matching code?

import h5py
import cv2
import numpy as np
from utils.kp_tools import kp_list_2_opencv_kp_list

img1 = cv2.imread('input/1.png')
img2 = cv2.imread('input/2.png')

f1 = h5py.File('output/1_desc.h5')
f2 = h5py.File('output/2_desc.h5')

kp1 = np.array(f1['keypoints'].value)
kp2 = np.array(f2['keypoints'].value)

opencv_kp1 = kp_list_2_opencv_kp_list(kp1)
opencv_kp2 = kp_list_2_opencv_kp_list(kp2)

des1 = np.array(f1['descriptors'].value)
des2 = np.array(f2['descriptors'].value)

bf = cv2.BFMatcher()
matches = bf.knnMatch(des1, des2, k=2)
good = []
for m, n in matches:
if m.distance < 0.4*n.distance:
good.append([m])

img5 = cv2.drawMatchesKnn(img1,opencv_kp1,img2,opencv_kp2,good,None,flags=2)
cv2.imwrite('output/test12.jpg', img5)
cv2.imshow("BFmatch", img5)
cv2.waitKey(0)
cv2.destroyAllWindows()

thanks for sharing

from tf-lift.

kmyi avatar kmyi commented on July 29, 2024

Just remove

for m,n in matches:
    if m.distance < 0.7*n.distance:
        good.append(m)

these lines. Don't do the ratio test.

from tf-lift.

kmyi avatar kmyi commented on July 29, 2024

Remove

good = []
for m, n in matches:
if m.distance < 0.4*n.distance:
good.append([m])

from tf-lift.

hjf579068 avatar hjf579068 commented on July 29, 2024

@kmyid Ok, thank you very much!

from tf-lift.

qiuweibo avatar qiuweibo commented on July 29, 2024

Remove

good = []
for m, n in matches:
if m.distance < 0.4*n.distance:
good.append([m])

Hi Professor,
I have two questions:

  1. Why the NN-ratio test is harmful to LIFT? BF-Knn match return k best matches, and NN-ratio test is just picking up the good matches. Why it will affect LIFT?

  2. Also is RANSAC still applicable to LIFT during matching ?

Best regards,
Weibo Qiu.

from tf-lift.

kmyi avatar kmyi commented on July 29, 2024

Why the NN-ratio test is harmful to LIFT? BF-Knn match return k best matches, and NN-ratio test is just picking up the good matches. Why it will affect LIFT?

NN-Ratio is bad, because of the fact that the distribution of descriptor distances for positives and negatives are not the same as SIFT. To do this properly, you need to run a test on a dataset to figure out the distance distributions for pos/neg pairs, and then use that threshold.

Also is RANSAC still applicable to LIFT during matching ?

Yes.

from tf-lift.

qiuweibo avatar qiuweibo commented on July 29, 2024

Why the NN-ratio test is harmful to LIFT? BF-Knn match return k best matches, and NN-ratio test is just picking up the good matches. Why it will affect LIFT?

NN-Ratio is bad, because of the fact that the distribution of descriptor distances for positives and negatives are not the same as SIFT. To do this properly, you need to run a test on a dataset to figure out the distance distributions for pos/neg pairs, and then use that threshold.

Also is RANSAC still applicable to LIFT during matching ?

Yes.

Thanks so much for your quick reply!

from tf-lift.

qiuweibo avatar qiuweibo commented on July 29, 2024

Why the NN-ratio test is harmful to LIFT? BF-Knn match return k best matches, and NN-ratio test is just picking up the good matches. Why it will affect LIFT?

NN-Ratio is bad, because of the fact that the distribution of descriptor distances for positives and negatives are not the same as SIFT. To do this properly, you need to run a test on a dataset to figure out the distance distributions for pos/neg pairs, and then use that threshold.

Also is RANSAC still applicable to LIFT during matching ?

Yes.

One more question:

Is TILDE fine with NN-ratio test?
Since Tilde is trained only on keypoints extraction, I generated descriptor using ORB.

Thanks in advance.

Best regards,
Weibo

from tf-lift.

kmyi avatar kmyi commented on July 29, 2024

Using NN-Ratio test with ANY descriptor is not fine. You need to use different ratio's for that. Which is why we did not do that test in comparing different methods.

from tf-lift.

qiuweibo avatar qiuweibo commented on July 29, 2024

Dear Professor,

In matching stage, did you use Brute Force Hamming distance matching for LIFT?
If you use Hamming distance, so LIFT is a binary descriptor?

Thanks!

Best regards,
Weibo.

from tf-lift.

etrulls avatar etrulls commented on July 29, 2024

from tf-lift.

qiuweibo avatar qiuweibo commented on July 29, 2024

We don't, because it's not a binary descriptor.

On Thu, Jun 13, 2019 at 6:02 AM qiuweibo @.***> wrote: Dear Professor, In matching stage, did you use Brute Force Hamming distance matching for LIFT? If you use Hamming distance, so LIFT is a binary descriptor? Thanks! Best regards, Weibo. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#22?email_source=notifications&email_token=ABIK5W4MUUGKLSRS42TKCX3P2JANJA5CNFSM4FKB4VD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXTTUKY#issuecomment-501692971>, or mute the thread https://github.com/notifications/unsubscribe-auth/ABIK5W2QMHWYA4RLV7OCMGTP2JANJANCNFSM4FKB4VDQ .

Then what matching method do you suggest me to use since I am doing evaluation on LIFT?

For example, in opencv library, cv2.BFMatcher(), they have cv2.NORM_L2 that is appropriate for SIFT, SURF;
cv2.NORM_HAMMING that is good for ORB that used Hamming distance as measurement.

from tf-lift.

hudsonmartins avatar hudsonmartins commented on July 29, 2024

@qiuweibo, I think you can use cv2.NORM_L2, because LIFT is similar to SIFT.

from tf-lift.

punisher220 avatar punisher220 commented on July 29, 2024

@Shu-HowTing
Excuse me, what is your h5py library version? My h5py version is 2.10.0 and I tried for feature match but a warning occurred:
H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.
I wonder whether my h5py version is suitable for the feature match code. So I want to make sure about the h5py version.
Thanks for reply.

from tf-lift.

kmyi avatar kmyi commented on July 29, 2024

That's just a warning saying that you should change the code to match the new way of reading data.

from tf-lift.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.