Comments (6)
Hi,
- Set self.root=True means using RootSIFT instead of SIFT. RootSIFT is l2-normalized by definition and SIFT extracted by opencv is also normalized by default (with a norm of 256 as I recall). So if you perform direct NN matching, the normalization should not be an issue. As a safe guard, you can l2-normalize these features manually.
- For Aachen day-night I use the original img resolution.
- Sorry but I think MNN doesn't have a thresholding. You can apply Lowe's ratio test(https://stackoverflow.com/questions/51197091/how-does-the-lowes-ratio-test-work) together with MNN to achieve better results. For RootSIFT, I think 0.8 is a good threshold for ratio test.
- 512 means setting seed_top_k as [512,512], which is proportionally increased with keypoint number(8k). 0.2 means setting p_th as 0.2.
- Yes, I think that's a correct setting for SG+RootSIFT.
- One last thing, for Aachen, using upright RoofSIFT/SIFT generally improves the performance. I used the following code to produce upright features.
def run(self, img_path): sift = cv2.xfeatures2d.SIFT_create(nfeatures=self.num_kpt, contrastThreshold=self.contrastThreshold)<br/> img = cv2.imread(img_path,cv2.IMREAD_GRAYSCALE) cv_kp= sift.detect(img, None) score=np.asarray([_kp.response for _kp in cv_kp]) if self.upright: cv_kp=[cv2.KeyPoint(x=kp.pt[0],y=kp.pt[1],_size=kp.size,_angle=0) for kp in cv_kp]<br/> cv_kp,desc=sift.compute(img,cv_kp) desc=desc/np.linalg.norm(desc,axis=-1)[:,np.newaxis] if self.root: desc=np.sqrt(abs(desc/(np.linalg.norm(desc,axis=-1,ord=1)[:,np.newaxis]+1e-4))) kp = np.array([[_kp.pt[0], _kp.pt[1]] for _kp in cv_kp]) # N*2 index=np.flip(np.argsort(score)) kp,desc,score=kp[index],desc[index],score[index] return kp, desc,score
You may use codes in this reply.
from sgmnet.
Hi,
-
Set self.root=True means using RootSIFT instead of SIFT. RootSIFT is l2-normalized by definition and SIFT extracted by opencv is also normalized by default (with a norm of 256 as I recall). So if you perform direct NN matching, the normalization should not be an issue. As a safe guard, you can l2-normalize these features manually.
-
For Aachen day-night I use the original img resolution.
-
Sorry but I think MNN doesn't have a thresholding. You can apply Lowe's ratio test(https://stackoverflow.com/questions/51197091/how-does-the-lowes-ratio-test-work) together with MNN to achieve better results. For RootSIFT, I think 0.8 is a good threshold for ratio test.
-
512 means setting seed_top_k as [512,512], which is proportionally increased with keypoint number(8k). 0.2 means setting p_th as 0.2.
-
Yes, I think that's a correct setting for SG+RootSIFT.
-
One last thing, for Aachen, using upright RoofSIFT/SIFT generally improves the performance. I used the following code to produce upright features.
def run(self, img_path):
sift = cv2.xfeatures2d.SIFT_create(nfeatures=self.num_kpt, contrastThreshold=self.contrastThreshold)<br/>
img = cv2.imread(img_path,cv2.IMREAD_GRAYSCALE)
cv_kp= sift.detect(img, None)
score=np.asarray([_kp.response for _kp in cv_kp])
if self.upright:
cv_kp=[cv2.KeyPoint(x=kp.pt[0],y=kp.pt[1],_size=kp.size,_angle=0) for kp in cv_kp]<br/>
cv_kp,desc=sift.compute(img,cv_kp)
desc=desc/np.linalg.norm(desc,axis=-1)[:,np.newaxis]
if self.root:
desc=np.sqrt(abs(desc/(np.linalg.norm(desc,axis=-1,ord=1)[:,np.newaxis]+1e-4)))
kp = np.array([[_kp.pt[0], _kp.pt[1]] for _kp in cv_kp]) # N*2
index=np.flip(np.argsort(score))
kp,desc,score=kp[index],desc[index],score[index]
return kp, desc,score
from sgmnet.
Hi! Thanks so much. I have tried...
Yes, for RootSIFT, using the ratio test in combination with MNN (with 0.8 thresholding value) provides better performance.
I actually get a little lower performance: 41.8 / 56.1 / 61.2 .... If you have any further suggestions for better performance, please kindly let me know...
Also, I have another question... I am using your work with some dataset ....
What should I do if the minimum number of keypoints are lower than 256?
Sometimes when I used Superpoint, the number of keypoints is quite low...
It seems that I get this error related to the seeding.
This happens when the value of mask_survive.sum(dim=1)[0]+2
is equal to the number of keypoints.
Should I reduce the confidence value? or the value of seed ?
For example, if the number of keypoints = 233, which is less than 256.
And if it happens that mask_survive.sum(dim=1)[0]+2=233
then, I will get the error below
def seeding(nn_index1,nn_index2,x1,x2,topk,match_score,confbar,nms_radius,use_mc=True,test=False):
....
#confidence bar
match_score[match_score<confbar]=-1
mask_survive=match_score>0
if test:
topk=min(mask_survive.sum(dim=1)[0]+2,topk)
_,topindex = torch.topk(match_score,topk,dim=-1)#b*k
seed_index1,seed_index2=topindex,nn_index1.gather(index=topindex,dim=-1)
return seed_index1,seed_index2
The error looks like the following
> /mnt/HDD4TB1/BnchMrk3DRecon/SGMNet/sgmnet/match_model.py(51)seeding()
-> seed_index1,seed_index2=topindex,nn_index1.gather(index=topindex,dim=-1)
(Pdb) c
Traceback (most recent call last):
File "local_feature_eval_scripts/matching_pipeline.py", line 163, in <module>
matcher.matching_images()
File "/mnt/HDD4TB1/local-feature-evaluation/scripts/custom_matching.py", line 324, in matching_images
matches = self.SGM_wrapper_cuda.match(pred)
File "/mnt/HDD4TB1/BnchMrk3DRecon/Matcher_OG_wrap.py", line 70, in match
corr1,corr2, predict_matches = self.SGMmatcher.run(test_data)
File "/mnt/HDD4TB1/BnchMrk3DRecon/SGMNet/SGMmatcher.py", line 46, in run
res=self.model(feed_data,test_mode=True)
File "/home/gabby-suwichaya/anaconda3/envs/SGAT/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/HDD4TB1/BnchMrk3DRecon/SGMNet/sgmnet/match_model.py", line 216, in forward
File "/mnt/HDD4TB1/BnchMrk3DRecon/SGMNet/sgmnet/match_model.py", line 51, in seeding
seed_index1,seed_index2=topindex,nn_index1.gather(index=topindex,dim=-1)
RuntimeError: invalid argument 5: k not in range for dimension at /pytorch/aten/src/THC/generic/THCTensorTopK.cu:24
from sgmnet.
Hi,
For aachen, using upright features should further boost performance.
We assume the input kpt number should be larger than seed number. If you are using it with very low kpt number(eg ~200), we suggest to sample fewer seeds, while the performance is not guaranteed.
from sgmnet.
Hi! Thank you so much...
I am very sorry for this question.. I am very new to this field... Could you help explain how to extract the upright
feature?
We assume the input kpt number should be larger than seed number. If you are using it with very low kpt number(eg ~200), we suggest to sample fewer seeds, while the performance is not guaranteed.
I see thank you very much for confirmation.. I will try accordingly.
from sgmnet.
Thank you very much ! I have missed it totally..
from sgmnet.
Related Issues (13)
- A request for a setting of SGMNet+SP HOT 6
- Why use 'step-start_step' in train_step func
- Descriptor HOT 1
- Can not find hdf5 file HOT 1
- Questions about downloading SGMNet datasets
- Hi! Could you provide more details on the dataset for training ? HOT 3
- Hi! Could you provide more details on the dataset for training ? HOT 4
- if we change the two image of order, we got the different size of matching-poing, ps : config, extractor:num_kpt=300 HOT 3
- SGMNet+SP error HOT 3
- currenttly, batch-size =1, i don not the sgmnet that can set batch-size = 2,( batch-size=2, it means that use 4 img) matching
- How many epochs did you use for training?
- CPU support
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sgmnet.