feixue94 / imp-release Goto Github PK
View Code? Open in Web Editor NEW[CVPR 2023] IMP: iterative matching and pose estimation with transformer-based recurrent module
Home Page: https://feixue94.github.io/
License: Other
[CVPR 2023] IMP: iterative matching and pose estimation with transformer-based recurrent module
Home Page: https://feixue94.github.io/
License: Other
Dear author, after reading your code, I feel that your idea of this paper is very novel. I would like to ask how the pre-training models you provided can achieve visual matching results of corresponding images in the code. As shown in Figure 9 in your paper, the corresponding matching line results are shown.
Hello, thank you very much for sharing your code. I am currently trying to reproduce your code on the YFCC dataset. However, when I use your dump/download_data.sh script, I receive a 503 error. Upon inspecting the internal script code, I noticed that you are downloading the dataset from https://research.altizure.com/data/oanet_data/raw_data.tar, but I still cannot download it. Is it possible that there is an issue with the website? Could you please provide more details?
Hello author, you mentioned this earlier in your reply, as shown in the link
#1 (comment)
Take eval_imp.py given in your code as an example, run the evaluation code of the corresponding scannet data set, how to achieve the operation of "Run the evaluation code and save the matched graph in each iteration"? After seeing the answer you gave, I tried, but found that it seemed unable to complete the operation of "save the matched graph". Because I don't seem to be familiar with this operation, and it seems difficult for me to add it in the code. So can I ask you about the specific code to do "run the evaluation code and save the matching graph at each iteration"? Thank you very much again!!
After looking at the code, it seems that I didn't see any estimation or supervision of pose during the training process. Additionally, the "Iterative" mentioned in the paper seems to correspond to the number of iterations for self-cross attention. I'm not sure if I misunderstood this.
Hi, thank you very much for your great work!
What should I do if I want to train the coco dataset, after all the Megadepth dataset is too big!
I think first use superpoint to export the coco dataset detections to generate a series of npz files, then execute the command python3 -m dump.dump_megadepth --feature_type spp --base_path path_of_megadepth --save_ path your_save_path, is this correct?
Looking forward to your reply!
It seems that I didn't find the any estimation or supervision of pose of the training phase in code.
@feixue94
Hi, author.
I found an error in the 'dump/dump_megadepth.py'
.
I ran the code which took about three days, and the code outputed the following error in the line 489.
FileNotFoundError: [Errno 2] No such file or directory: 'asserts/mega_nvalid_spp.npy'
I think 'asserts'
is a mistake of 'assets'
, right?
Also, what do the lines 486-489 of the code?
It seems that it just saves an empty dictionary.
Hello author, I am glad that you can reply to my previous questions. When I generated the corresponding key points and descriptors according to the readme you provided, I reported the following errors during training under Windows,
which could not be solved. Could you please tell me how to solve this problem? Thank you very much
@feixue94
Hi, thank you for your great work!
I want to reproduce the evaluation results on YFCC dataset to understand your model.
I ran the code to evaluate it on YFCC dataset as the instruction of README and got the results below.
Why are the results different from the results in your paper?
Are they related to some settings?
Evaluation Results (mean over 4000 pairs):
AUC@5 | AUC@10 | AUC@20 | AUC@50 | Prec | MScore | Mkpts | Ikpts |
---|---|---|---|---|---|---|---|
29.95 | 48.95 | 66.77 | 82.80 | 86.06 | 13.82 |
It 1 with 0.00
It 2 with 0.00
It 3 with 0.00
It 4 with 0.00
It 5 with 0.00
It 6 with 0.00
It 7 with 0.00
It 8 with 0.00
It 9 with 0.00
It 10 with 0.00
It 11 with 0.00
It 12 with 0.00
It 13 with 0.00
It 14 with 0.00
It 15 with 1.00
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████
Results of model IMP on yfcc dataset (iterative: False, sinkhorn: True, uncertainty: False)
hello, thanks for your job. I want to train the model on my dataset, what should I do?
Hi, author.
I have a question about "Undistorted_SfM" data which you provide via Google Drive.
Is this a data D2-net originally provided, which is employed for LoFTR training?
LoFTR
https://github.com/zju3dv/LoFTR/blob/master/docs/TRAINING.md
Hello, thank you very much for your great work!
I want to use feature matching network in C++, using TensorRT to accelerate the network, can you provide ideas for modification? Thank you for your recovery!
Dear author, thanks a lot for your released codes. I am wondering that how to train with my own dataset in the codes. Would you mind giving me some instances in your codes? Looking forward to your early reply.
你好!
首先感谢您的工作,这篇文章很有启发性和创新型,能利用到pose信息去优化matching。
但是我在阅读代码和paper后发现pose信息的利用可能并不多?
下面我说一下我对paper和代码的理解。首先是用到了pose的部分。第一点,在训练中,您把pose作为一项loss加入,相比原本的superglue提了点。第二点,推理上,您在adaptive sampling部分中用到了pose。第三点,您在推理过程中会用pose作为终止条件提前退出节省时间。第四点,您在推理过程中会将与极线距离过远的点作为outlier直接筛除(虽然这点最后好像没加入代码)。
不知我以上的理解是否正确,还请指正。
另外我还关心一点就是,descriptor augmentation的部分在推理过程是否也用到了pose信息。因为您论文中提到,如figure 2 (b)图中,f_p和f_M会作为一个网络层加入优化。所以我想知道pose信息在推理过程中是否有用到,还是只作为训练过程中的一个额外loss?
提前感谢您的回答!
祝好!
Hello author, you mentioned this earlier in your reply, as shown in the link
#1 (comment)
I want to visualize the matching results, but I cannot save the matching graph in each iteration. How should I perform this operation?
What exactly is a matching graph?
Thanks!
I follow the data preparation step to download the data you provide, and then try to extract local features as well as build correspondences for training using the command:
python3 -m dump.dump_megadepth --feature_type spp --base_path path_of_megadepth --save_path your_save_path
however, there is an error shows that No such file or directory in the directory 'Megadepth/phoenix/...'. As your provided the data structure of Megadepth folder, the folder 'phoenix' is empty.
So, could you help me find out what mistake I made?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.