ucuapps / openglue Goto Github PK
View Code? Open in Web Editor NEWOpen Source Graph Neural Net Based Pipeline for Image Matching
License: MIT License
Open Source Graph Neural Net Based Pipeline for Image Matching
License: MIT License
Hi, nice work there. But I could not find any training time info in the repo or in the paper. Eg: OpenGlue w/ magic leap superpoint on X GPU for 1 epoch .etc.
Will you kindly share more information so one can know everything is working as expected ?
Hi,
Thanks for the great implementation. I was wondering when the rest of the checkpoints were going to be released.
We also release the code of our CVPR work 'IMP: Iterative Matching and Pose Estimation with Adaptive Pooling' with training and testing code available at https://github.com/feixue94/imp-release.
HI,
I am trying to get openGlue running. I start training like this:
python train.py --config='config/config.yaml' --features_config='config/features_online/sift.yaml'
However during training superglue with sift I encounter an error.
Epoch 0: 3%|โ | 313/10851 [05:52<3:18:00, 1.13s/it, loss=7.73, v_num=0s2i, Train NLL loss=8.400, Train Metric loss=0.000]../aten/src/ATen/native/cuda/IndexKernel.cu:91: operator(): block: [1,0,0], thread: [41,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:91: operator(): block: [1,0,0], thread: [62,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
Running the debugger I get the error:
CUDA error: device-side assert triggered
File "/media/ssd4TB/software/OpenGlue/models/matching_module.py", line 81, in training_step
lafs1, responses1, desc1 = self.local_features_extractor(batch['image1'])
File "/media/ssd4TB/software/OpenGlue/models/features/base.py", line 79, in forward
lafs, scores = self.run_nms(lafs, scores, image.size())
File "/media/ssd4TB/software/OpenGlue/models/features/base.py", line 49, in run_nms
mask[0, 0, kpts_[:, 1], kpts_[:, 0]] = scores_
RuntimeError: CUDA error: device-side assert triggered
During handling of the above exception, another exception occurred:
File "/media/ssd4TB/software/OpenGlue/train.py", line 86, in main
trainer.fit(model, datamodule=dm, ckpt_path=config.get('checkpoint'))
File "/media/ssd4TB/software/OpenGlue/train.py", line 90, in <module>
main()
I use pytroch 1.11.0 with cuda 11.3 and pytorch lightning 1.6
GPU is a 3090 and operating system is Ubunutu.
# packages in environment at /home/wga2hi/anaconda3/envs/openGlue_2:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main defaults
_openmp_mutex 5.1 1_gnu defaults
absl-py 1.4.0 pypi_0 pypi
aiohttp 3.8.4 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
albumentations 1.3.0 pypi_0 pypi
antlr4-python3-runtime 4.9.3 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
async-timeout 4.0.2 pypi_0 pypi
asynctest 0.13.0 pypi_0 pypi
attrs 23.1.0 pypi_0 pypi
bosch-ca 1.0 1 defaults
ca-certificates 2022.10.11 boschca_h06a4308_0 [bosch-ca] defaults
cachetools 5.3.0 pypi_0 pypi
certifi 2022.12.7 boschca_py37h06a4308_0 [bosch-ca] defaults
charset-normalizer 3.1.0 pypi_0 pypi
click 8.1.3 pypi_0 pypi
cycler 0.11.0 pypi_0 pypi
deepdish 0.3.7 pypi_0 pypi
docker-pycreds 0.4.0 pypi_0 pypi
fonttools 4.38.0 pypi_0 pypi
frozenlist 1.3.3 pypi_0 pypi
fsspec 2023.1.0 pypi_0 pypi
gitdb 4.0.10 pypi_0 pypi
gitpython 3.1.31 pypi_0 pypi
google-auth 2.18.1 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
grpcio 1.54.2 pypi_0 pypi
idna 3.4 pypi_0 pypi
imageio 2.29.0 pypi_0 pypi
importlib-metadata 6.6.0 pypi_0 pypi
joblib 1.2.0 pypi_0 pypi
kiwisolver 1.4.4 pypi_0 pypi
kornia 0.6.12 pypi_0 pypi
kornia-moons 0.2.6 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1 defaults
libffi 3.4.4 h6a678d5_0 defaults
libgcc-ng 11.2.0 h1234567_1 defaults
libgomp 11.2.0 h1234567_1 defaults
libstdcxx-ng 11.2.0 h1234567_1 defaults
markdown 3.4.3 pypi_0 pypi
markupsafe 2.1.2 pypi_0 pypi
matplotlib 3.5.3 pypi_0 pypi
multidict 6.0.4 pypi_0 pypi
ncurses 6.4 h6a678d5_0 defaults
networkx 2.6.3 pypi_0 pypi
numexpr 2.8.4 pypi_0 pypi
numpy 1.21.6 pypi_0 pypi
oauthlib 3.2.2 pypi_0 pypi
omegaconf 2.3.0 pypi_0 pypi
opencv-python 4.7.0.72 pypi_0 pypi
opencv-python-headless 4.7.0.72 pypi_0 pypi
openssl 1.1.1t h7f8727e_0 defaults
packaging 23.1 pypi_0 pypi
pathtools 0.1.2 pypi_0 pypi
pillow 9.5.0 pypi_0 pypi
pip 22.3.1 py37h06a4308_0 defaults
protobuf 3.20.3 pypi_0 pypi
psutil 5.9.5 pypi_0 pypi
pyasn1 0.5.0 pypi_0 pypi
pyasn1-modules 0.3.0 pypi_0 pypi
pydeprecate 0.3.2 pypi_0 pypi
pyparsing 3.0.9 pypi_0 pypi
python 3.7.16 h7a1cb2a_0 defaults
python-dateutil 2.8.2 pypi_0 pypi
pytorch-lightning 1.6.0 pypi_0 pypi
pywavelets 1.3.0 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
qudida 0.0.4 pypi_0 pypi
readline 8.2 h5eee18b_0 defaults
requests 2.31.0 pypi_0 pypi
requests-oauthlib 1.3.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
scikit-image 0.19.3 pypi_0 pypi
scikit-learn 1.0.2 pypi_0 pypi
scipy 1.7.3 pypi_0 pypi
sentry-sdk 1.24.0 pypi_0 pypi
setproctitle 1.3.2 pypi_0 pypi
setuptools 65.6.3 py37h06a4308_0 defaults
shutup 0.2.0 pypi_0 pypi
six 1.16.0 pypi_0 pypi
smmap 5.0.0 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0 defaults
tables 3.7.0 pypi_0 pypi
tensorboard 2.11.2 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
threadpoolctl 3.1.0 pypi_0 pypi
tifffile 2021.11.2 pypi_0 pypi
tk 8.6.12 h1ccaba5_0 defaults
torch 1.11.0+cu113 pypi_0 pypi
torchmetrics 0.11.4 pypi_0 pypi
tqdm 4.65.0 pypi_0 pypi
typing-extensions 4.6.1 pypi_0 pypi
urllib3 1.26.16 pypi_0 pypi
wandb 0.15.3 pypi_0 pypi
werkzeug 2.2.3 pypi_0 pypi
wheel 0.38.4 py37h06a4308_0 defaults
xz 5.4.2 h5eee18b_0 defaults
yarl 1.9.2 pypi_0 pypi
zipp 3.15.0 pypi_0 pypi
zlib 1.2.13 h5eee18b_0 defaults
I also tried with different lightning and torch versions, however the issue still occurs, but at other iterations during training.
Are you aware of the issue? Are there fixes for it.
Could you please help? Thanks!
i can't find the .pth '/home/ostap/projects/superglue-lightning/logs/pretrain_superpoint_magicleap/Pretraining_SuperPointNet__attn_softmax__laf_none__2022-03-07-20-34-43/superglue_sp_magicleap.pth'
Hello,
Thanks for the work and great paper.
While training, I noticed that the number of keypoints for img1 and img2 is constant. The loss function is also built on this assumption.
Can you please let me know what may happen if I use a different number of keypoints for img1 and img2?
Would it have an adverse effect on the training?
For example: If I use 500 keypoints for Img1 and 1000 keypoints for Img2.
Thank you
What are some things to consider when training on my own dataset ?
Hi,
I would like to use models.features.sift.SIFT
instead of OPENCV_SIFT
as a feature extractor but the superglue matcher doesn't work with this configuration. Even if the model is trained using the features from OPENCV_SIFT
, shoudn't the two versions of sift compute the same features given the same parameters ? Here are the configurations i used :
features:
name: 'SIFT'
descriptor_dim: 128
parameters:
max_keypoints: 2048
nms_diameter: 9
rootsift: True
descriptor_dim: 128
With SIFT
no matches are found.
the superpoint weight you provided is not match with the name in both superpoint_coco.yaml and superpoint_kitti.yaml. And use the weight you provided for superpoint will raise the keyerror as follow:
File "d:\Develop\PythonProject\OpenGlue-main\models\features\superpoint\model.py", line 175, in _load_weights
state_dict = torch.load(str(weights), map_location='cpu')['model_state_dict']
KeyError: 'model_state_dict'
I can not train superpoint right now so could you please provide the weight for superpoint-coco or superpoint-kitti?
The first line directory in the screenshot is the default checkpoint path and the second line is the checkpoint path direct to the weight that you give. and it will cause the Keyerror.
Hello. First of all, congrats for the nice work, and thank you very much for making the source code and pretrained models available to all. Apparently, the /examples folder which is referenced in the README is missing. Could you please check it?
I tried to train a matcher for d2net using cached its features, but the AUC is always 0.
Could you please help? Thanks!
As the title, I can not find the .ipynb file.
Would you please upload it?
Best regards
Hello, I have encountered some unknown errors in the process of reproduction. I can't solve them personally. I want to ask you about the problem. The command I entered is as follows.
python inference. py --image0_ path '/home/ch/OpenGlue-main1/OpenGlue-main/freiburg/1341847980.722988.png' --image1_ path '/home/ch/OpenGlue-main1/OpenGlue-main/freiburg/1341847981.726650.png' --experiment_ path '/home/ch/OpenGlue-main1/OpenGlue-main/freiburg' --checkpoint_ name '/home/ch/OpenGlue-main1/OpenGlue-main/freiburg/openglue-SIFT-rotation. ckpt' --output_ dir 'results.png'
The error message displayed is as follows:
File "inference.py", line 99
def __ init__ (self, local_feature: nn.Module, matcher: nn.Module, match_config: Dict = {}) -> None:
^
SyntaxError: invalid syntax
Hello,
First of all, thank you for sharing your work!
Unfortunately, when trying to use inference.py, the function draw_LAF_matches() doesn't seem to be defined anywhere.
Could you provide it please?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.