shiaoming / aliked

ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation

Hi there 👋

aliked's Issues

train step

This link(train code) only contains code and no explanation.Can you tell us how to run the train code?
such as the dataset.json in megadepth.py is for what？

Missing data request

Thank you for amazing work!

I want to reproduce paper and try on my custom dataset using ALIKE training script.

During dataset download some of the scenes seem to be corrupted. I tried to download Megadepth DISK dataset from website and it is down. 😢

I tried to contact in disk repository. They doesn't react.

Could you please provide these scenes? The missing files are ["0411", "0472", "0476", "0478", '0482']

Is it possible to use ALIKED to train matching models (SuperGlue etc)?

Hi.
Thank you for your great job.
The results of your paper have inspired me to change the keypoint detector used to train SuperGlue from SuperPoint to ALIKED.
I believe it will work in principle, what do you think?
I would appreciate your opinion.

https://github.com/magicleap/SuperGluePretrainedNetwork

Error in compiling custom_ops

I wonder what is the reason for this problem? Usually, this problem is caused by a failure to bind successfully. But since the code has been around for a while, that shouldn't be the reason.

I use

• Windows 11
• MSVC x64 19.38.33130
• python3.10
• torch 2.1

How to convert the model to torch script file ?

Thank you for your excellent works. @Shiaoming

I want to convert the model (aliked-n16rot.pth) to a torch script file so that the ALIKED can be used directly in C + +. But I ran into a lot of issues during the export process. Here's my code for a pt file; could you help me?

Thank you very much for your assistance. I eagerly await your response.

`import torch, cv2
import argparse
import logging
from nets.aliked import ALIKED
from torchvision.transforms import ToTensor

def model_with_run(model):
def fn(image):
results = model.run(image)
return results
return fn

parser = argparse.ArgumentParser(description='ALIKED image pair Demo.')
parser.add_argument('--model', choices=['aliked-t16', 'aliked-n16', 'aliked-n16rot', 'aliked-n32'],
default="aliked-n16rot",
help="The model configuration")
parser.add_argument('--device', type=str, default='cuda', help="Running device (default: cuda).")
parser.add_argument('--top_k', type=int, default=-1,
help='Detect top K keypoints. -1 for threshold based mode, >0 for top K mode. (default: -1)')
parser.add_argument('--scores_th', type=float, default=0.2,
help='Detector score threshold (default: 0.2).')
parser.add_argument('--n_limit', type=int, default=5000,
help='Maximum number of keypoints to be detected (default: 5000).')
args = parser.parse_args()

logging.basicConfig(level=logging.INFO)

model = ALIKED(model_name=args.model,
device=args.device,
top_k=args.top_k,
scores_th=args.scores_th,
n_limit=args.n_limit)
model = model_with_run(model)

img_rgb = cv2.imread('1311868169.163498.png')
img_rgb = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2RGB)
img_tensor = ToTensor()(img_rgb)
img_tensor = img_tensor.to('cuda').unsqueeze_(0)

traced_model = torch.jit.trace(model, img_tensor)

traced_model.save('ALIKED.pt')
`

Train-code&&Test-code

Hello author, reading your article has greatly benefited me. I am also studying feature matching. I have a question to ask you, how should the code for training and testing the ALIKED model be implemented? thanks

Paper information inconsistent with code information

Hello !

Thanks again for this great work!

I maybe noticed an inconsistency between the information in the paper and in the code. In the Alike paper and in the Aliked paper it is specified that the trel temperature is equal to 1.0 In the training code it is 0.1.

This difference could be explained by a modification of the training code afterwards but in practice the value of 0.1 seems more "plausible" than the value of 1.

Intuitively, it seems logical that the temperature should be low to allow the model to converge (a temperature of 1 seems too brutal).

This hypothesis is confirmed by experience: with trel=1 the loss does not converge. But with a value of 0.1 there is convergence.

Do you have any information to share on this subject?
Thank you in advance for these clarifications.

The problem about MegaDepth and train

@Shiaoming Hello! Thank you very much for your excellent work. I read your paper the most and want to replicate the code. Here I have two questions：
1.You mentioned that you did not provide a training model, and directly used ALIKE's training model, which will have an impact on the results of later experiments.
2.You use the MegaDepth dataset, I checked that it has about 800G, did you run all this dataset when you ran the experiment? The equipment in our lab cannot support such a large data set, is there any other way?

dataset request

Excellent work, I would like to conduct further research based on this work. However, the official website for the megadepth disk dataset has been shut down, making it impossible to download. Could anyone share the megadepth disk dataset? I am willing to pay for it.

Is the training code open source?

May I ask if the training code is open source, please?

Run ALIKED on CPU

Thank you for releasing ALIKED.

I performed a demo on CPU as follows:

$ python demo_pair.py assets/st_pauls_cathedral --device cpu

Then the following error happened.

    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

My PC (MacOS) does not support CUDA.
Is ALIKED not yet CPU-compatible?

关于几何不变性的解释？

文中说的几何不变性，在算法设计中包括了旋转、平移、缩放变换，但包括仿射、投影变换的不变性吗？

Train Code

I want to follow your outstanding work, so I wonder if the training code and guideline will be released in the future?

About reliable loss

Hello,

I read your excellent paper and have a question about reliable loss in Section V.D.

According to the definition (Eq.(12)), the reliability seems to be a vector.
In order to consider it as a scalar (like ALIKE), can I interpret it as $r(\mathbf{P_A},I_B) = \mathbf{P_B} \cdot \text{softmax}(\text{sim}(\mathbf{d_A}, \mathbf{D_B})/t_{rel})$ using the $\mathbf{P_B}$ mentioned in section V.C?

AttributeError: '_OpNamespace' object has no attribute 'get_patches_forward'

INFO:root:Loading 5 images
loading D:\DL_matching\ALIKED-main\models\aliked-n16rot.pth
C:\software\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:2895.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
INFO:root:Press 'space' to start.
Press 'q' or 'ESC' to stop!
Traceback (most recent call last):
File "C:\software\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch_ops.py", line 198, in getattr
op, overload_names = torch._C._jit_get_operation(qualified_op_name)
RuntimeError: No such operator custom_ops::get_patches_forward

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\DL_matching\ALIKED-main\demo_pair.py", line 116, in
pred_ref = model.run(img_rgb)
File "D:\DL_matching\ALIKED-main\nets\aliked.py", line 166, in run
pred = self.forward(img_tensor)
File "D:\DL_matching\ALIKED-main\nets\aliked.py", line 148, in forward
descriptors, offsets = self.desc_head(feature_map, keypoints)
File "C:\software\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\nn\modules\module.py", line 1130, in call_impl
return forward_call(*input, **kwargs)
File "D:\DL_matching\ALIKED-main\nets\blocks.py", line 227, in forward
patch = self.get_patches_func(xi, kptsi_wh.long(), self.kernel_size) # [N_kpts, C, K, K]
File "D:\DL_matching\ALIKED-main\custom_ops_init.py", line 18, in forward
patches = torch.ops.custom_ops.get_patches_forward(fmap, points, kernel_size)
File "C:\software\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch_ops.py", line 202, in getattr
raise AttributeError(f"'_OpNamespace' object has no attribute '{op_name}'") from e
AttributeError: '_OpNamespace' object has no attribute 'get_patches_forward'

When to release the paper of ALIKED?

Thanks.

CPP implementation of ALIKED?

thanks for your work, and is there CPP implementation of ALIKED(similar to ALIKE-cpp))?

Regarding processing speed

Hi.
I have compared the processing speed between SuperPoint and ALIKED and SuperPoint is faster.
I have the same input images and number of detection points.
I used your ALIKED-N(16) model.
What conditions are you measuring in your paper?