xmba15 / torch_cpp

scripted and traced deep learning models and their applications

📝 torch_cpp

---

🎛 Dependencies

---

• CUDA Toolkits: if you use GPU.

• pytorch c++ API: the easiest way is to reuse build binary provided from pytorch official website. Here is the sample script to use the build binary. For convenience, this repo assumes torch c++ api is installed into /opt/libtorch

• other dependencies:

sudo apt-get install -y --no-install-recommends \
    libopencv-dev

🔨 How to Build

---

# build library
make default

# build examples
make apps -j`nproc`

# build gpu examples
make gpu_apps -j`nproc`

🏃 How to Run

SuperPoint and SuperGlue

---

Usage

• Script pre-trained superpoint and superglue weights

git submodule update --init --recursive
export ROOT_DIR="$(pwd)"
cd $ROOT_DIR/scripts/superglue/SuperGluePretrainedNetwork/
git apply ../jit_patch.patch
python3 -m pip install -r $ROOT_DIR/scripts/superglue/SuperGluePretrainedNetwork/requirements.txt
cd $ROOT_DIR
python3 $ROOT_DIR/scripts/superglue/jit_superglue_model.py
python3 $ROOT_DIR/scripts/superglue/jit_superpoint_model.py

• Test inference apps

./build/examples/match_images_superglue/match_images_superglue_app path/to/superpoint_model.pt path/to/superglue_model.pt ./data/images/VisionCS_0a.png ./data/images/VisionCS_0b.png

(back to top)

💎 References

---

• SuperGlue trained by magicleap
• SuperPoint SLAM
• Torch Tracing vs Torch Scripting

(back to top)

torch_cpp's Issues

Difference with implementation of SuperPoint SLAM

Hello,
Thank you so much for this implementation. I am new to Deep Learning and PyTorch in general, so my question may be a bit naive.
I notice there are differences in SuperPoint implementation between yours and SuperPoint SLAM, such as yours doesn't have structure SuperPoint and define its conv layers like here. Is it have any pros and cons between this two implementation?
I Hope can hear from you!

how to build the source code?

Hi, I am not so familiar with the building of c++, if possible, would you please show some building details?
Thank you very much.

Cannot create model

Hi! Thank you for this code. I am trying to convert the model, and i see this error:

Loaded SuperGlue model ("outdoor" weights)
Traceback (most recent call last):
File "D:\tracking_2022\deepfeatures\torch_cpp-master\scripts\superglue\jit_superglue_model.py", line 17, in
main()
File "D:\tracking_2022\deepfeatures\torch_cpp-master\scripts\superglue\jit_superglue_model.py", line 11, in main
scripted_module = torch.jit.script(superglue)
File "C:\Users\B\AppData\Roaming\Python\Python39\site-packages\torch\jit_script.py", line 1257, in script
return torch.jit._recursive.create_script_module(
File "C:\Users\B\AppData\Roaming\Python\Python39\site-packages\torch\jit_recursive.py", line 451, in create_script_module
return create_script_module_impl(nn_module, concrete_type, stubs_fn)
File "C:\Users\B\AppData\Roaming\Python\Python39\site-packages\torch\jit_recursive.py", line 513, in create_script_module_impl
script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
File "C:\Users\B\AppData\Roaming\Python\Python39\site-packages\torch\jit_script.py", line 587, in _construct
init_fn(script_module)
File "C:\Users\B\AppData\Roaming\Python\Python39\site-packages\torch\jit_recursive.py", line 491, in init_fn
scripted = create_script_module_impl(orig_value, sub_concrete_type, stubs_fn)
File "C:\Users\B\AppData\Roaming\Python\Python39\site-packages\torch\jit_recursive.py", line 517, in create_script_module_impl
create_methods_and_properties_from_stubs(concrete_type, method_stubs, property_stubs)
File "C:\Users\B\AppData\Roaming\Python\Python39\site-packages\torch\jit_recursive.py", line 368, in create_methods_and_properties_from_stubs
concrete_type._create_methods_and_properties(property_defs, property_rcbs, method_defs, method_rcbs, method_defaults)
RuntimeError:
Can not iterate over a module list or tuple with a value that does not have a statically determinable length
:
File "D:\tracking_2022\deepfeatures\torch_cpp-master\scripts\superglue\SuperGluePretrainedNetwork\models\superglue.py", line 133
def forward(self, desc0: torch.Tensor, desc1: torch.Tensor) -> Tuple[torch.Tensor,torch.Tensor]:
for layer, name in zip(self.layers, self.names):
~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
if name == 'cross':
src0, src1 = desc1, desc0

Please ask how much memory is used？

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 3.81 GiB total capacity; 1.45 GiB already allocated; 51.69 MiB free; 1.50 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF