itailang / samplenet Goto Github PK

Hi!
Thank you very much for sharing the code . How should the registration results be visualized with point cloud and cad model (like Fig10 in your paper)?
Thank you again!

In this paper, I notice that you first sample points and then apply PoinetNet for classification.
Can we replace the FPS stage of PointNet++ with the sampling strategy in this work?

Thanks.
Jiaheng.

Hi Itailang, I'm trying to rework your paper, but I met some problems. To provide some context, now I've successfully downloaded the docker image and enter the container. However, when I tried to run the script sh compile_ops.sh I got an error

: not found.sh: 2: compile_ops.sh: 
compile_ops.sh: 3: cd: can't cd to ./grouping
: not found.sh: 4: compile_ops.sh: 
sh: 0: Can't open tf_grouping_compile.sh
: not found.sh: 7: compile_ops.sh: 
compile_ops.sh: 8: cd: can't cd to ../structural_losses/
: not found.sh: 9: compile_ops.sh: 
sh: 0: Can't open tf_nndistance_compile.sh
: not found.sh: 12: compile_ops.sh: 
sh: 0: Can't open tf_approxmatch_compile.sh

I use a test.sh to test if there is anything wrong

test.sh

cd ./grouping

My shell file can work normally. But I don't know why compile_ops.sh can't work.

So I decide to compile the ops separately. I go the the grouping directory, and tried to run tf_grouping_compile.sh, but failed again. I have ensured the cuda path is correct. Now the error is like

: not found_compile.sh: 2: tf_grouping_compile.sh: 
: not found_compile.sh: 5: tf_grouping_compile.sh: 
’; did you mean ‘-fPIC’?
: not found_compile.sh: 7: tf_grouping_compile.sh: 
g++: error: tf_grouping_g.cu.o: No such file or directory

To offer a better background, tf_grouping_compile.sh is

#!/usr/bin/env bash

TF_INC=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')
TF_LIB=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())')

/usr/local/cuda-10.0/bin/nvcc tf_grouping_g.cu -o tf_grouping_g.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

# TF1.13
g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-10.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-10.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0

I kindly request your assistance in troubleshooting the issue. I would appreciate any guidance or suggestions you can provide to help resolve the problem. Looking forward to your reply!

Best regards,
Zhirong

When I was trying to start the reconstructing task, it seems the auto downloading failed.

I downloaded the zip file from dropbox manually and it works. Is it due to the network issue or something is wrong with the code?

I tried to simply run the docker, following the instructions in readme, while an error occur, telling me that the file /workspace does not exist.
Then a series of errors occured, so that I have to manually clone the Pointnet2_Pytorch repo.
However, after I clone it, and placed it in the correct path I suppose, which should be ./registration/workspace/Pointnet2_Pytorch, another error occured.

Maybe some improvements of the install instructions in readme can be improved. That would be helpful.

the modelnet40 is large dataset, but i noticed that the batchsize is 32,why is it an epoch to run only five batches?

Hi great author,

I would work on your code, here I have problem with docker environment. Must I install "Nvidia-Docker" in my system? not just in the docker container? because I have no access to operate the system files, so I can't install Nvidia-Docker in system layer. could I just install it in docker container? before I pull your docker ,what should I do? pleaes give me some advices, thanks alot

"
Installation
We strongly recommend working with Docker Engine and Nvidia-Docker. At this moment, the container can only run on a CUDA (linux/amd64) enabled machine due to specific compiled ops from Pointnet2_PyTorch.

Pull and run the Docker container
docker pull asafmanor/pytorch:samplenetreg_torch1.4
docker run --runtime nvidia -v $(pwd):/workspace/ -it --name samplenetreg asafmanor/pytorch:samplenetreg_torch1.4
"

Hi!
I implemented your reconstruction network, and the results are very good, but I want to use a model point cloud of my own for reconstruction sampling, that is to say, use your trained network to achieve downsampling of a point cloud? what should i do？

hi, where can i find the knn_cuda file ? thanks

Could you please show me the tf_approxmatch_so.so and other so.so

Hello, your work has helped me a lot, but I encountered a problem in the process of using it. If I want to use the reflectivity information of the lidar, how can I preserve this feature during the SampleNet downsampling process?

Could you please show me the specific data about the classification when sampling with SampleNet，because I found SampleNet64 and SampleNet32 have the same Classification Accuracy.

The classification accuracy is eval accuracy during training or eval accuracy in the evaluate_samplenet.py?

I also found the eval accuracy during training is 3% higher than the eval accuracy in the evaluate_samplenet.py.

The paper says that after training the S-NET model or SampleNet model,it will use the trained model parameters to train the classification network, but the code does not,could you please give me some advices?

I'm looking forward to your reply！ @itailang @asafmanor

Thanks for your great contributions on point cloud downsampling. What does "Classification accuracy" mean in the paper? Is it corresponding to "eval accuracy" or "eval avg class acc"?

Thank you for your reply.

Question: Is there any difference of the network or loss between the registration task and the classification task.

Description: I want to change the classification task of SampleNet's code to PyTorch framework(like the example code snippet in README.md). PointNet classifier is working normally(get a 90 eval acc pretrained model). But I got a terrible training result in SampleNet. Like train acc (in last epoch=500) =23% and eval acc = 61%. I check my code and didn't found error in calculate acc part. I don't know Is there any difference of the network or loss between the registration task and the classification task , because I haven't used Tensorflow in the past.

Thank you for sharing great work. I wonder why this issue is occured

soft_projection.py

    def _get_distances(self, grouped_points, query_cloud):
        deltas = grouped_points - query_cloud.unsqueeze(-1).expand_as(grouped_points)
        dist = torch.sum(deltas ** 2, dim=_axis_to_dim(3), keepdim=True) / self.sigma()
        return dist

Traceback (most recent call last): File "soft_projection.py", line 262, in <module> projected_points = propagator.project(query_cloud_pl, point_cloud_pl) File "soft_projection.py", line 141, in project dist = self._get_distances(grouped_points, query_cloud) File "soft_projection.py", line 95, in _get_distances dist = torch.sum(deltas ** 2, dim=_axis_to_dim(3), keepdim=True) / self.sigma() TypeError: 'Tensor' object is not callable

help me! thank you

Hi!
Thanks for sharing the great work. I am wondering that if two SampleNet can be trained on the same point cloud simultaneously?
Say I have one object point cloud with two shapes with very different features, the first SampleNet should only sample the points from shape1 and the second SampleNet should sample the points from shape2. And the task can be trained with some contrastive loss. Does it make any sense? I have tried a toy example but both SampleNet just samples the same points. Any comments are very welcome!

class FCN_sampler(nn.Module):
    def __init__(self, shape1_num_out_points=512, shape2_num_out_points=512):
        super(FCN_sampler, self).__init__() 
        self.sampler1 = SampleNet(
                    num_out_points=fg_num_out_points,
                    bottleneck_size=128,
                    group_size=8,
                    initial_temperature=1.0,
                    input_shape="bnc",
                    output_shape="bnc")  

        self.sampler2 = SampleNet(
                    num_out_points=bg_num_out_points,
                    bottleneck_size=128,
                    group_size=8,
                    initial_temperature=1.0,
                    input_shape="bnc",
                    output_shape="bnc") 

    def forward(self, x, shape1=True):
        if shape1:
            simp_pc, proj_pc = self.sampler1 (x)
        else:
            simp_pc, proj_pc = self.sampler2 (x)    
        return simp_pc, proj_pc

## Sample points
sampler = FCN_sampler()
simp_pc1, coord1 = sampler(coord)
simp_pc2, coord2 = sampler(coord, shape1=False)

# Compute losses
simplification_loss = sampler.sampler1.get_simplification_loss(
        coord, simp_pc1, 512
)
projection_loss = sampler.sampler1.get_projection_loss()
loss1 = 0.01 * simplification_loss + 0.01* projection_loss

simplification_loss = sampler.sampler2.get_simplification_loss(
        coord, simp_pc2, 512
)
projection_loss = sampler.sampler2.get_projection_loss()

loss2 = 0.01 * simplification_loss + 0.01* projection_loss

samplenet_loss = loss1 + loss2 

Hi,
I have trained and evaluated Sample Net for registration, and the result seems good.

But i don't know how to process my point cloud file(.txt or .ply) with the network, nor do I know how to output it and save it. Could you tell me what should i do? How to modify codes？
This will be very helpful to me, thanks a lot!

When I run "git checkout 5ff4382f56a8cbed2b5edd3572f97436271aba89", the error is as below:
fatal: reference is not a tree: 5ff4382f56a8cbed2b5edd3572f97436271aba89

However, when I skip the "git checkout 5ff4382f56a8cbed2b5edd3572f97436271aba89", the error is as below:
CUDA kernel failed : no kernel image is available for execution on the device
void group_points_kernel_wrapper(int, int, int, int, int, const float, const int, float*) at L:38 in pointnet2/_ext-src/src/group_points_gpu.cu**

Look forward to your help! Thank you very much!

Hi Authors:

 I have one question. Can the proposed SampleNet be used on 3D Point Cloud Semantic Segmentation and obtain better performances on 3D Point Cloud Semantic Segmentation task? Thanks.

Xiaobing

Hi!
Thank you for this impressive work. I'm interested in using the SampleNet to sample the task-aware points for LiDAR pointcloud. As we know, LiDAR pointcloud is sparse, not uniform and complex. Do you think this is feasible, or do you have any suggestions for me?

Why is the test set accuracy of train_samplenet.py much higher than evaluate_samplenet.py in the classification task?

Hey,When I train the Autoencoder model,the error shows NameError: name 'farthest_point_sample' is not defined
@itailang @asafmanor

Thanks for your great contributions on point cloud downsampling. I'd like to ask some questions if you don't mind.

In this paper, when training the sample net, you use the R (sub-point cloud) after softly projection as the input of the task network.
However, in your first paper, you use the generated point cloud but not the "hard projection" point cloud after nn_matching as the input of the task network.
Why not unify the two algorithms?

Thank you for your reply.

How to use code for classification task. Means i have a existing model, optimizer etc but just want to change farthest point sampling to samplenet.

HI!
First of all your ideas are very good, and thank you very much for sharing the code . I have a question：how should the registration results be visualized? I hope you can help me! THx

What does 16 mean here(line 141)？Why 16？Shouldn't the num_points for each point cloud file feeded be 1024?
Thank you for your answer.