itailang / samplenet Goto Github PK
View Code? Open in Web Editor NEWDifferentiable Point Cloud Sampling (CVPR 2020 Oral)
Home Page: https://arxiv.org/abs/1912.03663
License: Other
Differentiable Point Cloud Sampling (CVPR 2020 Oral)
Home Page: https://arxiv.org/abs/1912.03663
License: Other
Thanks for your great contributions on point cloud downsampling. What does "Classification accuracy" mean in the paper? Is it corresponding to "eval accuracy" or "eval avg class acc"?
Thank you for your reply.
Hi!
Thank you for this impressive work. I'm interested in using the SampleNet to sample the task-aware points for LiDAR pointcloud. As we know, LiDAR pointcloud is sparse, not uniform and complex. Do you think this is feasible, or do you have any suggestions for me?
Could you please show me the specific data about the classification when sampling with SampleNet,because I found SampleNet64 and SampleNet32 have the same Classification Accuracy.
The classification accuracy is eval accuracy during training or eval accuracy in the evaluate_samplenet.py?
I also found the eval accuracy during training is 3% higher than the eval accuracy in the evaluate_samplenet.py.
The paper says that after training the S-NET model or SampleNet model,it will use the trained model parameters to train the classification network, but the code does not,could you please give me some advices?
I'm looking forward to your reply! @itailang @asafmanor
Hello, your work has helped me a lot, but I encountered a problem in the process of using it. If I want to use the reflectivity information of the lidar, how can I preserve this feature during the SampleNet downsampling process?
Thanks for your great contributions on point cloud downsampling. I'd like to ask some questions if you don't mind.
In this paper, when training the sample net, you use the R (sub-point cloud) after softly projection as the input of the task network.
However, in your first paper, you use the generated point cloud but not the "hard projection" point cloud after nn_matching as the input of the task network.
Why not unify the two algorithms?
Thank you for your reply.
Question: Is there any difference of the network or loss between the registration task and the classification task.
Description: I want to change the classification task of SampleNet's code to PyTorch framework(like the example code snippet in README.md). PointNet classifier is working normally(get a 90 eval acc pretrained model). But I got a terrible training result in SampleNet. Like train acc (in last epoch=500) =23% and eval acc = 61%. I check my code and didn't found error in calculate acc part. I don't know Is there any difference of the network or loss between the registration task and the classification task , because I haven't used Tensorflow in the past.
When I run "git checkout 5ff4382f56a8cbed2b5edd3572f97436271aba89", the error is as below:
fatal: reference is not a tree: 5ff4382f56a8cbed2b5edd3572f97436271aba89
However, when I skip the "git checkout 5ff4382f56a8cbed2b5edd3572f97436271aba89", the error is as below:
CUDA kernel failed : no kernel image is available for execution on the device
void group_points_kernel_wrapper(int, int, int, int, int, const float, const int, float*) at L:38 in pointnet2/_ext-src/src/group_points_gpu.cu**
Look forward to your help! Thank you very much!
the modelnet40 is large dataset, but i noticed that the batchsize is 32,why is it an epoch to run only five batches?
Hi!
Thanks for sharing the great work. I am wondering that if two SampleNet can be trained on the same point cloud simultaneously?
Say I have one object point cloud with two shapes with very different features, the first SampleNet should only sample the points from shape1 and the second SampleNet should sample the points from shape2. And the task can be trained with some contrastive loss. Does it make any sense? I have tried a toy example but both SampleNet just samples the same points. Any comments are very welcome!
class FCN_sampler(nn.Module):
def __init__(self, shape1_num_out_points=512, shape2_num_out_points=512):
super(FCN_sampler, self).__init__()
self.sampler1 = SampleNet(
num_out_points=fg_num_out_points,
bottleneck_size=128,
group_size=8,
initial_temperature=1.0,
input_shape="bnc",
output_shape="bnc")
self.sampler2 = SampleNet(
num_out_points=bg_num_out_points,
bottleneck_size=128,
group_size=8,
initial_temperature=1.0,
input_shape="bnc",
output_shape="bnc")
def forward(self, x, shape1=True):
if shape1:
simp_pc, proj_pc = self.sampler1 (x)
else:
simp_pc, proj_pc = self.sampler2 (x)
return simp_pc, proj_pc
## Sample points
sampler = FCN_sampler()
simp_pc1, coord1 = sampler(coord)
simp_pc2, coord2 = sampler(coord, shape1=False)
# Compute losses
simplification_loss = sampler.sampler1.get_simplification_loss(
coord, simp_pc1, 512
)
projection_loss = sampler.sampler1.get_projection_loss()
loss1 = 0.01 * simplification_loss + 0.01* projection_loss
simplification_loss = sampler.sampler2.get_simplification_loss(
coord, simp_pc2, 512
)
projection_loss = sampler.sampler2.get_projection_loss()
loss2 = 0.01 * simplification_loss + 0.01* projection_loss
samplenet_loss = loss1 + loss2
In this paper, I notice that you first sample points and then apply PoinetNet for classification.
Can we replace the FPS stage of PointNet++ with the sampling strategy in this work?
Thanks.
Jiaheng.
Hi Authors:
I have one question. Can the proposed SampleNet be used on 3D Point Cloud Semantic Segmentation and obtain better performances on 3D Point Cloud Semantic Segmentation task? Thanks.
Xiaobing
Hi great author,
I would work on your code, here I have problem with docker environment. Must I install "Nvidia-Docker" in my system? not just in the docker container? because I have no access to operate the system files, so I can't install Nvidia-Docker in system layer. could I just install it in docker container? before I pull your docker ,what should I do? pleaes give me some advices, thanks alot
"
Installation
We strongly recommend working with Docker Engine and Nvidia-Docker. At this moment, the container can only run on a CUDA (linux/amd64) enabled machine due to specific compiled ops from Pointnet2_PyTorch.
Pull and run the Docker container
docker pull asafmanor/pytorch:samplenetreg_torch1.4
docker run --runtime nvidia -v $(pwd):/workspace/ -it --name samplenetreg asafmanor/pytorch:samplenetreg_torch1.4
"
Hey,When I train the Autoencoder model,the error shows NameError: name 'farthest_point_sample' is not defined
@itailang @asafmanor
Hi!
I implemented your reconstruction network, and the results are very good, but I want to use a model point cloud of my own for reconstruction sampling, that is to say, use your trained network to achieve downsampling of a point cloud? what should i do?
I tried to simply run the docker, following the instructions in readme, while an error occur, telling me that the file /workspace does not exist.
Then a series of errors occured, so that I have to manually clone the Pointnet2_Pytorch repo.
However, after I clone it, and placed it in the correct path I suppose, which should be ./registration/workspace/Pointnet2_Pytorch, another error occured.
Maybe some improvements of the install instructions in readme can be improved. That would be helpful.
Hi,
I have trained and evaluated Sample Net for registration, and the result seems good.
But i don't know how to process my point cloud file(.txt or .ply) with the network, nor do I know how to output it and save it. Could you tell me what should i do? How to modify codes?
This will be very helpful to me, thanks a lot!
hi, where can i find the knn_cuda file ? thanks
Hi Itailang, I'm trying to rework your paper, but I met some problems. To provide some context, now I've successfully downloaded the docker image and enter the container. However, when I tried to run the script sh compile_ops.sh
I got an error
: not found.sh: 2: compile_ops.sh:
compile_ops.sh: 3: cd: can't cd to ./grouping
: not found.sh: 4: compile_ops.sh:
sh: 0: Can't open tf_grouping_compile.sh
: not found.sh: 7: compile_ops.sh:
compile_ops.sh: 8: cd: can't cd to ../structural_losses/
: not found.sh: 9: compile_ops.sh:
sh: 0: Can't open tf_nndistance_compile.sh
: not found.sh: 12: compile_ops.sh:
sh: 0: Can't open tf_approxmatch_compile.sh
I use a test.sh to test if there is anything wrong
test.sh
cd ./grouping
My shell file can work normally. But I don't know why compile_ops.sh can't work.
So I decide to compile the ops separately. I go the the grouping
directory, and tried to run tf_grouping_compile.sh
, but failed again. I have ensured the cuda path is correct. Now the error is like
: not found_compile.sh: 2: tf_grouping_compile.sh:
: not found_compile.sh: 5: tf_grouping_compile.sh:
’; did you mean ‘-fPIC’?
: not found_compile.sh: 7: tf_grouping_compile.sh:
g++: error: tf_grouping_g.cu.o: No such file or directory
To offer a better background, tf_grouping_compile.sh is
#!/usr/bin/env bash
TF_INC=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')
TF_LIB=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())')
/usr/local/cuda-10.0/bin/nvcc tf_grouping_g.cu -o tf_grouping_g.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC
# TF1.13
g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-10.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-10.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0
I kindly request your assistance in troubleshooting the issue. I would appreciate any guidance or suggestions you can provide to help resolve the problem. Looking forward to your reply!
Best regards,
Zhirong
Why is the test set accuracy of train_samplenet.py much higher than evaluate_samplenet.py in the classification task?
How to use code for classification task. Means i have a existing model, optimizer etc but just want to change farthest point sampling to samplenet.
Hi!
Thank you very much for sharing the code . How should the registration results be visualized with point cloud and cad model (like Fig10 in your paper)?
Thank you again!
HI!
First of all your ideas are very good, and thank you very much for sharing the code . I have a question:how should the registration results be visualized? I hope you can help me! THx
What does 16 mean here(line 141)?Why 16?Shouldn't the num_points for each point cloud file feeded be 1024?
Thank you for your answer.
Could you please show me the tf_approxmatch_so.so and other so.so
Thank you for sharing great work. I wonder why this issue is occured
soft_projection.py
def _get_distances(self, grouped_points, query_cloud):
deltas = grouped_points - query_cloud.unsqueeze(-1).expand_as(grouped_points)
dist = torch.sum(deltas ** 2, dim=_axis_to_dim(3), keepdim=True) / self.sigma()
return dist
Traceback (most recent call last): File "soft_projection.py", line 262, in <module> projected_points = propagator.project(query_cloud_pl, point_cloud_pl) File "soft_projection.py", line 141, in project dist = self._get_distances(grouped_points, query_cloud) File "soft_projection.py", line 95, in _get_distances dist = torch.sum(deltas ** 2, dim=_axis_to_dim(3), keepdim=True) / self.sigma() TypeError: 'Tensor' object is not callable
help me! thank you
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.