jonasschult / mask3d Goto Github PK

Hi, thank you for sharing your work! Will you be able to add a license to the repo in the near future?

Thanks for benchmarking on STPLS3D, and congrats on achieving state-of-the-art performance! We are looking forward to your scripts for STPLS3D.

I found there is a 3-dims labels per point, first is the class-id, the second is instance-id, but I don't know the 3rd dim means.
I guess it may be able to be named as "segment-id". because it usually generates items like "point2segment","segment2label".
So, what does the 3rd dim in point's labels mean, or what does the segment represent?

Hi and thanks for your code,
Could you clarify the use of random cuboid? I see that it is called once on points here and once again later on coordinates and labels, the latter of which inputs 3 arguments while your implementation of random cuboid only takes in 1

When trying to run the preprocessing script for ScanNet it fails with ValueError: files doesn't have same coordinates,, from scannet_preprocessing:156. Could you please advise if this is expected or what may be going on here?

Hello Author！
When I can't get git using the URL you provided,so I used the latest version of detectron2,it need hydra-core>=1.1 and omegaconf>=2.1.
I upgraded two dependent versions--------hydra-core 1.2.0 and omegaconf 2.2.3
I encountered the following problem：

Traceback (most recent call last):
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
return target(*args, **kwargs)
File "/home/ds/data/mask3d/Mask3D-main/models/mask3d.py", line 57, in init
self.backbone = hydra.utils.instantiate(config.backbone)
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 249, in instantiate
raise InstantiationException(
hydra.errors.InstantiationException: Cannot instantiate config of type Res16UNet34C.
Top level config must be an OmegaConf DictConfig/ListConfig object,
a plain dict/list, or a Structured Config class or instance.

Traceback (most recent call last):
File "/home/ds/data/mask3d/Mask3D-main/main_instance_segmentation.py", line 104, in
main()
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/main.py", line 90, in decorated_main
_run_hydra(
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 389, in _run_hydra
_run_app(
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 452, in _run_app
run_and_report(
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 216, in run_and_report
raise ex
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 213, in run_and_report
return func()
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 453, in
lambda: hydra.run(
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run
_ = ret.return_value
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value
raise self._return_value
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
File "/home/ds/data/mask3d/Mask3D-main/main_instance_segmentation.py", line 100, in main
test(cfg)
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/main.py", line 79, in decorated_main
return task_function(cfg_passthrough)
File "/home/ds/data/mask3d/Mask3D-main/main_instance_segmentation.py", line 85, in test
cfg, model, loggers = get_parameters(cfg)
File "/home/ds/data/mask3d/Mask3D-main/main_instance_segmentation.py", line 51, in get_parameters
model = InstanceSegmentation(cfg)
File "/home/ds/data/mask3d/Mask3D-main/trainer/trainer.py", line 58, in init
self.model = hydra.utils.instantiate(config.model)
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 222, in instantiate
return instantiate_node(
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 339, in instantiate_node
return _call_target(target, partial, args, kwargs, full_key)
File "/home/ds/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target
raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error in call to target 'models.mask3d.Mask3D':
InstantiationException('Cannot instantiate config of type Res16UNet34C.\nTop level config must be an OmegaConf DictConfig/ListConfig object,\na plain dict/list, or a Structured Config class or instance.')
full_key: model

I want to know which the version of detectron2 hydra-core and omegaconf should be installed or
'config.backbone' and 'config.model' can be ''hydra.utils.instantiate'' called ?

Thank you！！！

When I run the processing code for scannet, the following results is obtained.


Then, the code is finished, and the data is not processed.

Besides, I check the code, but I cannot find any processing functions are used in "init".

Could you give me any advice on how to processing the data.

Thanks.

First of all, thank you for releasing your code to the public.

I'm currently trying to train the network from scratch on the S3DIS dataset and I noticed a mistake in the training script

My second question relates to out of memory errors. In your paper you state that you've used a GPU with 48 GB of memory. I currently only have access to a GPU with 24 GB of memory but thought I could counteract this by using a batch size of 1. However, after 7 epochs the training crashes with OOM. I noticed that each epoch increases the used memory slightly from initial 16 GB to the full 24 GB. I did not modify your code apart from the changes to the s3dis_from_scratch.sh. Do you think this is due to memory fragmentation from the sparse backbone or could this be a memory leak?

I would really appreciate it if you could help me solve this issue.

Hi, I just use the pretrained model on S3DIS (area1) on my processed dataset using "Stanford3dDataset_v1.2_Aligned_Version" by running following script.

The results are as follows, which is lower that the reported results a lot.

When I use multi-gpu by change the $gpus in config, I meet the error TypeError: cannot pickle 'MinkowskiConvolutionFunction' object

First of all, congratulations for your great work. I currently use softgroup to perform instance detection and want to replace it with your work. I only need to make the inference with a 3d scene obtained by Bundlefusion. I am a windows user and have installed all the python packages with no problem. The only problem I have found is the use of MinkowskiEngine since it is not compatible with Windows. My question is if you know if I can make the Mask3D method compatible with Windows, taking into account that I am only going to perform the inference of the model and that the MinkowskiEngine only works on Linux. Thanks for your attention

Hallo. Congratulations again on your excellent work.

If my own dataset is a training scenario of 5 to 40 million points, does Mask3D support it?
I can convert my data structure to the same type of data structure as the S3DIS data, is there anything I should be aware of in the case of such mega scenes?

Hi.
I want to know whether this model can be trained with more than one GPU. I changed the 'general.gpus' in config_base_instance_segmentation.yaml, but it reported errors.
Thanks.

Hi, this is a very exciting paper. Could you release the processed test segments on the test split of Scannet dataset?

Thanks.
Jiaqing.

Hello, thans for your work! when I learn the s3dis config to train model on my custom data, I'm confused why "data.num_labels(13)" is less than "general.num_targets(14)", and other datasets equal.

can you please help me with this confusion?
thanks in advance

Hi! I'm trying to run stpls3d_val.sh - inference on STPLS3D with the pretrained checkpoint.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
detectron2 0.6 requires hydra-core>=1.1, but you have hydra-core 1.0.5 which is incompatible.
detectron2 0.6 requires iopath<0.1.10,>=0.1.7, but you have iopath 0.1.10 which is incompatible.
detectron2 0.6 requires omegaconf>=2.1, but you have omegaconf 2.0.6 which is incompatible.

I have installed detectron2 as in the instruction:
pip install 'git+https://github.com/facebookresearch/detectron2.git@710e7795d0eeadf9def0e7ef957eea13532e34cf' --no-deps
In the requirement.txt the following packages versions are specified:
hydra-core==1.0.5, iopath==0.1.9, omegaconf==2.0.6

Maybe some other version of detectron2, than given in the installation guide, needs to be installed to be compatible with the rest of the packages?

Hello!

When I'm evaluating on ScanNet200, the outputs don't include floor and wall classes. How can this problem be solved?

Congrats on the release and thank you for open sourcing your code!
I would love to know if you have any plans to open-source the inference code used on the demo you provide. Currently, it seems that only the train/test code is available, and would love to use a python interface to use this method with some different point clouds.

I should preprocess scannet200 in this scannet200-original Repostitory(LanguageGroundedSemseg,https://github.com/RozDavid/LanguageGroundedSemseg) or the Repostitory（scannet :https://github.com/ScanNet/ScanNet/blob/master/BenchmarkScripts/ScanNet200/preprocess_scannet200.py）?

Hey guys, I find your project page is 404:"https://jonasschult.github.io/Mask3D/". I also can't find the paper and youtube video, is there any thing wrong? I think this is a good work, when will it be released? Thank you.

I'm trying to replicate your, frankly, amazing results in Scannet200 instance segmentation. However, the description and checkpoints for the backbone are missing, making it really hard to replicate your results. I believe this is the repository of the backbone, but, it would be nice if you could cite the exact backbone weights file and the appropriate configs somewhere within the repo.

I really hope I haven't missed you mentioning the backbone somewhere - if so, please point it out here, and I'll close this.

Thanks, and again, great work!

Hello! Thank you for your excellent work, it is an excellent method in the field of instance segmentation.

I have been working on a demo code for the interference of a single ply file using the scrips provided. The idea behind it is to create a scannet dataset where we only have one pointcloud in the test split. This approach has some limitations that require preparations and modifications:

• The pointcloud must only have the colors for the “segmentator” program to work.

• For “scannet_preprocessing” and “main_instance_segmentation” to work on our files, we need to name them like scannet files: scene0001_00.ply for example. The “_vh_clean_2” suffix is added in the provided demo code.

• We need to make sure that the path where we move the output of segmentator (line 9 of the provided code) is the same that the used in the preprocessing ( scannet_preprocessing.py line 178).

• We need a certain structure in the demo folder in order to simulate a dataset. The folder needs the files “./Tasks/Benchmark/scannetv2_test.txt”, “color_mean_std.yaml”, “label_database.yaml”, “train_database.yaml”, “train_validation_database.yaml” and “validation_database.yaml”, (all generated with a full preprocess of the scannet dataset). And the folder train with at least the .npy files for scene 0270_00, 0270_02 and 0384_00.

• The folder should look like this:

├── train
│   ├── 0270_00.npy
│   ├── 0270_02.npy
│   └── 0384_00.npy
├── Tasks
│   └── Benchmark
│       └── scannetv2_test.txt
├── color_mean_std.yaml
├── label_database.yaml
├── train_database.yaml
├── train_validation_database.yaml
├── validation_database.yaml
└── scene0001_00.ply

The code takes care of the modification of the txt file of the test split, and the generation and movement of the output of “segmentator”. Then uses the provided python scripts for the preprocessing of the test split of a scannet dataset. Lastly, it uses the provided python script main_instance_segmentation.py for the generation of the results. Obtaining the predictions of our pointcloud.

# demo.py

import subprocess
import argparse
import os
import shutil


def segment(d_dir, scene):
    # Execute "segmentator" binary for the current ply file
    args = ("/home/admin2/PycharmProjects/Mask3d/Mask3D/datasets/preprocessing/segmentator",  # Path to the compiled segmentator binary
            str(d_dir + "/" + scene + ".ply"))
    popen = subprocess.Popen(args)
    popen.wait()

    # Rename the file and move it to the segments_test folder
    os.rename(d_dir + "/" + scene + ".0.010000.segs.json",
              d_dir + "/" + scene + "_vh_clean_2.0.010000.segs.json")
    shutil.move(d_dir + "/" + scene + "_vh_clean_2.0.010000.segs.json",
                "/home/admin2/Datasets/Mask3d_Processed/segments/" + scene + "_vh_clean_2.0.010000.segs.json")


def preprocess(d_dir):
    # Execute "scannet_preprocessing.py" for the ply file
    subprocess.run(["python", "/home/admin2/PycharmProjects/Mask3d/Mask3D/datasets/preprocessing/scannet_preprocessing.py",  # Path to scannet_preprocessing python file
                    "preprocess",
                    str("--data_dir="+d_dir),
                    str("--save_dir="+d_dir),
                    str("--git_repo="+d_dir),
                    '--modes=("test",)'])  # Only preprocess test split


def instance_segmentation(d_dir):
    # Execute "main_instance_segmentation.py"
    subprocess.run(["python", "/home/admin2/PycharmProjects/Mask3d/Mask3D/main_instance_segmentation.py",  # Path to main_instance_segmentation python file
                    'general.experiment_name="singleplytest_query_150_topk_300_dbscan_0.95"',
                    'general.project_name="scannet_eval"',
                    "general.checkpoint='checkpoints/scannet_benchmark.ckpt'",  # Path to checkpoint
                    "general.eval_on_segments=true",
                    "general.train_on_segments=true",
                    "general.train_mode=false",
                    "general.export=true",
                    "data.test_mode=test",  # Use test split
                    "data/datasets=demo",  # Copy of scannet.yaml with test_dataset.data_dir changed
                    "model.num_queries=150",
                    "general.topk_per_image=300",
                    "general.use_dbscan=true",
                    "general.dbscan_eps=0.95"])

def main():
    # Create the parser
    parser = argparse.ArgumentParser()
    # Add arguments
    parser.add_argument('--data_dir', type=str, required=True)
    parser.add_argument('--scene', type=str, required=True)

    # Parse the arguments
    args = parser.parse_args()
    data_dir = args.data_dir
    scene = args.scene

    # Update txt of the test split with the scene name
    f = open(data_dir + "/Tasks/Benchmark/scannetv2_test.txt", "w")
    f.write(scene + "\n")
    f.close()

    # Check if the folders exist, create them if not
    if not os.path.exists(data_dir + "/scans_test"):
        os.mkdir(data_dir + "/scans_test")
    if not os.path.exists(data_dir + "/scans_test/" + scene):
        os.mkdir(data_dir + "/scans_test/" + scene)

    # Rename file and move it to ./scans_test/<scene>/
    shutil.copy(data_dir + "/" + scene + ".ply", data_dir + "/scans_test/" + scene + "/" + scene + "_vh_clean_2.ply")

    # Call segment function. Runs segmentator on the file and moves the output to the segments_test folder
    segment(data_dir, scene)

    # Call preprocess function. Runs "scannet_preprocessing.py" only for test split
    preprocess(data_dir)

    # Call instance_segmentation. Runs "main_instance_segmentation.py" for test split
    instance_segmentation(data_dir)

if __name__ == "__main__":
    main()

However, the predictions obtained using the configurations for the scannet_benchmark and the scannet_benchmark checkpoint (ScanNetv2 test) are different from the predictions obtained via the web demo.

@JonasSchult In the paper it is stated that you filter out the predictions with a score lower than 0.8. I was wondering if there is more post processing filters or modifications to the predictions (such as aggrupation of overlapping clusters) and if the configurations and checkpoints used are the same as the scannet benchmark. In addition, I noticed some degree of down sampling of the ply files uploaded to the web, and was wondering how much is it down sampled or if there is any down sampling at all for the inference.

Here are some examples of the discrepancy between the predictions obtained with the code and the web demo:

Predictions demo.py	Predictions web demo

As you can see, there is some scenes where the clusters are notably different and even some scenes where some of the instances are detected only in one of the predictions.

@Charlescai123 @reynoldscem @claragarciamoll @Augustab @lintong-zhang @pablovela5620 I have noticed that you have expressed your interest in processing your own pointclouds. If you are interested, we could work on this collectively.

Thank you!

Hallo Jonas,

Zunächst Herzlichen Glückwunsch zu Ihrer hervorragenden Arbeit!

Ihre WebSeit-Publikation kann nicht geöffnet werden. Wie kann ich das lesen?

Wenn Sie mir sagen könnten, würde ich sehr freuen, was Ihre speziellen Trainingsgeräte sind und wie lange Sie für das Training brauchen? Wie viel RAM haben Sie? Welche Grafikkarten wurden verwendet und wie viele Karten wurden insgesamt eingesetzt?

Und werden Sie einen Leitfaden für die Training von S3DIS (from scratch) auf herausgeben? Wann wird sie verfügbar sein?

Beste Grüße

——————————————————————————————————
Hello Jonas,

First of all congratulations for your excellent work!

Your webpage publication can not be opened. How can I read it?

If you could tell me, I would be very happy to know what your special training equipment is and how long it takes you to train? How much RAM do you have? What graphics cards were used and how many cards were used in total?

And will you release a guide for training S3DIS (from scratch) on? When will it be available?

Best regards

Hi, Thanks for the amazing work!

I had a question about training within segments for Scannet. In the paper, it is mentioned that it is used to reduce memory consumption at dot product. I wanted to ask what is point2segment and how is it used while training.

Thanks!

How to use fp16 in model training and testing

Currently the inference procedure is wrapped up inside the evaluation stuff. It would be nice if the code used to generate the visualisations were released, even if in a messy form.

Thanks for releasing this repo!
I am try to write a interfacing code based on this repo and I found I couldn't load the backbone checkpoint using the provided checkpoint? Does it need a independent checkpoint file for backbone?

Hi Jonas. Thx for your great work and effort. I am trying to use your code to do the inference for my own indoor point cloud, which is .ply format. Now I have met some problems.

1. When installing the dependencies according to the README, some errors will pop up. Could you please create a new virtual environment, install all dependencies according to the README and run this scannet_benchmark.sh script? This workflow will help to find out what the errors are and what stuff still missing.
2. What should I do if I want to do the inference for my own point cloud? Could you give me some advice about how to use your code for the inference?

Looking forward to your reply. Have a nice day!

Hello,
I would like to ask the means of "label==253", because I found some examples' ground truth has the label which is named as 253, and it is out of the number of classes in dataset.

Hallo,

Now that it's September, when will your RELEASE version of the code be released! Very much looking forward to your code!

Best Regards

Hello @JonasSchult,

Thanks for this Mask3D model. I trained the Mask3D model with a custom PCD dataset and followed the S3DIS dataset format. When I run inference, the .txt file is generated at eval_output/instance_evaluation_singleplytest_query_150_topk_300_dbscan_0.95_0/decoder_-1/. The output of that file is all follow:

Total 7 labels were used for training. Each scene contains two objects (an object of class 6 is common in scene). The length 3D point cloud data in 'Area_1_scene2_envelope.ply' is 306858 points. Of that, 226902 belongs to class 6, and 79956 points belong to class 1 in gt. I run inference for 40 scenes, and it shows the prediction for class 6 correct but either wrong or with confidence less than 0.40 for other classes. And the generated mask files have all values 1 as follows:

.
.
.
.

The expected value should be 1 only for points belonging to that class, and the rest values as 0. Non-zero integers indicate part of the predicted instance.

Is this due to the long length of point clouds of one label compared to the other? Or am I missing something in the code?

Hey @Pamogar as you are also working on the custom dataset. Any input from your side would be helpful.

Thanks,
Vishal

Hi, JonasSchult!
I really appreciate your excellent work！When I ran the training code (scannet200), I met some problems:

Restoring states from the checkpoint path at saved/scannet200_benchmark/last-epoch.ckpt
Traceback (most recent call last):
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 347, in
lambda: hydra.run(
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 107, in run
return run_job(
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/core/utils.py", line 128, in run_job
ret.return_value = task_function(task_cfg)
File "/home/ang/Mask3D-main/main_instance_segmentation.py", line 97, in main
train(cfg)
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/hydra/main.py", line 27, in decorated_main
return task_function(cfg_passthrough)
File "/home/ang/Mask3D-main/main_instance_segmentation.py", line 78, in train
runner.fit(model)
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 696, in fit
self._call_and_handle_interrupt(
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 737, in _fit_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1112, in _run
self._restore_modules_and_callbacks(ckpt_path)
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1067, in _restore_modules_and_callbacks
self._checkpoint_connector.restore_model()
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py", line 182, in restore_model
self.trainer.strategy.load_model_state_dict(self._loaded_checkpoint)
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 343, in load_model_state_dict
self.lightning_module.load_state_dict(checkpoint["state_dict"])
File "/home/ang/anaconda3/envs/mask3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for InstanceSegmentation:
Missing key(s) in state_dict: "criterion.empty_weight".

I've tried many ways but I cannot still solve it. Could you tell me How to train from scratch?
Thanks in advance!

First of all, congrats on your work.
I am working on instance detection and I want to know if you have any plans to upload the inference code? Currently, it seems that only the train/test code is available, and would like to use a python interface to use the method with my own point clouds.

Hi, I've been training scannet200 model using the default setting in the repo. However after 200 epochs, the reported APs on the validation set are already a lot higher than the checkpoints linked in this repo (https://github.com/JonasSchult/Mask3D/blob/main/docs/detailed_scores/scannet200_val.txt).

The AP/AP50/AP25 I got are 0.428 0.578 0.654.

Checkpoint can be downloaded here

Hi Jonas! This is a fantastic job.
When I train on the scannet dataset, I met a problem about the dataset prepare module. In semseg.py line 194, when the code runs to "self._labels = self._select_correct_labels(labels, num_labels)", there exists an exception. So I'd like to ask whether the Scannet data need some other processing.

The detail error information is below:
"/Desktop/program/Mask3D/trainer/trainer.py", line 872, in prepare_data
self.train_dataset = hydra.utils.instantiate(self.config.data.train_dataset)

raise HydraException(f"Error calling '{cls}' : {e}") from e
hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from:
200, 200

I would also like to know what kind of runtime performance we get with this

Is the model already trained? Do I need to train after I download it or can I just use it without training it at first?

When running the command cd third_party/pointnet2 && python setup.py install to set up the environment, it shows RuntimeError: Unexpected error from cudaGetDeviceCount().. Below are the details. Do you have any solutions for that problem? Thanks!

/usr/local/lib/python3.10/site-packages/torch/cuda/init.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
return torch._C._cuda_getDeviceCount() > 0
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running install
/usr/local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
/usr/local/lib/python3.10/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running bdist_egg
running egg_info
creating pointnet2.egg-info
writing pointnet2.egg-info/PKG-INFO
writing dependency_links to pointnet2.egg-info/dependency_links.txt
writing top-level names to pointnet2.egg-info/top_level.txt
writing manifest file 'pointnet2.egg-info/SOURCES.txt'
reading manifest file 'pointnet2.egg-info/SOURCES.txt'
writing manifest file 'pointnet2.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'pointnet2._ext' extension
creating /root/Mask3D/third_party/pointnet2/build
creating /root/Mask3D/third_party/pointnet2/build/temp.linux-x86_64-cpython-310
creating /root/Mask3D/third_party/pointnet2/build/temp.linux-x86_64-cpython-310/_ext_src
creating /root/Mask3D/third_party/pointnet2/build/temp.linux-x86_64-cpython-310/_ext_src/src
Traceback (most recent call last):
File "/root/Mask3D/third_party/pointnet2/setup.py", line 19, in
setup(
File "/usr/local/lib/python3.10/site-packages/setuptools/init.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 177, in setup
return run_commands(dist)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
dist.run_commands()
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.10/site-packages/setuptools/command/install.py", line 74, in run
self.do_egg_install()
File "/usr/local/lib/python3.10/site-packages/setuptools/command/install.py", line 123, in do_egg_install
self.run_command('bdist_egg')
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 165, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/usr/local/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
self.run_command(cmdname)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.10/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
self.run_command('build_ext')
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
build_ext.build_extensions(self)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 459, in build_extensions
self._build_extensions_serial()
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 485, in _build_extensions_serial
self.build_extension(ext)
File "/usr/local/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
_build_ext.build_extension(self, ext)
File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 540, in build_extension
objects = self.compiler.compile(
File "/usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 649, in unix_wrap_ninja_compile
cuda_post_cflags = unix_cuda_flags(cuda_post_cflags)
File "/usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 548, in unix_cuda_flags
cflags + _get_cuda_arch_flags(cflags))
File "/usr/local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1767, in _get_cuda_arch_flags
capability = torch.cuda.get_device_capability(i)
File "/usr/local/lib/python3.10/site-packages/torch/cuda/init.py", line 357, in get_device_capability
prop = get_device_properties(device)
File "/usr/local/lib/python3.10/site-packages/torch/cuda/init.py", line 371, in get_device_properties
_lazy_init() # will define _get_device_properties
File "/usr/local/lib/python3.10/site-packages/torch/cuda/init.py", line 229, in _lazy_init
torch._C._cuda_init()
RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW

python main_instance_segmentation.py
general.project_name="s3dis_eval"
general.experiment_name="area1_pretrained_eps_0.6_topk_-1_q_100"
general.checkpoint="/home/jalam/mythili_wd/Mask3D/checkpoints/area1_scannet_pretrained.ckpt"
general.train_mode=false
data.batch_size=4
data/datasets=s3dis
general.num_targets=14
data.num_labels=13
general.area=1
model.num_queries=100
general.topk_per_image=-1
general.use_dbscan=true
general.save_visualizations=true
general.dbscan_eps=0.6
After running the above command my terminal looks like this, and one folder is created saved which have only logs , how can i save the instance from the network , can you please help me here

Hi, I have finished the train and test in scannet200. However, when I attempt to submit the output in "eval_output/" to the official website of scannet, it recalled back that the format is error.

Hi,

I was able to setup your environment successfully on Colab but while trying to run train/inference on the STPLS3D dataset, I get this error of segmentation fault (core dumped). Do you have any insights on why this might be happening?

Thanks for this amazing work! When will the code be released?

Congrats on your great work and thank you for releasing it in open source.
Is there any news about the release of the interference code used in the demo? It would be very convenient to use it with our own pointclouds.

Thank you!

Hello.

I'm really interested to know how to train the model using my own pointcloud dataset, for my thesis. Can you provide some instructions about how to do that?

Thanks in advance!!

ModuleNotFoundError: No module named 'datasets.preprocessing'
How to fix this error?

Hi Jonas！
I 'd like to ask how to prepare the STPLS3D datset. When I ran the prepare stpls3d script, it turns out
“No such file or directory in datasets/STPLS3D/validation". But when I download the origin Synthetic_v3_InstanceSegmentation dataset, there are only .txt file in the directory. So I'd like to consult this problem, thank you.

During the data collation process here, the code subtracts label_offset from the remapped semantic id (i.e. the semantic id obtained after _remap_from_zero in here) and clamps the resulting semantic id to be at least zero. During the evaluation process here, the code adds label_offset back to the semantic id.

However, this poses an issue for Scannet200 since its ignored remapped semantic ids are [0,2] with label_offset=2 (see here). In Scannet200, 0 in the remapped semantic id corresponds to the wall, 1 the chair, and 2 the floor. When performing evaluation, the 1 of the chair will become 0 after subtracting label_offset and then clamping to be at least zero. It will then become 2 during evaluation after adding label_offset back to 0, which causes the floor class to correspond to the chair class.

Hello and Thanks for your great work.

I checked the performance by training mask 3D on ScanNet, and it is lower than the performance of the weight provided.
I used the configuration with the weight provided( scripts/scannet/scannet_val.sh).
Did I miss anything?

Hi! I'm trying to run your preprocessing script for STPLS3D data and I've got an error:
stpls3d_preprocessing.py: command not found
I assume there's some problem with using Fire module.

In

datasets/preprocessing/stpls3d_preprocessing.py preprocess \
--data_dir="PATH_TO_STPLS3D" \
--save_dir="../../data/processed/stpls3d"

are we calling for 'preprocess' function? Because it is present only in base_preprocessing.py and matterport_preprocessing.py but there's no such function in other datasets scripts.