Train on custom dataset about dvis

[zhang-tao-whu]

I didn't quite understand what you meant. Are you referring to how to pretrain a segmenter on an image dataset?

[Steviaey]

Thank you for your reply.
Exactly and also how to train the DVIS Online like in the GETTING_STARTED.md. I prepared and registered the image dataset. I then adjusted the "DVIS/configs/ovis /Base-OVIS-VideoInstanceSegmentation.yaml" from TRAIN: ("ovis_train",) and TEST: ("ovis_val",) to my registered dataset. But the training isn't working. Should I load my dataset somewhere else to train it on a pretrained Neuronal Network?

[zhang-tao-whu]

Yes, you need to first run MinVIS.yaml on your dataset to train the segmenter, and then load the trained segmenter weights to use DVIS_online.yaml for training the referring tracker. Can you provide a detailed description of your issue? Are you unable to run the training code, or are you encountering errors in the predictions after training is completed?

[Steviaey]

I'm unable to run the training code. It says "the dataset is not registered", but I registered my COCO dataset in "C:\Users\svi-cp\DVIS\dvis\data_video\datasets\own_dataset.py" and put the registration name "my_dataset" from "register_coco_instances("my_dataset", ...." in the config-file "C:/Users/svi-cp/DVIS/configs/ovis/MinVIS_R50.yaml" and also the "DVIS/configs/ovis/Base-OVIS-VideoInstanceSegmentation.yaml"

Traceback (most recent call last):
File "c:\users\svi-cp\detectron2\detectron2\data\catalog.py", line 51, in get
f = self[name]
File "C:\Users\svi-cp.conda\envs\dvis\lib\collections_init_.py", line 1010, in getitem
raise KeyError(key)
KeyError: 'my_dataset'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "train_net_video.py", line 321, in
launch(
File "c:\users\svi-cp\detectron2\detectron2\engine\launch.py", line 84, in launch
main_func(*args)
File "train_net_video.py", line 312, in main
trainer = Trainer(cfg)
File "c:\users\svi-cp\detectron2\detectron2\engine\defaults.py", line 378, in init
data_loader = self.build_train_loader(cfg)
File "train_net_video.py", line 108, in build_train_loader
return build_detection_train_loader(cfg, mapper=mapper, dataset_name=cfg.DATASETS.TRAIN[0])
File "c:\users\svi-cp\detectron2\detectron2\config\config.py", line 207, in wrapped
explicit_args = _get_args_from_config(from_config, *args, **kwargs)
File "c:\users\svi-cp\detectron2\detectron2\config\config.py", line 245, in _get_args_from_config
ret = from_config_func(*args, **kwargs)
File "C:\Users\svi-cp\DVIS\dvis\data_video\build.py", line 119, in _train_loader_from_config
dataset = get_detection_dataset_dicts(
File "C:\Users\svi-cp\DVIS\dvis\data_video\build.py", line 92, in get_detection_dataset_dicts
dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names]
File "C:\Users\svi-cp\DVIS\dvis\data_video\build.py", line 92, in
dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names]
File "c:\users\svi-cp\detectron2\detectron2\data\catalog.py", line 53, in get
raise KeyError(
KeyError: "Dataset 'my_dataset' is not registered!

[zhang-tao-whu]

You need to add from . import own_dataset at 112 line of the builtin.py.

[Steviaey]

Thank you so much for your help!
I didn't think about that.

Problem now is, my COCO .json annotation file is missing the "video_lenght". Is it still possible to use the dataset or preset the length and width somewhere, or do I have to add it manualy? I couldn't find it in the ovis annotation json file.

criterion.empty_weight
sem_seg_head.predictor.class_embed.{bias, weight}
[10/25 17:14:03 d2.engine.train_loop]: Starting training from iteration 0
ERROR [10/25 17:14:05 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
File "c:\users\svi-cp\detectron2\detectron2\engine\train_loop.py", line 155, in train
self.run_step()
File "c:\users\svi-cp\detectron2\detectron2\engine\defaults.py", line 494, in run_step
self._trainer.run_step()
File "c:\users\svi-cp\detectron2\detectron2\engine\train_loop.py", line 488, in run_step
data = next(self._data_loader_iter)
File "c:\users\svi-cp\detectron2\detectron2\data\common.py", line 329, in iter
for d in self.dataset:
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data\dataloader.py", line 633, in next
data = self._next_data()
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data\dataloader.py", line 1345, in _next_data
return self._process_data(data)
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data\dataloader.py", line 1371, in _process_data
data.reraise()
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch_utils.py", line 644, in reraise
raise exception
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data_utils\worker.py", line 308, in _worker_loop
data = fetcher.fetch(index)
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data_utils\fetch.py", line 32, in fetch
data.append(next(self.dataset_iter))
File "c:\users\svi-cp\detectron2\detectron2\data\common.py", line 296, in iter
yield self.dataset[idx]
File "c:\users\svi-cp\detectron2\detectron2\data\common.py", line 125, in getitem
data = self._map_func(self._dataset[cur_idx])
File "C:\Users\svi-cp\DVIS\dvis\data_video\dataset_mapper.py", line 296, in call
video_length = dataset_dict["length"]
KeyError: 'length'

[10/25 17:14:05 d2.engine.hooks]: Total training time: 0:00:02 (0:00:00 on hooks)
[10/25 17:14:05 d2.utils.events]: iter: 0 lr: N/A max_mem: 171M
Traceback (most recent call last):
File "train_net_video.py", line 321, in
launch(
File "c:\users\svi-cp\detectron2\detectron2\engine\launch.py", line 84, in launch
main_func(*args)
File "train_net_video.py", line 314, in main
return trainer.train()
File "c:\users\svi-cp\detectron2\detectron2\engine\defaults.py", line 484, in train
super().train(self.start_iter, self.max_iter)
File "c:\users\svi-cp\detectron2\detectron2\engine\train_loop.py", line 155, in train
self.run_step()
File "c:\users\svi-cp\detectron2\detectron2\engine\defaults.py", line 494, in run_step
self._trainer.run_step()
File "c:\users\svi-cp\detectron2\detectron2\engine\train_loop.py", line 488, in run_step
data = next(self._data_loader_iter)
File "c:\users\svi-cp\detectron2\detectron2\data\common.py", line 329, in iter
for d in self.dataset:
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data\dataloader.py", line 633, in next
data = self._next_data()
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data\dataloader.py", line 1345, in _next_data
return self._process_data(data)
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data\dataloader.py", line 1371, in _process_data
data.reraise()
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch_utils.py", line 644, in reraise
raise exception
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data_utils\worker.py", line 308, in _worker_loop
data = fetcher.fetch(index)
File "C:\Users\svi-cp.conda\envs\dvis\lib\site-packages\torch\utils\data_utils\fetch.py", line 32, in fetch
data.append(next(self.dataset_iter))
File "c:\users\svi-cp\detectron2\detectron2\data\common.py", line 296, in iter
yield self.dataset[idx]
File "c:\users\svi-cp\detectron2\detectron2\data\common.py", line 125, in getitem
data = self._map_func(self._dataset[cur_idx])
File "C:\Users\svi-cp\DVIS\dvis\data_video\dataset_mapper.py", line 296, in call
video_length = dataset_dict["length"]
KeyError: 'length'

Thank you in advance, I really want to use your great work.

[Steviaey]

Hello Mr. Zhang,
is there no solution to this error?
I still couldn't fix the error.
Thank you in advance!

[zhang-tao-whu]

If you want to use the data_mapper of YTVIS/OVIS, you need to organize your data strictly according to the format of YTVIS.