DetectionDataset.from_yolo bad conversion with autodistill_grounded_sam DetectionDataset object about supervision

[Youho99]

Could modifying autodistill's label function theoretically fix the problem?

By ensuring that the base conversion is done in COCO and not in YOLO (COCO has more information, so it is simpler to convert COCO to another annotation than the reverse).

# By modifying this code for example, with as_coco instead of as_yolo
         dataset.as_yolo(
             output_folder + "/images",
             output_folder + "/annotations",
             min_image_area_percentage=0.01,
             data_yaml_path=output_folder + "/data.yaml",
         )

Could this work?

[SkalskiP]

Hi @Youho99 👋🏻 The reason might be the disconnected masks generated by GroundedSAM - that is, two separate masks represent one bounding box. The YOLO format does not allow for such a situation, so such objects are split and saved separately. As a result, you end up with more objects.

[Youho99]

@SkalskiP
This may indeed be a reason.
But in some cases, I also end up with fewer items.

[SkalskiP]

@Youho99 because sometimes you get detection that gets removed because it is too large or too small. There might be a bug. But I need to see a concrete, reproducible example of something working wrong.

[Youho99]

In fact, my goal is to directly convert the dataset into a format that CVAT supports.
CVAT does not support segmentation with the YOLO format, so in any case I will not use it as such.
I will convert to COCO format via Supervision, I will import the annotations to CVAT, and export them to YOLO format from CVAT.
Then I will compare them with the result of the label function of autodistill_grounded_sam.

I will get back to you if there is a difference already on that (probably on the autodistill_grounded_sam repo if there is a problem).

Indeed, there is certainly a bug with the from_yolo for an object generated with autodistill_grounded_sam, but I have no application case to verify the veracity of this (because I am not going to use it as a result).

[Youho99]

@SkalskiP
Edit:

I face the same problem with the .as_coco() function to put my dataset in coco format. Except that here, the function crashes because of the difference in detection (1100 versus 900) for the cat image given above.
For other images it doesn't crash, but potentially the problem is just underwater.

I get the error with any image

My first conclusion is that we cannot export a dataset generated via Grounded SAM (autodistill_grounded_sam) at the moment.

# use the code above
results.as_coco(
         #images_directory_path=os.path.join("temp_result_coco", 'images'),
         annotations_path=os.path.join("temp_result_coco", 'labels.json'))

2024-03-27 10:28:06.724 Uncaught app exception
Traceback (most recent call last):
File "/home/ggiret/miniconda3/envs/deeplabel/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 542, in _run_script
exec(code, module.dict)
File "/home/ggiret/Documents/deeplabel/pages/3_Labels_Generation.py", line 64, in
auto_labeling()
File "/home/ggiret/Documents/deeplabel/backend/labels_generation.py", line 27, in auto_labeling
st.session_state.dataset.as_coco(
File "/home/ggiret/miniconda3/envs/deeplabel/lib/python3.10/site-packages/supervision/dataset/core.py", line 457, in as_coco
save_coco_annotations(
File "/home/ggiret/miniconda3/envs/deeplabel/lib/python3.10/site-packages/supervision/dataset/formats/coco.py", line 218, in save_coco_annotations
coco_annotation, annotation_id = detections_to_coco_annotations(
File "/home/ggiret/miniconda3/envs/deeplabel/lib/python3.10/site-packages/supervision/dataset/formats/coco.py", line 114, in detections_to_coco_annotations
approximate_mask_with_polygons(
IndexError: list index out of range

[Youho99]

No longer working with the from_yolo function, I close the issue.
I will reopen this issue if I notice an actual bug with this function.