roboflow / supervision

We write your reusable computer vision tools. 💜

notebooks | inference | autodistill | maestro

👋 hello

We write your reusable computer vision tools. Whether you need to load your dataset from your hard drive, draw detections on an image or video, or count how many detections are in a zone. You can count on us! 🤝

💻 install

Pip install the supervision package in a Python>=3.8 environment.

pip install supervision

Read more about conda, mamba, and installing from source in our guide.

🔥 quickstart

models

Supervision was designed to be model agnostic. Just plug in any classification, detection, or segmentation model. For your convenience, we have created connectors for the most popular libraries like Ultralytics, Transformers, or MMDetection.

import cv2
import supervision as sv
from ultralytics import YOLO

image = cv2.imread(...)
model = YOLO('yolov8s.pt')
result = model(image)[0]
detections = sv.Detections.from_ultralytics(result)

len(detections)
# 5

👉 more model connectors

• inference

  Running with Inference requires a Roboflow API KEY.

  import cv2
  import supervision as sv
  from inference.models.utils import get_roboflow_model
  
  image = cv2.imread(...)
  model = get_roboflow_model(model_id="yolov8s-640", api_key=<ROBOFLOW API KEY>)
  result = model.infer(image)[0]
  detections = sv.Detections.from_inference(result)
  
  len(detections)
  # 5

annotators

Supervision offers a wide range of highly customizable annotators, allowing you to compose the perfect visualization for your use case.

import cv2
import supervision as sv

image = cv2.imread(...)
detections = sv.Detections(...)

bounding_box_annotator = sv.BoundingBoxAnnotator()
annotated_frame = bounding_box_annotator.annotate(
    scene=image.copy(),
    detections=detections
)

supervision-0.16.0-annotators.mp4

datasets

Supervision provides a set of utils that allow you to load, split, merge, and save datasets in one of the supported formats.

import supervision as sv

dataset = sv.DetectionDataset.from_yolo(
    images_directory_path=...,
    annotations_directory_path=...,
    data_yaml_path=...
)

dataset.classes
['dog', 'person']

len(dataset)
# 1000

👉 more dataset utils

• load

  dataset = sv.DetectionDataset.from_yolo(
      images_directory_path=...,
      annotations_directory_path=...,
      data_yaml_path=...
  )
  
  dataset = sv.DetectionDataset.from_pascal_voc(
      images_directory_path=...,
      annotations_directory_path=...
  )
  
  dataset = sv.DetectionDataset.from_coco(
      images_directory_path=...,
      annotations_path=...
  )

• split

  train_dataset, test_dataset = dataset.split(split_ratio=0.7)
  test_dataset, valid_dataset = test_dataset.split(split_ratio=0.5)
  
  len(train_dataset), len(test_dataset), len(valid_dataset)
  # (700, 150, 150)

• merge

  ds_1 = sv.DetectionDataset(...)
  len(ds_1)
  # 100
  ds_1.classes
  # ['dog', 'person']
  
  ds_2 = sv.DetectionDataset(...)
  len(ds_2)
  # 200
  ds_2.classes
  # ['cat']
  
  ds_merged = sv.DetectionDataset.merge([ds_1, ds_2])
  len(ds_merged)
  # 300
  ds_merged.classes
  # ['cat', 'dog', 'person']

• save

  dataset.as_yolo(
      images_directory_path=...,
      annotations_directory_path=...,
      data_yaml_path=...
  )
  
  dataset.as_pascal_voc(
      images_directory_path=...,
      annotations_directory_path=...
  )
  
  dataset.as_coco(
      images_directory_path=...,
      annotations_path=...
  )

• convert

  sv.DetectionDataset.from_yolo(
      images_directory_path=...,
      annotations_directory_path=...,
      data_yaml_path=...
  ).as_pascal_voc(
      images_directory_path=...,
      annotations_directory_path=...
  )

🎬 tutorials

Want to learn how to use Supervision? Explore our how-to guides, end-to-end examples, and cookbooks!

Dwell Time Analysis with Computer Vision | Real-Time Stream Processing

Created: 5 Apr 2024

Learn how to use computer vision to analyze wait times and optimize processes. This tutorial covers object detection, tracking, and calculating time spent in designated zones. Use these techniques to improve customer experience in retail, traffic management, or other scenarios.

Speed Estimation & Vehicle Tracking | Computer Vision | Open Source

Created: 11 Jan 2024

Learn how to track and estimate the speed of vehicles using YOLO, ByteTrack, and Roboflow Inference. This comprehensive tutorial covers object detection, multi-object tracking, filtering detections, perspective transformation, speed estimation, visualization improvements, and more.

💜 built with supervision

Did you build something cool using supervision? Let us know!

football-players-tracking-25.mp4

traffic_analysis_result.mov

vehicles-step-7-new.mp4

📚 documentation

Visit our documentation page to learn how supervision can help you build computer vision applications faster and more reliably.

🏆 contribution

We love your input! Please see our contributing guide to get started. Thank you 🙏 to all our contributors!

supervision's Issues

Separate line and zone counters for separate classes

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

My YOLO V8 is detecting & classifying several classes, say classes = [2, 3, 5, 7]:

I then follow examples by @SkalskiP and do something like:

line_counter = LineZone()
results = model.track(source=source_video, classes=classes, ... )
for result in results:
    detections = Detections.from_yolov8(result)
    detections.tracker_id = result.boxes.id.cpu().numpy().astype(int)
    line_counter.trigger(detections=detections)

Finally I can return and use line_counter.in_count and line_counter.out_count for other tasks.

Currently, LineZone().trigger will +=1 the in/out counts if a detection of any class crosses the line.

I'd like to keep a running tally of in/out counts for each class. Two possibilities:

• instantiate multiple LineZone objects, one for each class, or
• a single LineZone object to keep track of which class triggered how many counts.

I have modified my local code to accomplish this for now, but this would be a useful feature.

Perhaps LineZone.in_count and LineZone.out_count could old a vector corresponding to the class

Potential problem: Sometimes the classification of the tracked object flips between different classes during the tracking or line-crossing. My instinct is to use the majority class. For this, the line would have to be at the edge of the frame, so that we have seen all the classifications.

Anyone else solving a similar problem? Any better ideas?

Additional

No response

How to use yolov8 track with supervision?

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

i know if we ues object detection we can use the following code

>>> from ultralytics import YOLO
>>> from supervision import Detections

>>> model = YOLO('yolov8s.pt')
>>> results = model(frame)[0]
>>> detections = Detections.from_yolov8(results)

if i am using yolov8 tracking api
results = model.track(source=0, stream=True,show=True,classes=[0])
then how can i put the results in our Detection?

Additional

No response

Show_frame_in_notebook not working in Jupyter notebooks by default

I'm following how-to-track-and-count-vehicles-with-yolov8 notebook on my local machine and found that show_frame_in_notebook() runs but the output frame is not rendered.

After some googling I found out that we need to add %matplotlib inline to the cell where we use the function for it to work. and it is used only once.

Also when I try to add it to the first cell of the notebook the function doesn't render too it needs to be in the same cell or after it for the first run.

I know my problem is solved but I'd like to see if it can be added to the library directly and that it was not used on colab.

in real time video missing some frames and missing counting yolov8 and byte track code with supervision

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

in following code i used open cv and supervision with byte track for tracking and counting when with video it is working correctly and giving good out put. when i work with real time it is missing some frame and video writer also giving very speed video due to that. Please check code running you can see it. please help me really appreciate. I am really struck

https://www.youtube.com/watch?v=OS5qI9YBkfk I use above video ..

Additional

from typing import List
import supervision
import cv2
from supervision.draw.color import ColorPalette
from supervision.geometry.dataclasses import Point
from supervision.video.dataclasses import VideoInfo
from supervision.video.source import get_video_frames_generator
from supervision.video.sink import VideoSink
from supervision.notebook.utils import show_frame_in_notebook
from supervision.tools.detections import Detections, BoxAnnotator
from supervision.tools.line_counter import LineCounter, LineCounterAnnotator
from yolox.tracker.byte_tracker import BYTETracker, STrack
from onemetric.cv.utils.iou import box_iou_batch
from dataclasses import dataclass
import numpy as np
from ultralytics import YOLO
from tqdm.notebook import tqdm
import time
import os
import random
import sys

LINE_START = Point(630, 300)
LINE_END = Point(50, 300)
model = YOLO("best_yolov8.pt")
CLASS_NAMES_DICT = model.model.names
# class_ids of interest - car, motorcycle, bus and truck
CLASS_ID = [0]

fps = 1/30
width = 1280
height = 1280
video_codec = cv2.VideoWriter_fourcc("D", "I", "V", "X")

@dataclass(frozen=True)
class BYTETrackerArgs:
    track_thresh: float = 0.25
    track_buffer: int = 30
    match_thresh: float = 0.8
    aspect_ratio_thresh: float = 3.0
    min_box_area: float = 1.0
    mot20: bool = False

# converts Detections into format that can be consumed by match_detections_with_tracks function
def detections2boxes(detections: Detections) -> np.ndarray:
    return np.hstack((
        detections.xyxy,
        detections.confidence[:, np.newaxis]
    ))


# converts List[STrack] into format that can be consumed by match_detections_with_tracks function
def tracks2boxes(tracks:List[STrack]) -> np.ndarray:
    return np.array([
        track.tlbr
        for track
        in tracks
    ], dtype=float)


# matches our bounding boxes with predictions
def match_detections_with_tracks(
    detections: Detections, 
    tracks: List[STrack]
) -> Detections:
    if not np.any(detections.xyxy) or len(tracks) == 0:
        return np.empty((0,))

    tracks_boxes = tracks2boxes(tracks=tracks)
    iou = box_iou_batch(tracks_boxes, detections.xyxy)
    track2detection = np.argmax(iou, axis=1)
    
    tracker_ids = [None] * len(detections)
    
    for tracker_index, detection_index in enumerate(track2detection):
        if iou[tracker_index, detection_index] != 0:
            tracker_ids[detection_index] = tracks[tracker_index].track_id

    return tracker_ids

byte_tracker = BYTETracker(BYTETrackerArgs())
# create VideoInfo instance
name = random.randint(0, 1000)
print(name)
if os.path.isdir(str(name)) is False:
    name = random.randint(0, 1000)
    name = str(name)

name = os.path.join(os.getcwd(), str(name))
print("ALl logs saved in dir:", name)
os.mkdir(name)
#video_info = VideoInfo.from_video_path("cocnrec.mp4")
# create frame generator
#generator = get_video_frames_generator("cocnrec.mp4")

# create LineCounter instance
line_counter = LineCounter(start=LINE_START, end=LINE_END)
# create instance of BoxAnnotator and LineCounterAnnotator
box_annotator = BoxAnnotator(color=ColorPalette(), thickness=4, text_thickness=4, text_scale=2)
line_annotator = LineCounterAnnotator(thickness=4, text_thickness=4, text_scale=2)


#cap = cv2.VideoCapture("cocnrec.mp4")
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_BUFFERSIZE, 1)
ret = cap.set(3, 864)
ret = cap.set(4, 480)
cur_dir = os.path.dirname(os.path.abspath(sys.argv[0]))
#fps_MS = int(fps * 1000)


start = time.time()
video_file_count = 1
video_file = os.path.join(name, str(video_file_count) + ".avi")
print("Capture video saved location : {}".format(video_file))

video_writer = cv2.VideoWriter(
    video_file, video_codec, fps, (int(cap.get(3)), int(cap.get(4)))
)

# open target video file
#with VideoSink("test.mp4", video_info) as sink:
    # loop over video frames
    #for frame in tqdm(generator, total=video_info.total_frames):
while cap.isOpened():
    start_time = time.time()
    ret, frame = cap.read()
    if ret == True:
        # model prediction on single frame and conversion to supervision Detections
        results = model(frame)
        detections = Detections(
            xyxy=results[0].boxes.xyxy.cpu().numpy(),
            confidence=results[0].boxes.conf.cpu().numpy(),
            class_id=results[0].boxes.cls.cpu().numpy().astype(int)
        )
        # filtering out detections with unwanted classes
        mask = np.array([class_id in CLASS_ID for class_id in detections.class_id], dtype=bool)
        detections.filter(mask=mask, inplace=True)
        # tracking detections
        tracks = byte_tracker.update(
            output_results=detections2boxes(detections=detections),
            img_info=frame.shape,
            img_size=frame.shape
        )
        tracker_id = match_detections_with_tracks(detections=detections, tracks=tracks)
        detections.tracker_id = np.array(tracker_id)
        # filtering out detections without trackers
        mask = np.array([tracker_id is not None for tracker_id in detections.tracker_id], dtype=bool)
        detections.filter(mask=mask, inplace=True)
        # format custom labels
        labels = [
            f"#{tracker_id} {CLASS_NAMES_DICT[class_id]} {confidence:0.2f}"
            for _, confidence, class_id, tracker_id
            in detections
        ]
        # updating line counter
        line_counter.update(detections=detections)
        # annotate and display frame
        frame = box_annotator.annotate(frame=frame, detections=detections, labels=labels)
        line_annotator.annotate(frame=frame, line_counter=line_counter)
        cv2.imshow("frame", frame)
        if time.time() - start > 1000:
            start = time.time()
            video_file_count += 1
            video_file = os.path.join(name, str(video_file_count) + ".avi")
            video_writer = cv2.VideoWriter(
                video_file, video_codec, fps, (int(cap.get(3)), int(cap.get(4)))
            )
            # No sleeping! We don't want to sleep, we want to write
            # time.sleep(10)

        # Write the frame to the current video writer
        video_writer.write(frame)
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        break
cap.release()
cv2.destroyAllWindows()

0.3.0 IndexError on polygon_zones.py

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

I'm consistently getting an IndexError when using polygon zones:

Traceback (most recent call last):
  File "/home/rodrigo/Documents/counter-pog/main.py", line 63, in <module>
    main()
  File "/home/rodrigo/Documents/counter-pog/main.py", line 53, in main
    zone1.trigger(detections=detections)
  File "/home/rodrigo/Documents/counter-pog/.venv/lib/python3.10/site-packages/supervision/detection/polygon_zone.py", line 34, in trigger
    is_in_zone = self.mask[anchors[:, 1], anchors[:, 0]]
IndexError: index 720 is out of bounds for axis 0 with size 720

Here is the python file in question:

import cv2

from ultralytics import YOLO
import supervision as sv
import numpy as np


VIDEO_PATH = "video.mp4"

def main():
    zone1_polygon = np.array([[0, 100], [320, 100], [320, 539], [0, 539]])

    video_info = sv.VideoInfo.from_video_path(VIDEO_PATH)
    print(video_info)


    zone1= sv.PolygonZone(polygon=zone1_polygon, frame_resolution_wh=(video_info.resolution_wh))

    zone1_annotator = sv.PolygonZoneAnnotator(zone = zone1, color=sv.Color.white(), thickness=2)



    box_annotator = sv.BoxAnnotator(
        thickness=2,
        text_thickness=1,
        text_scale=0.4
    )

    model = YOLO("yolov8n.pt")

    for result in model.track(VIDEO_PATH, show=False, stream=True, agnostic_nms=True):
        
        frame = result.orig_img
        detections = sv.Detections.from_yolov8(result)

        if result.boxes.id is not None:
            detections.tracker_id = result.boxes.id.cpu().numpy().astype(int)
        
        detections = detections[detections.class_id == 0]

        labels = [
            f"#{tracker_id}"
            for _, confidence, class_id, tracker_id
            in detections
        ]

        frame = box_annotator.annotate(
            scene=frame, 
            detections=detections,
            labels=labels
        )

        zone1.trigger(detections=detections)
        frame = zone1_annotator.annotate(scene=frame)

        cv2.imshow("yolov8", frame)

        if (cv2.waitKey(30) == 27):
            break


if __name__ == "__main__":
    main()

Here is a google colab with the code:

https://colab.research.google.com/drive/1qPiu6dTjNxjpblLRlIUKgsWtN078usEs?usp=sharing

video.mp4

Environment

• Supervision: 0.3.0
• OS: Ubuntu 22.10
• Python: 3.10.7

Minimal Reproducible Example

Execute the python script on the video (both posted above). After a few seconds, the error shows up.

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

How to export the mask as a png image?

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

I am using supervision in combination with segment-anynthing and would like to obtain a simple PNG image for the mask, with a different pixel value for each segmented area. It is optional to provide a list of pixel values and their corresponding label types (e.g. glasses, faces, cars), but this is not essential.

Is there an easy way to do it?

Just found

>>> dataset.as_yolo(
...     images_directory_path='...',
...     annotations_directory_path='...',
...     data_yaml_path='...'
... )

>>> dataset.as_pascal_voc(
...     images_directory_path='...',
...     annotations_directory_path='...'
... )

and

sv.plot_images_grid(
    images=[img_bgr, annotated_image],
    grid_size=(1, 2),
    titles=['source image', 'segmented image']
)

It would be enough to export to plot to a png (of the annotated_image)

And print out the labels somehow.

Additional

No response

To get the count of objects (in/out) in the polygon area, like we have it for line_zone using the supervision library for yolo-v8

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

Count of object that goes in or goes out of the polygon area. Also, the count(in/out) by each class. The yolo model can be either a detection or a segmentation model.

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

`Detections.export()` - a method to write detections to file

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

A new method for Detections class to write the contents to file in various formats.

Use case

• Bounding boxes and annotations saved to file in YOLO format (txt), without saving the video.
• These may be opened in CVAT for manual inspection or correction etc.
• e.g., https://opencv.github.io/cvat/docs/manual/advanced/formats/format-yolo/

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Use Webcam Script in Touchdesigner

Discussed in #24

^{Originally posted by taygon February 17, 2023}
I'm attempting to repurpose the webcam script from this video to use within Touchdesigner. What I'd like is to circumvent using cv2.VideoCapture, frame, etc. and instead convert a live webcam feed within Touchdesigner to a numpyArray and use that as my source.

Instead of using cv2.imshow, I'd then like to convert the final video output with the bounding boxes, confidence score etc. back to a numpyArray so that I can further manipulate it in Touchdesigner. Here's my script so far:

The error I'm getting is: RuntimeError: Given groups=1, weight of size [16, 3, 3, 3], expected input[1, 4, 384, 640] to have 3 channels, but got 4 channels instead

There might be other things I'm doing wrong as well. Any help would be much appreciated.

`tracker_id=None` when merging a sequence of detections `det_1`, `det_2`, ..., starting with an initial empty `Detection`

          I had a similar issue when merging a sequence of detections   `det_1`, `det_2`, ..., starting with an initial empty `Detection`: 

dets_all = Detections.empty()
for det in ...:
    dets_all = Detections.merge([dets_all, det])

The empty detection doesn't have a tracker_id:

supervision/supervision/detection/core.py

Lines 345 to 364 in e16cca4

	@classmethod
	def empty(cls) -> Detections:
	"""
	Create an empty Detections object with no bounding boxes, confidences, or class IDs.
	Returns:
	(Detections): An empty Detections object.
	Example:
	```python
	>>> from supervision import Detections
	>>> empty_detections = Detections.empty()
	```
	"""
	return cls(
	xyxy=np.empty((0, 4), dtype=np.float32),
	confidence=np.array([], dtype=np.float32),
	class_id=np.array([], dtype=int),
	)

For all subsequent detections, when merged (concatenated is a better name), tracker_id seems to be getting set to None here:

supervision/supervision/detection/core.py

Lines 366 to 413 in e16cca4

	@classmethod
	def merge(cls, detections_list: List[Detections]) -> Detections:
	"""
	Merge a list of Detections objects into a single Detections object.
	This method takes a list of Detections objects and combines their respective fields (`xyxy`, `mask`,
	`confidence`, `class_id`, and `tracker_id`) into a single Detections object. If all elements in a field are not
	`None`, the corresponding field will be stacked. Otherwise, the field will be set to `None`.
	Args:
	detections_list (List[Detections]): A list of Detections objects to merge.
	Returns:
	(Detections): A single Detections object containing the merged data from the input list.
	Example:
	```python
	>>> from supervision import Detections
	>>> detections_1 = Detections(...)
	>>> detections_2 = Detections(...)
	>>> merged_detections = Detections.merge([detections_1, detections_2])
	```
	"""
	if len(detections_list) == 0:
	return Detections.empty()
	detections_tuples_list = [astuple(detection) for detection in detections_list]
	xyxy, mask, confidence, class_id, tracker_id = [
	list(field) for field in zip(*detections_tuples_list)
	]
	all_not_none = lambda l: all(x is not None for x in l)
	xyxy = np.vstack(xyxy)
	mask = np.vstack(mask) if all_not_none(mask) else None
	confidence = np.hstack(confidence) if all_not_none(confidence) else None
	class_id = np.hstack(class_id) if all_not_none(class_id) else None
	tracker_id = np.hstack(tracker_id) if all_not_none(tracker_id) else None
	return cls(
	xyxy=xyxy,
	mask=mask,
	confidence=confidence,
	class_id=class_id,
	tracker_id=tracker_id,
	)

Originally posted by @zburq in #88 (comment)

`ClassificationDataset` API

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

import numpy as np

@dataclass
class Classifications:
    # you only get class_id if it is loading an existing dataset, 
    # but you get class_id and confidence if you are getting the result of the classification
    class_id: np.ndarray # class_id.shape = (N, )
    confidence: Optional[np.ndarray] = None # if confidence exists, confidence.shape = (N, )

    def __post_init__(self):
        # validation of consistency 
        pass

    # easy way to extract top N classifications

@dataclass
class ClassificationDataset(BaseDataset):
    classes: List[str]
    images: Dict[str, np.ndarray]
    annotations: Dict[str, Classifications]

    def __len__(self) -> int:
        return len(self.images)

    def split(
        self, split_ratio=0.8, random_state=None, shuffle: bool = True
    ) -> Tuple[ClassificationDataset, ClassificationDataset]:
        image_names = list(self.images.keys())
        train_names, test_names = train_test_split(
            data=image_names,
            train_ratio=split_ratio,
            random_state=random_state,
            shuffle=shuffle,
        )

        train_dataset = ClassificationDataset(
            classes=self.classes,
            images={name: self.images[name] for name in train_names},
            annotations={name: self.annotations[name] for name in train_names},
        )
        test_dataset = ClassificationDataset(
            classes=self.classes,
            images={name: self.images[name] for name in test_names},
            annotations={name: self.annotations[name] for name in test_names},
        )
        return train_dataset, test_dataset

    def as_multiclass_folder_structure(self, root_directory_path: str) -> None:
        pass

    @classmethod
    def from_multiclass_folder_structure(cls, root_directory_path: str) -> ClassificationDataset:
        # we are not implementing that yet
        pass

annotate masks

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

if we use the segmentation model, we should annotate the segments masks with supervision, as you did with boses and ids
thanks :)

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Getting the class_id of the tracked object

I saw your youtube video on how to track objects with live video stream (great video BTW). I was wondering if there is a way to get the class_id/ name of the in/out objects that cross the line zone. For example, I would like to know that an Apple went in and an Orange went out. In your video you are counting all the objects going in and out regardless of their class_id.

How to select detections by index?

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

I'd like to get a single detection by index but the way I've found to do it seems a little clunky. Is there a cleaner way to do this?

For example:

detections = sv.Detections.from_sam(sam_annotations)
big_detections = detections[detections.area > 1000] # filters detections by area
gets_single_detection = detections[np.array([0])] # gets the first detection
gives_error = detectons[0] # could return the first detection but throws an error
gives_error = detectons[[0,1,2]] # could return the first detection but throws an error

Question: Is there a different intended way to access detections by index?

If not, you could update the __getitem__ method of the Detections dataclass to allow passing ints, or lists of ints instead of only allowing np.array objects.

Suggested modification:

    def __getitem__(self, index: np.ndarray) -> Detections:
        index = np.asarray(index) # ADDED
        if isinstance(index, np.ndarray) and (
            index.dtype == bool or index.dtype == int
        ):
            return Detections(
                xyxy=self.xyxy[index],
                mask=self.mask[index] if self.mask is not None else None,
                confidence=self.confidence[index]
                if self.confidence is not None
                else None,
                class_id=self.class_id[index] if self.class_id is not None else None,
                tracker_id=self.tracker_id[index]
                if self.tracker_id is not None
                else None,
            )
        raise TypeError(
            f"Detections.__getitem__ not supported for index of type {type(index)}."
        )

Additional

No response

Draw ellipse from bounding box

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

How do I draw ellipse from the bounding box from BoxAnnotator class with YOLOv8?

Additional

No response

An elegant way to apply thresholds and area filters etc. at the class level

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

Thanks once more, this package has saved me a lot of time, and already improved how my code looks and feels.

Possible I'm just requesting something to be added to the docs.

I can't work out a semi-elegant way to apply thresholds at the class level, or apply area filters at the class level. If I just have to iterated and then merge_detections or something like that that's fine, just thought you may already have an approach.

Use case

You have some small object, filter any detections of that small object above a certain size.

Every other object has a very large range of sizes.

Additional

nope

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

using from_yolov8 in google colab

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Hello guys

I'm trying to use ur code provided here in this tuto : (YOLOv8 native tracking | Step-by-step tutorial | Tracking with Live Webcam Stream)

the first code : to read frames from my video, and for each frame I will call the function maintracher(). to track objects in each frame.

from supervision.draw.color import ColorPalette
from supervision.geometry.dataclasses import Point
from supervision.video.dataclasses import VideoInfo
from supervision.video.source import get_video_frames_generator
from supervision.video.sink import VideoSink
from supervision.notebook.utils import show_frame_in_notebook
from supervision.tools.detections import Detections, BoxAnnotator
from supervision.tools.line_counter import LineCounter, LineCounterAnnotator

import cv2
from ultralytics import YOLO
from supervision.tools.line_counter import LineCounter
from supervision.geometry.dataclasses import Point
import numpy as np
from google.colab.patches import cv2_imshow


LINE_START = Point(50, 1500)
LINE_END = Point(3790, 1500)


video_path = '/content/gdrive/MyDrive/Videos/Street.mp4'

# Open the input video
video = cv2.VideoCapture(video_path)

# Get the video properties
fps = video.get(cv2.CAP_PROP_FPS)
width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))

# Define the output video codec and create the output video writer
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter('output_video.mp4', fourcc, fps, (width, height))

while True:
    # Read the next frame from the video
    ret, frame = video.read()
    if not ret:
        break
    frame = mainTracker(frame)
    out.write(frame)
video.release()
out.release()

the second part

def mainTracker(frame):
    line_counter = LineCounter(start=LINE_START, end=LINE_END)
    # create instance of BoxAnnotator and LineCounterAnnotator
    box_annotator = BoxAnnotator(color=ColorPalette(), thickness=2, text_thickness=2, text_scale=1)
    line_annotator = LineCounterAnnotator(thickness=2, text_thickness=2, text_scale=1)
      
    model = YOLO("yolov8l.pt")
    for result in model.track(source=frame, show=True, stream=True, agnostic_nms=True):
        
        #frame = result.orig_img
        detections = from_yolov8(result)

        if result.boxes.id is not None:
            detections.tracker_id = result.boxes.id.cpu().numpy().astype(int)
        
        detections = detections[(detections.class_id != 60) & (detections.class_id != 0)]

        labels = [
            f"{tracker_id} {model.model.names[class_id]} {confidence:0.2f}"
            for _, confidence, class_id, tracker_id
            in detections
        ]

        frame = box_annotator.annotate(
            scene=frame, 
            detections=detections,
            labels=labels
        )

        line_counter.trigger(detections=detections)
        line_annotator.annotate(frame=frame, line_counter=line_counter)

        cv2_imshow("yolov8", frame)

        if (cv2.waitKey(30) == 27):
            break
        return frame 

I got an error because I don't know how to call this function from_yolov8
can u please help me on this error

Thanks

Additional

No response

The line anottator doesn't work well

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

When I draw the line to count people passing it with the linezone and linezone annotator classes it sometimes counts people passing far from the line.

All this has been done using yolov8.

Environment

No response

Minimal Reproducible Example

import cv2

from ultralytics import YOLO
import supervision as sv
import numpy as np

#Line for example_01

LINE_START = sv.Point(0, 227)

LINE_END = sv.Point(401, 227)

#Line for prueba2

LINE_START = sv.Point(0, 500)

LINE_END = sv.Point(1275, 500)

#Line for beach_video
#Linea puesta mas arriba de lo que seria logico porque si no la gente de fuera la linea detecta la bounding box y los cuenta como que han entrado
LINE_START = sv.Point(98,1737)
LINE_END = sv.Point(1182, 1477)

def main():
line_counter = sv.LineZone(start=LINE_END, end=LINE_START) #LineZone is a line that can count objects
line_annotator = sv.LineZoneAnnotator(thickness=2, text_thickness=1, text_scale=0.5) #Format of LineZone
box_annotator = sv.BoxAnnotator(
thickness=2,
text_thickness=1,
text_scale=0.5
)

segmentation_model = "yolov8n-seg.pt"
detection_model = "yolov8n.pt"
onnx = "yolov8n.onnx"
model = YOLO(detection_model) #yolov8n faster but less accurate than yolov8x
path = "Examples/prueba2.mp4"
path2 = "102 - Armada1 - 2023-05-09 12-30-31-345.mp4"

for result in model.track(source=path2, show=False, stream=True, agnostic_nms=False,imgsz=[640,640],device='0',classes=[0]):
    
    frame = result.orig_img
    detections = sv.Detections.from_yolov8(result)
    
    if result.boxes.id is not None: #prevent crashing when no object is detected
        detections.tracker_id = result.boxes.id.cpu().numpy().astype(int)
    
    detections = detections[detections.class_id == 0] #Only detect people

    
    labels = [
        f"{model.model.names[class_id]} {confidence:0.2f}"
         for _, _, confidence, class_id, _
         in detections
    ]


    frame = box_annotator.annotate(  #Draws boxes around objects
        scene=frame, 
        detections=detections,
        labels=labels
    )
    #Draws line and counts objects
    line_counter.trigger(detections=detections)
    line_annotator.annotate(frame=frame, line_counter=line_counter)
    cv2.namedWindow('yolov8', cv2.WINDOW_NORMAL)
    cv2.imshow("yolov8", frame)
    if (cv2.waitKey(30) == 27):
        break
    #Prints the number of objects that have crossed the line
    print("in_counter: ",line_counter.in_count)
    print("out_counter: ",line_counter.out_count)

if name == "main":
main()

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Unable to change the In and Out Text in line_counter.py

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Hi, I am currently checking the line_counter.py file and trying to change the in and out text. Even after changing the word here as:

out_text = f"Total: {line_counter.out_count}"

It is not getting updated in the video. What am I doing wrong?
Thank you

Additional

No response

Take the frame by sec

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Is there any way to collect the frame by sec, like if i have a clip length 400sec, then i'll have a list 400 frame? I have search through process_video() and figure out it took the frame base on the index of the frame.

Additional

No response

0.2.1 bug: AttributeError: 'list' object has no attribute 'boxes'

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

I was running the Roboflow tutorial: https://www.youtube.com/watch?v=QV85eYOb7gk

And after adding in this code:
while True:
ret, frame = cap.read()
result = model(frame)
detections = sv.Detections.from_yolov8(result)

    frame = box_annotator.annotate(scene=frame, detections=detections)

    cv2.imshow("yolo8", frame)

    if (cv2.waitKey(30) == 27):
        break

It errored with this message:
File "/Data/Dropbox/Python/YOLO/YOLO8/VehicleID/vehicleID_env/lib/python3.10/site-packages/supervision/detection/core.py", line 130, in from_yolov8
xyxy=yolov8_results.boxes.xyxy.cpu().numpy(),
AttributeError: 'list' object has no attribute 'boxes'

Environment

-Supervision 0.2.1 Latest PIP install on 2/20/2022
-Ultralytics: 8.0.26
-Ubuntu 22.04 LTS

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Zone_annotator error when Yolo predictions/tracking are out of bounds (resolution)

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

Hello, I am reporting an issue I found when using Yolo tracking/prediction results. Zone_annotator does not work properly when using the Yolo track model results. Sometimes, prediction index goes out of bounds when the object goes out of the image to fast. Out of bounds values in the xyxy detections/predictions (e.g. predicted 480.4, but limit is 480.0 in the resolution). I think it is due to X and Y being interpolated out of the index while moving? Then the zone_annotator gives an error at "zone.trigger(detections=detections)" and stop the code:
Detections(xyxy=array([[136.70734, 166.26642, 641.2425 , 480.40762],
[157.55916, 270.19525, 212.24619, 369.16534]], dtype=float32), class_id=array([ 0, 67]), confidence=array([0.96505105, 0.938522 ], dtype=float32), tracker_id=array([2, 4]))
Traceback (most recent call last):
File "C:\Users\artur\miniconda3\lib\site-packages\supervision\detection\polygon_zone.py", line 34, in trigger
is_in_zone = self.mask[anchors[:, 1], anchors[:, 0]]
IndexError: index 480 is out of bounds for axis 0 with size 480

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Using Unique id for counting objects

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

In most of the tutorials I saw we are using a line or a polygon to count of number of objects in the entire video or in the frame inside the polygon. In cases like counting vehicles in an unstructured environment it would be very difficult to count the vehicles using a line or a polygon for that matter.
So, since there is a unique id assigned for each of the objects we detect and then track, we can then use the id's to count the number of objects in the entire video or in the frame. So, whenever there is a unique id that we have not seen before then we can increase the counter.

Additional

Since we also know the type of class an object belongs to whenever a id is assigned to it, we can use this information to make counters for different type of objects. Then we can get a total count of each type of objects at the end of a video.

Unexpected parameter ordering in Detection class during iteration

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

In the Detections class I expected to unpack the values in this order

for xyxy, mask, class_id, confidence, tracker_id in detections:
  ...   

due to the documentation string listing as

xyxy: np.ndarray
mask: np.Optional[np.ndarray] = None
class_id: Optional[np.ndarray] = None
confidence: Optional[np.ndarray] = None
tracker_id: Optional[np.ndarray] = None

but this works instead

for xyxy, mask, confidence, class_id, tracker_id in detections:
  ...   

This is due to the confidence and class_id being in a different order in the __iter__ function

yield (
    self.xyxy[i],
    self.mask[i] if self.mask is not None else None,
    self.confidence[i] if self.confidence is not None else None,
    self.class_id[i] if self.class_id is not None else None,
    self.tracker_id[i] if self.tracker_id is not None else None,
)

Is there some consistency here that could be improved?

Environment

• Supervision: 0.5.2

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Always tracker_id=None

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Hello everybody

I still have tracker_id in None , I have just tried everything on the net

ret, frame = cap.read()
       result= model.track(frame,tracker="botsort.yaml",device=0,agnostic_nms=True,show_labels=True,half=True,conf=confidence,vid_stride=True,iou=0.25)[0]

detections = sv.Detections.from_yolov8(result)
if result.boxes.id is not None:
     detections.tracker_id = result.boxes.id.cpu().numpy().astype(int)

Additional

No response

Can we display the contours of detections in dotted lines

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Hello everyone

Can we display the contours of detections in dotted lines in BoxAnnotator?
best regards

Additional

No response

Hide console output when objects are detected

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Is there a way to hide the output to console when objects are detected?

Additional

No response

crop bounding boxes as separate image

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Im curious as to how one can crop the sv.Detections as a separate image for further analysis >

Additional

No response

LineZone and LineZoneAnnotator not exposed at top level

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

fails to expose latest LineZone changes

line_counter = sv.LineZone(start=LINE_START, end=LINE_END)
AttributeError: module 'supervision' has no attribute 'LineZone'

Environment

Supervision: 0.2.1

Minimal Reproducible Example

No response

Additional

just need to add

from supervision.detection.line_counter import LineZone, LineZoneAnnotator

to top level init.py

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Detections.__getitem__ not supported for index of type <class 'numpy.ndarray'>.

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

this is a link to my notebook, please take a look
https://colab.research.google.com/drive/1nFgpu2bTKLDuFfxeSW6yzzHzIkKtQi9w?authuser=1#scrollTo=jbzTzHJW22up
I downloaded the dataset after doing annotations from roboflow and then uploaded the zip file to google drive.

Environment

torch: 1.13
cuda: cu116
roboflow: 0.2.34
supervision: 0.3.0
transformers: 4.26.1

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

How can I plot multiple mask on the same image

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Thanks for this library. Following segment anything tutorial, I would like to plot on the same image all the masks and on the original image all the annotation boxes. Can you share a code on how can I do it please?

Additional

No response

Model Evaluation

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

It would be great if we can evaluate model performance with supervision detections.

Use case

• Easy performance comparision between multiple models
• Easy debug process of models

Additional

Object detections models can be evaluated by mean average precision.
Instance segmentation models can be evaluated by mean average precision with mIOU.
Multi object trackers can be evaluated by py-motmetrics

If one can create/suggest architecture then I can help with implementation.

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

text color on Start and End Point Circles in LineZone annotator

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

just a minor thing but i noticed that circles at the end and start of the Line Counters are getting the Text Color instead of the Line color . if im not mistaken the circles colors should match the Lines Right?

Also i suggest if it's possible to add some "roundness" to the text boxes of the LineZone annotator and PolygonZone annotator IMO it looks more appealing than a plain rectangle

cv2.circle(
frame,
line_counter.vector.start.as_xy_int_tuple(),
radius=5,
color=self.text_color.as_bgr(),
thickness=-1,
lineType=cv2.LINE_AA,
)
cv2.circle(
frame,
line_counter.vector.end.as_xy_int_tuple(),
radius=5,
color=self.text_color.as_bgr(),
thickness=-1,
lineType=cv2.LINE_AA,
)

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Unite efforts with backpack developers?

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Hello, I am the maintainer of the panorama-backpack library, a toolset of computer vision utilities. Skimming through the features of supervision, it seems that the two libraries have very similar objectives. Originally, backpack was born to aim application development for AWS Panorama computer vision platform, but some 90% of the code is platform-agnostic (for example, there are annotation drawers for both Panorama and OpenCV).

I wanted to ask you if you are interested joining our efforts, and if yes, whether you have an idea in which format could it be down. If you are interested, you can have a look at backpack and its features at the project's GitHub page or the documentation.

Have a nice day!
Janos

Additional

No response

Track and Count Objects Reset Counter

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Every time I create a video from the model, the counter continues from where it is and does not reset.

Additional

No response

Use for custom dataset

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

Thanks, it was so amazing.
I have one question. I'll be so grateful if you could help:

1. I watched the video that explained how to train yolov5 with the custom dataset. I used that notebook and exported the best.pt file, containing model weights. My question is: how to use it for inference?

• I used model = torch.hub.load('ultralytics/yolov5', 'custom', path='best.pt') # PyTorch
  but It loads the original .pt file, not my custom one.

Environment

colab

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Combine Detections objects

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

def combine_detections(detections_list, overwrite_class_ids):
    if len(detections_list) == 0:
        return Detections.empty()

    if overwrite_class_ids is not None and len(overwrite_class_ids) != len(detections_list):
        raise ValueError("Length of overwrite_class_ids must match the length of detections_list.")

    xyxy = []
    mask = []
    confidence = []
    class_id = []
    tracker_id = []

    for idx, detection in enumerate(detections_list):
        xyxy.append(detection.xyxy)

        if detection.mask is not None:
            mask.append(detection.mask)

        if detection.confidence is not None:
            confidence.append(detection.confidence)

        if detection.class_id is not None:
            if overwrite_class_ids is not None:
                # Overwrite the class IDs for the current Detections object
                class_id.append(np.full_like(detection.class_id, overwrite_class_ids[idx]))
            else:
                class_id.append(detection.class_id)

        if detection.tracker_id is not None:
            tracker_id.append(detection.tracker_id)

    xyxy = np.vstack(xyxy)
    mask = np.vstack(mask) if mask else None
    confidence = np.hstack(confidence) if confidence else None
    class_id = np.hstack(class_id) if class_id else None
    tracker_id = np.hstack(tracker_id) if tracker_id else None

    return Detections(xyxy=xyxy, mask=mask, confidence=confidence, class_id=class_id, tracker_id=tracker_id)

Use case

Combining Detections if you've inferred multiple times

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

sv.plot_images_grid shows nothing

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

As in the title, function sv.plot_images_grid returns nothing even though right images (numpy.ndarray) are given as arguments.

I'm using SAM model, to detect holes in targets, and results [2][1] is just one of four parts of detected_target, because without partitioning, model wasn'tso accurate in his detections. For example, cv2_imshow(results[2][1]) properly shows detected image.

I need this to visualize which masks are holes, and due to that bug I can't run sv.plot_images_grid on masks

Am I doing something wrong or it's just a bug? Thanks for help in advance.

Environment

I installed packages using this command from yours tutorial: !pip install -q jupyter_bbox_widget dataclasses-json supervision
so I assume it's the newest version of supervision. Python version on colab is 3.10 if I'm not mistaken.

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

`core.py` uses deprecated alias `np.bool`

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

Looks like NumPy deprecated the np.bool alias with release 1.20.0. They recommend changing np.bool to bool

Environment

• Supervision 0.2.0
• Windows 10
• Python 3.9.11

Minimal Reproducible Example

I got this error when trying to follow the Detect and Count Objects in Polygon Zone with YOLOv5 / YOLOv8 / Detectron2 + Supervision tutorial on my machine at home (not using a notebook).

Additional

In the /detection/core.py file

supervision/supervision/detection/core.py

Line 175 in 2e76e3d

if isinstance(index, np.ndarray) and index.dtype == np.bool:

Change np.bool to bool

I'm kind of curious why this error doesn't get thrown when following along in Colab.

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

LineZone expecting 4 values rather than the new 5 from adding mask to Detections Class.

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

After upgrading to the latest commit today, (v0.5.1) The LineZone Class tries to iterate over the detections and unpack 4 values rather than the new 5.

supervision/supervision/detection/line_counter.py

Line 38 in e091f9c

for xyxy, confidence, class_id, tracker_id in detections:

For now on my end I just added _ as I'm not using mask at all. and it solved the error message of:

too many values to unpack (expected 4)

I don't know if mask is needed or is planned to be implemented in the LineZone trigger function, otherwise I would just submit a PR with the line changed to:

for xyxy, _, confidence, class_id, tracker_id in detections:

Environment

• Supervision 0.5.1
• Ubuntu 22.10
• Python 3.8.10
• YOLOv8

Minimal Reproducible Example

import supervision as sv

line_zone = sv.LineZone(start=LINE_START, end=LINE_END)
result = model.predict(source=frame, stream=False)
detections = Detections(
            xyxy=result.boxes.xyxy.cpu().numpy(),
            confidence=result.boxes.conf.cpu().numpy(),
            class_id=result.boxes.cls.cpu().numpy().astype(int),
        )
tracker_ids = match_detections_with_tracks(detections=detections, tracks=tracks)
detections.tracker_id = np.array(tracker_ids)
detections = detections[detections.tracker_id != None]
labels = [
                f"Item: #{tracker_id} {model.names[class_id]} {confidence:0.2f}"
                for _, _, confidence, class_id, tracker_id in detections
            ]
line_zone.trigger(detections)  <--------- This is where it fails
        

Additional

I'm using Yolov8, and using ByteTracker stuff from some of the previous videos to get my tracker_id's.

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Add object trace annotation to Supervision 🤩

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

Scope

• Create Trace and TraceAnnotator classes.
• Trace is responsible for calculating the history of bounding box positions within a given time (frame count) window. Every time you use the update method, you update the self.trace dictionary inside. self.trace stores the history of bounding box positions for every trace_id.
• TraceAnnotator draws Trace on frame using utilities from https://github.com/roboflow/supervision/blob/main/supervision/draw/utils.py

API

class Trace:

    def __init__(
        self,
        position: Position = Position.BOTTOM_CENTER,
        max_length: int = 10,
    ):
        self.trace: Dict[int, List[sv.Point]] = {}
        self.position = position
        self.max_length = max_length

    def update(self, detections: Detections) -> None:
        pass


class TraceAnnotator:

    def __init__(
        self,
        color: Color = Color.white(),
        thickness: int = 2
    ):
        pass

    def annotate(self, scene: np.ndarray, trace: Trace) -> np.ndarray:
        pass

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

YOLOv7 and supervision

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Hi searched around and have not found any implimentations of supervision on YOLOv7. Have you experimented with this? Should it be the same method as demonstrated in the YOLOv8?

Additional

No response

line counter is not working anymore?

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

this is my code I wanted to add multiple lines and count the objects, but the line counter is not working anymore with the latest supervision.

import os
import torch
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0"
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
import numpy as np
import json
from ultralytics import YOLO
import supervision as sv
# from supervision.tools.line_counter import LineCounter
# from supervision.geometry.dataclasses import Point


# LINE_START = Point(50, 1500)
# LINE_END = Point(3790, 1500)

# line_counter = LineCounter(start=LINE_START, end=LINE_END)

model = YOLO('yolov8n.pt')

def main_func(input_data,frame):

    global model

    if 'ignore' in input_data:
        return "ignore"
    else:

        input_data = json.loads(input_data)

        results= model.track(frame, imgsz=640, classes=0,tracker="bytetrack.yaml")[0]
        detections = sv.Detections.from_yolov8(results)

        if results.boxes.id is not None:
            detections.tracker_id = results.boxes.id.cpu().numpy().astype(int)
            detections.conf = results.boxes.conf.cpu().numpy().astype(float)
            detections.xyxy = results.boxes.xyxy.cpu().numpy().astype(int)
            
            # print(detections.tracker_id)
            # print(detections.conf)
            # print(detections.xyxy)
        else:
            detections.tracker_id = np.array([])
            detections.conf = np.array([])
            detections.xyxy = np.array([])
 
        new_box=detections.xyxy
        new_box = new_box.astype("int").tolist()
        new_ids = detections.tracker_id.astype("int").tolist()
        new_confs = detections.conf.astype("float").tolist()    



        print("count",count)

        output = {
                  'frame_index':input_data['frame_index'],
                  'time_stamp':input_data['time_stamp'],
                  'detections':new_box,
                  'demo_run':input_data['demo_run'],
                  'ids':new_ids,
                  'confs':new_confs
                  }
        
        print(output)
        return output

except for the polygon , how to have multiple lines and update the counts?

Additional

No response

Detections.__getitem__ not supported for index of type <class 'numpy.ndarray'>

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

I downloaded the dataset after doing annotations from roboflow and then uploaded the zip file to google drive. I have used the same code with my own dataset of just 2 classes. Why did this error occurs?
My test annotation file is shown below:
{"info":{"year":"2023","version":"1","description":"Exported from roboflow.ai","contributor":"","url":"[https://public.roboflow.ai/object-detection/undefined","date_created":"2023-03-13T13:21:50+00:00"},"licenses":{"id":1,"url":"https://creativecommons.org/licenses/by/4.0/","name":"CC BY 4.0"}],"categories":[{"id":0,"name":"labels","supercategory":"none"},{"id":1,"name":"rice leaf caterpillar","supercategory":"labels"},{"id":2,"name":"rice leaf roller","supercategory":"labels"}],"images":[{"id":0,"license":1,"file_name":"IP001000460_jpg.rf.7a6253055fc0141a270d2c2275d20f9b.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":1,"license":1,"file_name":"IP000000279_jpg.rf.16cb9316608f637b9ab411191a627c4a.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":2,"license":1,"file_name":"IP000000061_jpg.rf.1fd888279be601c1b8c0e703d2a227bd.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":3,"license":1,"file_name":"IP001000375_jpg.rf.3092d271809e96e0c409a00b98bb04ab.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":4,"license":1,"file_name":"IP001000016_jpg.rf.54b10e8843b41abbb7274d9fec3b2fbe.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":5,"license":1,"file_name":"IP001000179_jpg.rf.1897a09345d8cdab3c8e657bf2656fb0.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":6,"license":1,"file_name":"IP001000455_jpg.rf.005f5066752d3d20d153d83c1c8737a8.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":7,"license":1,"file_name":"IP000000075_jpg.rf.d139beb487b65d39bf1516d8ab548660.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":8,"license":1,"file_name":"IP000000320_jpg.rf.c755fca43e8eef8ff9df0bdb816c376d.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":9,"license":1,"file_name":"IP001000419_jpg.rf.61bd6c3cf3c3a704339ebeea92136de2.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":10,"license":1,"file_name":"IP000000378_jpg.rf.a2d06efb45af1b0fd7d3dc56c8d34e20.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":11,"license":1,"file_name":"IP001000390_jpg.rf.dfe752a3d5e42ae05ae87c4c5a7b0f0a.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":12,"license":1,"file_name":"IP000000134_jpg.rf.df0dd9906e7a8190b572c1ffc6d9c7fc.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":13,"license":1,"file_name":"IP001000070_jpg.rf.51a596f95247ded8d76830faf6cbcef6.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":14,"license":1,"file_name":"IP000000212_jpg.rf.f2490939f1b149abe8c8f617c48056a9.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"}],"annotations":[{"id":0,"image_id":0,"category_id":1,"bbox":[76,14,215.5,461],"area":99345.5,"segmentation":[],"iscrowd":0},{"id":1,"image_id":0,"category_id":1,"bbox":[344,0,269.5,469.5],"area":126530.25,"segmentation":[],"iscrowd":0},{"id":2,"image_id":1,"category_id":2,"bbox":[63,126,526.5,467],"area":245875.5,"segmentation":[],"iscrowd":0},{"id":3,"image_id":2,"category_id":2,"bbox":[216,73,102.5,427.5],"area":43818.75,"segmentation":[],"iscrowd":0},{"id":4,"image_id":3,"category_id":1,"bbox":[0,0,626,561],"area":351186,"segmentation":[],"iscrowd":0},{"id":5,"image_id":4,"category_id":1,"bbox":[29,208,575.5,271],"area":155960.5,"segmentation":[],"iscrowd":0},{"id":6,"image_id":5,"category_id":1,"bbox":[108,96,378,519],"area":196182,"segmentation":[],"iscrowd":0},{"id":7,"image_id":6,"category_id":1,"bbox":[12,137,609.5,401.5],"area":244714.25,"segmentation":[],"iscrowd":0},{"id":8,"image_id":7,"category_id":2,"bbox":[74,112,484,345.5],"area":167222,"segmentation":[],"iscrowd":0},{"id":9,"image_id":8,"category_id":2,"bbox":[40,150,534.5,416],"area":222352,"segmentation":[],"iscrowd":0},{"id":10,"image_id":9,"category_id":1,"bbox":[46,207,533,262.5],"area":139912.5,"segmentation":[],"iscrowd":0},{"id":11,"image_id":10,"category_id":2,"bbox":[174,290,435,128],"area":55680,"segmentation":[],"iscrowd":0},{"id":12,"image_id":11,"category_id":1,"bbox":[233,97,192,248.5],"area":47712,"segmentation":[],"iscrowd":0},{"id":13,"image_id":11,"category_id":1,"bbox":[221,430,142.5,145],"area":20662.5,"segmentation":[],"iscrowd":0},{"id":14,"image_id":12,"category_id":2,"bbox":[64,348,458,204.5],"area":93661,"segmentation":[],"iscrowd":0},{"id":15,"image_id":13,"category_id":1,"bbox":[161,73,322.5,552],"area":178020,"segmentation":[],"iscrowd":0},{"id":16,"image_id":14,"category_id":2,"bbox":[103,27,422,528],"area":222816,"segmentation":[],"iscrowd":0}]}

Additional

No response

Black zones has black text in count box

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

While using the black color for zone annotator, the number of objects detected in written with black text inside the black box.

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

How to flash the zones when an object is detected in the zone?

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Use Case:
There are zones in a mall which have restricted access. People can walk around the zone. I want to flash the zone(not annotating it earlier) as soon as a person walks right into it.

Methodology: The zone should get annotated as soon as an object is detected inside the zone defined by the polygon method.

Additional

No response

Better handling of wrong data types of mask for sv.PolygonZone()

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

OpenCV errors out with an uninformative error when the dtype of the mask is wrong.

Would be good to add catch if dtype != np.int32 and raise an Exception.

Use case

I was trying to create a mask in CVAT, the exported segmentation had decimal places, np.array(coords) did not convert to np.int32, and I was confused by the bad error message.

This is likely to happen to other users too .

Additional

Happy to submit PR

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Live Inference using YoloV8

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

The argument show=True give the live inference in YoloV8 with it own bounding boxes.
How to get the live prediction in a window in similar way using supervision with the custom bounding boxes and zones.

Additional

No response

Locale breaking "!" in zero-shot-object-detection-with-grounding-dino.ipynb

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

As of 2023-04-06, there is a somewhat obscure problem in

https://github.com/roboflow-ai/notebooks/blob/main/notebooks/zero-shot-object-detection-with-grounding-dino.ipynb

If you go back to load new images after running the demo, you get a locale related exception. System calls require an UTF-8 locale, and it is changed to something else on the way, I do not know where.

The locale changes from UTF-8 at some point, I think after running the demo. After this, calls to the system do not longer work. they fail like this:

!wget -q https://media.roboflow.com/notebooks/examples/dog.jpeg

NotImplementedError                       Traceback (most recent call last)

[<ipython-input-117-bbf546bacccd>](https://localhost:8080/#) in <cell line: 1>()
----> 1 get_ipython().system('wget -q https://media.roboflow.com/notebooks/examples/dog.jpeg')

2 frames

[/usr/local/lib/python3.9/dist-packages/google/colab/_system_commands.py](https://localhost:8080/#) in _run_command(cmd, clear_streamed_output)
    165   locale_encoding = locale.getpreferredencoding()
    166   if locale_encoding != _ENCODING:
--> 167     raise NotImplementedError(
    168         'A UTF-8 locale is required. Got {}'.format(locale_encoding)
    169     )

NotImplementedError: A UTF-8 locale is required. Got ANSI_X3.4-1968

So
"!wget ..."
does not work.
My workaround was replacing "!":
python:
# Run the command with a specific encoding (UTF-8)
def shell(command):
print(subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, text=True, encoding="UTF-8"))

After running that in a new cell, this works:
%cd {HOME}/data
shell("wget ...")
works.

I think the problem has no effect when running the notebook cells in order, but it does when goung back to load more example images with wget.

Environment

Colab

Minimal Reproducible Example

Run the notebook up to the demo, then go back to the cell with wget and try to load more example images.

!wget -q https://media.roboflow.com/notebooks/examples/dog.jpeg

NotImplementedError                       Traceback (most recent call last)

[<ipython-input-117-bbf546bacccd>](https://localhost:8080/#) in <cell line: 1>()
----> 1 get_ipython().system('wget -q https://media.roboflow.com/notebooks/examples/dog.jpeg')

2 frames

[/usr/local/lib/python3.9/dist-packages/google/colab/_system_commands.py](https://localhost:8080/#) in _run_command(cmd, clear_streamed_output)
    165   locale_encoding = locale.getpreferredencoding()
    166   if locale_encoding != _ENCODING:
--> 167     raise NotImplementedError(
    168         'A UTF-8 locale is required. Got {}'.format(locale_encoding)
    169     )

NotImplementedError: A UTF-8 locale is required. Got ANSI_X3.4-1968

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Using Unique id and class to count objects

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

In all of the implementations I saw, we are using a line or a polygon to keep track of the count of objects. But let say in case of traffic surveillance for a intersection where we have to keep count of all the vehicles(possibly a counter for each category of vehicle).
Then how can we count all the vehicles belonging to different categories?

Additional

If we have unique id for each object detected and we are also detecting the class it belongs to, then can we not just use the unique id to keep count of each object along with the class it belongs to?