hyuto / yolov8-seg-onnxruntime-web

YOLOv8 Segmentation Right in The Browser Using onnxruntime-web

YOLOv8 Segmentation with onnxruntime-web

---

Object Segmentation application right in your browser. Serving YOLOv8 segmentation in browser using onnxruntime-web with wasm backend.

Setup

git clone https://github.com/Hyuto/yolov8-seg-onnxruntime-web.git
cd yolov8-seg-onnxruntime-web
yarn install # Install dependencies

Scripts

yarn start # Start dev server
yarn build # Build for productions

Models

Main Model

YOLOv8n-seg model converted to onnx.

used model : yolov8n-seg.onnx
size       : 14 Mb

NMS

ONNX model to perform NMS operator [CUSTOM].

Mask

ONNX model to produce mask for every object detected [CUSTOM].

Use another model

⚠️ Size Overload : used YOLOv8 segmentation model in this repo is the smallest with size of 14 MB, so other models is definitely bigger than this which can cause memory problems on browser.

Use another YOLOv8 model.

1. Export YOLOv8 model to onnx format. Read more on the official documentation

   from ultralytics import YOLO
   
   # Load a model
   model = YOLO("yolov8*-seg.pt")  # load an official yolov8* model
   
   # Export the model
   model.export(format="onnx")

2. Copy yolov8*.onnx to ./public/model

3. Update modelName in App.jsx to new model name

   ...
   // configs
   const modelName = "yolov8*-seg.onnx";
   const modelInputShape = [1, 3, 640, 640];
   const topk = 100;
   const iouThreshold = 0.45;
   const scoreThreshold = 0.2;
   ...

4. Done! 😊

Note: Custom Trained YOLOv8 Segmentation Models

Please update src/utils/labels.json with your YOLOv8 Segmentation classes.

Reference

• https://github.com/ultralytics/ultralytics
• https://github.com/Hyuto/yolov8-onnxruntime-web
• https://github.com/Hyuto/yolov5-seg-onnxruntime-web

yolov8-seg-onnxruntime-web's Issues

Get pixel coordinates

Hello i work your project, it works, thanks but i cant get pixel coordinates.
My code on colab is below

from ultralytics import YOLO

# Load a pretrained YOLOv8n model
model = YOLO('yolov8x-seg.pt')

# Run inference on 'bus.jpg'
results = model('../images/31_set1-middle.jpg')  # results list

# Show the results
for r in results:
    print(r.masks.xy)
    # print(r.boxes)
    im_array = r.plot()  # plot a BGR numpy array of predictions
    im = Image.fromarray(im_array[..., ::-1])  # RGB PIL image
    im.show()  # show image
    im.save('results-seg-x-middle.jpg')  # save image

In this code i take detected object's pixel coordinates that present border but how can i take in your project these coordinates?
Can you help me?

Update config used in detect.js

Hello, i added my trained model's export file in this app but it threw an error, i think the reason is the config and i dont know the meaning of 'topk' in config, can you explain to me?

    "float32",
    new Float32Array([
      numClass, // num class
      topk, // topk per class
      iouThreshold, // iou threshold
      scoreThreshold, // score threshold
    ])
  ); // nms config tensor

This is the error message:

Uncaught (in promise) Error: input 'images' is missing in 'feeds'.
    run inference-session-impl.ts:95
    detectImage detect.js:48

the 48. line in detect.js is
const { selected } = await session.nms.run({ detection: output0, config: config }); // perform nms and filter boxes

yolov8x-seg.onnx can not work

yolov8x-seg.onnx can not work

custom pretrained model

latest custom pretrained model (2 classes) not workin deh even after updated modelname and label
compare onnx on netron.app mine yours different
did u try custom model?