hsvdo / flutter_vision

A Flutter plugin for managing both Yolov5 model and Tesseract v4, accessing with TensorFlow Lite 2.x. Support object detection and OCR on both iOS and Android.

flutter_vision

A Flutter plugin for managing Yolov5, Yolov8 and Tesseract v5 accessing with TensorFlow Lite 2.x. Support object detection and OCR on Android. iOS not updated, working in progress.

Installation

Add flutter_vision as a dependency in your pubspec.yaml file.

Android

In android/app/build.gradle, add the following setting in android block.

    android{
        aaptOptions {
            noCompress 'tflite'
            noCompress 'lite'
        }
    }

iOS

Comming soon ...

Usage

For YoloV5 and YoloV8 MODEL

1. Create a assets folder and place your labels file and model file in it. In pubspec.yaml add:

  assets:
   - assets/labels.txt
   - assets/yolovx.tflite

2. Import the library:

import 'package:flutter_vision/flutter_vision.dart';

3. Initialized the flutter_vision library:

 FlutterVision vision = FlutterVision();

4. Load the model and labels: modelVersion: yolov5 or yolov8

await vision.loadYoloModel(
        labels: 'assets/labelss.txt',
        modelPath: 'assets/yolov5n.tflite',
        modelVersion: "yolov5",
        numThreads: 1,
        useGpu: false);

For camera live feed

5. Make your first detection: confThreshold work with yolov5 other case it is omited.

Make use of camera plugin

final result = await vision.yoloOnFrame(
        bytesList: cameraImage.planes.map((plane) => plane.bytes).toList(),
        imageHeight: cameraImage.height,
        imageWidth: cameraImage.width,
        iouThreshold: 0.4,
        confThreshold: 0.4,
        classThreshold: 0.5);

For static image

5. Make your first detection:

final result = await vision.yoloOnImage(
        bytesList: byte,
        imageHeight: image.height,
        imageWidth: image.width,
        iouThreshold: 0.8,
        confThreshold: 0.4,
        classThreshold: 0.7);

6. Release resources:

await vision.closeYoloModel();

For Tesseract 5.0.0 MODEL

1. Create an assets folder, then create a tessdata directory and tessdata_config.json file and place them into it. Download trained data for tesseract from here and place it into tessdata directory. Then, modifie tessdata_config.json as follow.

{
    "files": [
      "spa.traineddata"
    ]
}

2. In pubspec.yaml add:

assets:
    - assets/
    - assets/tessdata/

3. Import the library:

import 'package:flutter_vision/flutter_vision.dart';

4. Initialized the flutter_vision library:

 FlutterVision vision = FlutterVision();

5. Load the model:

await vision.loadTesseractModel(
      args: {
        'psm': '11',
        'oem': '1',
        'preserve_interword_spaces': '1',
      },
      language: 'spa',
    );

For static image

6. Get Text from static image:

    final XFile? photo = await picker.pickImage(source: ImageSource.gallery);
    if (photo != null) {
      final result = await vision.tesseractOnImage(bytesList: (await photo.readAsBytes()));
    }

7. Release resources:

await vision.closeTesseractModel();

About results

For Yolo

result is a List<Map<String,dynamic>> where Map have the following keys:

   Map<String, dynamic>:{
    "box": [x1:top, y1:left, x2:bottom, y2:right, class_confidence]
    "tag": String: detected class
   }

For Yolov8 with segmentation model

result is a List<Map<String, dynamic>> where Map[0] has the same keys as normal detection Yolo result and Map[1] is represented as following:

   Map<String, dynamic>:{
    "yourLabel1": List<float[]>
    "yourLabel2": List<float[]>
   }

Where each label is mapped to its corresponding segmentation mask. As flutter method channel do not support nested arrays (e.g. float[][]) the output was converted into a List<float[]>. Each list entry is a row with the corresponding mask values. Those values are only the prototype masks and require postprocessing. Refer to ultralytics docs/github issues for further information.

For Tesseract

result is a List<Map<String,dynamic>> where Map have the following keys:

    Map<String, dynamic>:{
      "text": String
      "word_conf": List:int
      "mean_conf": int}

Example