feigechuanshu / ncnn-android-yolox

Real time yolox Android demo by ncnn

The yolox object detection

This is a sample ncnn android project, it depends on ncnn library and opencv

https://github.com/Tencent/ncnn

https://github.com/nihui/opencv-mobile

how to build and run

step1

https://github.com/Tencent/ncnn/releases

• Download ncnn-YYYYMMDD-android-vulkan.zip or build ncnn for android yourself
• Extract ncnn-YYYYMMDD-android-vulkan.zip into app/src/main/jni and change the ncnn_DIR path to yours in app/src/main/jni/CMakeLists.txt

step2

https://github.com/nihui/opencv-mobile

• Download opencv-mobile-XYZ-android.zip
• Extract opencv-mobile-XYZ-android.zip into app/src/main/jni and change the OpenCV_DIR path to yours in app/src/main/jni/CMakeLists.txt

step3

• Open this project with Android Studio, build it and enjoy!

some notes

• Android ndk camera is used for best efficiency
• Crash may happen on very old devices for lacking HAL3 camera interface
• All models are manually modified to accept dynamic input shape
• Most small models run slower on GPU than on CPU, this is common
• FPS may be lower in dark environment because of longer camera exposure time

screenshot

reference

https://github.com/nihui/ncnn-android-nanodet
https://github.com/Megvii-BaseDetection/YOLOX

ncnn-android-yolox's Issues

Question about chage model

if I want to increase yolox-s in this project,what should I do ??

attention：the version of opencv and ncnn

导入AS之后可能出现的错误

导入AS之后可能出现的错误

ndk版本问题，建议使用ndk21,

cmake版本的问题，我使用3.22.0出错了，建议按原工程使用的3.10.2版本；

where is tnn code ?

Can you pls add a tutorial on how to add a custom trained yolox (tiny/nano) model to the app?

Can you pls add a tutorial on how to add a custom trained yolox (tiny/nano) model to the app? Thanks.

Still cannot run custom training model in this Project.

I, i have an issue about using my trainning model.

I am following this issue in:
#1 #3 #8

But when running app in android studio, the app crashed. Change back your model in to the app, it worked nomarlly.
My onnx model can run infer normal, so i think that my ncnn model has an issue or the app does not run my model correctly.
I am stuck at this for some days, so please give me a hand.

Many thanks.

ninja: build stopped: subcommand failed.

Build command failed.
Error while executing process D:\android with arguments {skd\cmake\3.10.2.4988404\bin\ninja.exe -C F:\ncnn-android-yolox-main\ncnn-android-yolox-main\app.cxx\cmake\debug\arm64-v8a ncnnyolox}
ninja: Entering directory `F:\ncnn-android-yolox-main\ncnn-android-yolox-main\app.cxx\cmake\debug\arm64-v8a'
[1/1] Linking CXX shared library F:\ncnn-android-yolox-main\ncnn-android-yolox-main\app\build\intermediates\cmake\debug\obj\arm64-v8a\libncnnyolox.so
FAILED: F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/build/intermediates/cmake/debug/obj/arm64-v8a/libncnnyolox.so
cmd.exe /C "cd . && "D:\android skd\ndk\20.1.5948944\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe" --target=aarch64-none-linux-android24 --gcc-toolchain="D:/android skd/ndk/20.1.5948944/toolchains/llvm/prebuilt/windows-x86_64" --sysroot="D:/android skd/ndk/20.1.5948944/toolchains/llvm/prebuilt/windows-x86_64/sysroot" -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -O0 -fno-limit-debug-info -fno-experimental-isel -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -shared -Wl,-soname,libncnnyolox.so -o F:\ncnn-android-yolox-main\ncnn-android-yolox-main\app\build\intermediates\cmake\debug\obj\arm64-v8a\libncnnyolox.so CMakeFiles/ncnnyolox.dir/yoloxncnn.cpp.o CMakeFiles/ncnnyolox.dir/yolox.cpp.o CMakeFiles/ncnnyolox.dir/ndkcamera.cpp.o F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/ncnn-20221128-android-vulkan/arm64-v8a/lib/libncnn.a F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/opencv-mobile-4.6.0-android/sdk/native/staticlibs/arm64-v8a/libopencv_core.a F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/opencv-mobile-4.6.0-android/sdk/native/staticlibs/arm64-v8a/libopencv_imgproc.a -lcamera2ndk -lmediandk -fopenmp -static-openmp -Wl,-wrap,__kmp_affinity_determine_capable "D:/android skd/ndk/20.1.5948944/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/lib/aarch64-linux-android/24/libvulkan.so" F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/ncnn-20221128-android-vulkan/arm64-v8a/lib/libglslang.a F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/ncnn-20221128-android-vulkan/arm64-v8a/lib/libSPIRV.a F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/ncnn-20221128-android-vulkan/arm64-v8a/lib/libMachineIndependent.a F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/ncnn-20221128-android-vulkan/arm64-v8a/lib/libOGLCompiler.a F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/ncnn-20221128-android-vulkan/arm64-v8a/lib/libOSDependent.a -pthread F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/ncnn-20221128-android-vulkan/arm64-v8a/lib/libGenericCodeGen.a -landroid -ljnigraphics F:/ncnn-android-yolox-main/ncnn-android-yolox-main/app/src/main/jni/opencv-mobile-4.6.0-android/sdk/native/staticlibs/arm64-v8a/libopencv_core.a -ldl -lm -llog -latomic -lm && cd ."
clang++: error: unknown argument: '-static-openmp'
ninja: build stopped: subcommand failed.
我在下手机时候 出现了这个问题 请问 是什么原因呢，求大佬解答

自己的数据集训练，屏幕出现很多检测框

我用自己的数据集进行训练，测试过pth模型和转的onnx，都可以正常检测，但是编译ncnn后会出现很多框，模型类别为3类
param

检测效果

如何在rk3399运行

我尝试步骤，但却不能在rk3399 - mtb903运行，系统版本是Android 7.1.2，AS日志都没有，我猜测是接口不对，但是不知道怎么解决，求助~~

如何调用PC外接的摄像头实现实时检测的？

你好，我是一个AS工程的新手 当我将工程构件好后，运行是这个样子的，是我的某一步操作有误吗，我该如何调用我PC的外接摄像头呢，谢谢您

Can't get custom network to perform in ncnn

Hi,

1. I train a custom yolox-s network
2. It works fine when testing with .\tools\demo.py
3. I export the onnx model using simplify
4. Convert to ncnn using onnx2ncnn
5. I follow all instructions to replace the head of the .param file

original:
7767517
235 268
Input images 0 1 images
Split splitncnn_input0 1 4 images images_splitncnn_0 images_splitncnn_1 images_splitncnn_2 images_splitncnn_3
Crop Slice_4 1 1 images_splitncnn_3 467 -23309=1,0 -23310=1,2147483647 -23311=1,1
Crop Slice_9 1 1 467 472 -23309=1,0 -23310=1,2147483647 -23311=1,2
Crop Slice_14 1 1 images_splitncnn_2 477 -23309=1,0 -23310=1,2147483647 -23311=1,1
Crop Slice_19 1 1 477 482 -23309=1,1 -23310=1,2147483647 -23311=1,2
Crop Slice_24 1 1 images_splitncnn_1 487 -23309=1,1 -23310=1,2147483647 -23311=1,1
Crop Slice_29 1 1 487 492 -23309=1,0 -23310=1,2147483647 -23311=1,2
Crop Slice_34 1 1 images_splitncnn_0 497 -23309=1,1 -23310=1,2147483647 -23311=1,1
Crop Slice_39 1 1 497 502 -23309=1,1 -23310=1,2147483647 -23311=1,2
Concat Concat_40 4 1 472 492 482 502 503 0=0
Convolution Conv_41 1 1 503 877 0=32 1=3 11=3 2=1 12=1 3=1 13=1 4=1 14=1 15=1 16=1 5=1 6=3456

Modified:
7767517
226 268
Input images 0 1 images
Input focus 1 1 images 503
Convolution Conv_41 1 1 503 877 0=32 1=3 11=3 2=1 12=1 3=1 13=1 4=1 14=1 15=1 16=1 5=1 6=3456

5. Then using ncnnoptimize with fp16 switch 65536
6. Replace second dummy input with YoloV5Focus
   7767517
   220 250
   Input images 0 1 images
   YoloV5Focus focus 1 1 images 503
   Convolution Conv_41 1 1 503 877 0=32 1=3 4=1 5=1 6=3456
   ...

When doing this with yolox-tiny I end up with the exact same network as the provided demo one except the number of classes in the output which is 191 in my case.

Now everything looks fine but the network does not perform at all - like not working at all. For training of the final network I use 640x640 images and this is used in training and also set in the app. I also tested tiny networks with 416x416 but always the same result.

The environment I train and test is on Windows using onnx 1.10.1, onnxruntime 1.9.0, onnx-simplifier 0.3.6. Using yolox 0.1.0 and latest ncnn.
I have tested almost everything over weeks now and just can't find what the issue could be.
I also looked at other posts where they suggested to use onnx version 1.8.1, onnxruntime 1.8.0 and onnx-simplifier 0.3.5 - but also having this set in an environment it is the same result as well as doing the conversion on a linux box.
Followed instructions here https://zhuanlan.zhihu.com/p/391788686 and others but always end up with the same non-performing network in ncnn.

Please let me know what I'm missing. I'm aware that this is not an issue with this project.

在on_image_render函数下对图像rgb进行灰度化，结果会导致手机上闪退

具体代码

void MyNdkCamera::on_image_render(cv::Mat& rgb) const
{
    if (rgb.empty())
        return;
    cv::Mat src = rgb.clone();
    cv::Mat matGray;
    cv::cvtColor(src, matGray, CV_RGB2GRAY);
    {
        ncnn::MutexLockGuard g(lock);
        if (g_yolox)
        {
            std::vector<Object> objects;
            g_yolox->detect(rgb, objects);
            g_yolox->draw(rgb, objects);
        }
        else
        {
            draw_unsupported(rgb);
        }
    }
    draw_fps(rgb);
}

进行debug，具体报错

`

E/cv::error(): OpenCV(4.5.3) Error: Requested object was not found (could not open directory: /data/app/com.tencent.ncnnyolox-tvY0PkV6XrQyBx4J6SbZxw==/base.apk!/lib/arm64-v8a) in glob_rec, file /home/runner/work/opencv-mobile/opencv-mobile/opencv-4.5.3/modules/core/src/glob.cpp, line 273
A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 975 (ImageReader-640), pid 32479 (ncent.ncnnyolox)

`

Android SDK

最低版本是24吗？

W/ncnn: AAssetManager_open yolox-nano.param failed

自己训练的yolox_nano模型，编译没有问题，但是run的时候就会显示
W/ncnn: AAssetManager_open yolox-nano.param failed W/ncnn: AAssetManager_open yolox-nano.bin failed
对比了能运行的.param文件与我自己的.param文件，是没差的啊

as 在debug模式下 会自动断到这 然后重复循环， 放开后会直接程序崩溃

自动断点的代码位置

报错如下图

Hello

Hello， I am new in CV and I am very interested in this project，can I add your Wechat？ I want to learn more，and communicate more with you! Thank you!

ncnn-20210720-android-vulkan 编译出错

下载ncnn-YYYYMMDD-android-vulkan.zip和下载opencv-mobile-XYZ-android.zip在ncnn-android-yolox项目中，编译后，opencv没问题，就是ncnn报错

cmake的设置如此，几乎没有改动，出现这个错误，找不到ncnn的线程，文件的目录如下，要改改，麻烦告知，谢谢

在on_image_render函数下增加对输入图像rgb进行灰度化，结果会导致手机上闪退

具体代码
`void MyNdkCamera::on_image_render(cv::Mat& rgb) const
{
if (rgb.empty())
return;
cv::Mat src = rgb.clone();
cv::Mat matGray;
cv::cvtColor(src, matGray, CV_RGB2GRAY);
// nanodet
{
ncnn::MutexLockGuard g(lock);

    if (g_yolox)
    {
        std::vector<Object> objects;
        g_yolox->detect(rgb, objects);

        g_yolox->draw(rgb, objects);
    }
    else
    {
        draw_unsupported(rgb);
    }
}

draw_fps(rgb);

}进行debug，具体报错E/cv::error(): OpenCV(4.5.3) Error: Requested object was not found (could not open directory: /data/app/com.tencent.ncnnyolox-tvY0PkV6XrQyBx4J6SbZxw==/base.apk!/lib/arm64-v8a) in glob_rec, file /home/runner/work/opencv-mobile/opencv-mobile/opencv-4.5.3/modules/core/src/glob.cpp, line 273
A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 975 (ImageReader-640), pid 32479 (ncent.ncnnyolox)`

Question about ncnnoptimize model.param model.bin yolox.param yolox.bin 65536

In:
https://github.com/Megvii-BaseDetection/YOLOX/tree/main/demo/ncnn/cpp

Step 5:

suppose you are still under ncnn/build/tools/ncnn dir.
../ncnnoptimize model.param model.bin yolox.param yolox.bin 65536

Suppose I trained a new customed model, how to adjust parameter (65536)?
Thanks a lot.

训练自己数据集后,生成onnx文件,转ncnn部署到手机上没有检测出目标

用官方提供的没有问题,自己训练数据集没有更改网络信息也没有检测框

Is this project support latest yolox??

I try to use latest yolox(version 0.1.0) in this project, I modified the corresponding parameters, when I run, the program force quit

只显示视频没有目标检测

你好，我使用YoloX官方的ONNX模型转成NCNN的文件，生成的param和bin文件大小完全一致，对比parame里面的内容也完全一致，替换后APP正常运行但是没有进行检测。请问这是什么问题呢？ 非常感谢~

YOLOX sluggish on GPU

Hey, I've a problem with GPU on newer devices like s20, poco x3 pro, etc. models: nano, tiny and s are faster on a CPU than on the GPU. On GPU models work slower (about 30-40%) and sometimes get sluggish (droping to 3 fps when on CPU I've got 15).

生成的Android demo出现了闪退

先生您好,如题,毫无修改的使用您的代码可以成功release出apk包,但是一打开就出现了闪退,请问是什么原因呢?

你好，修改网络结构之后还能继续用这个么？

你好，我想问一下，对yolox的网络结构进行修改后，还能继续用这个进行部署到移动端么？感谢答复