nihui / ncnn-small-board

ncnn benchmark on various single board computers

ncnn 小板子

ncnn is a high-performance neural network inference framework.

See https://github.com/Tencent/ncnn for more info about ncnn.

• The benchmark score is the inference time, less is better.
• Runs on all cpu big cores or vulkan, and fp16 arithmetic is enabled if supported.
• Table contents are ordered by mobilenet.
• rpi5b data from Tencent/ncnn#5058

small-board	image	spec	squeezenet	mobilenet	shufflenet
NVIDIA Jetson AGX Orin		Ampere 1.3GHz x 2048 Tensor Core x 64	2.13	2.03	2.57
NVIDIA Jetson AGX Orin		A78AE 2.2GHz x 12	3.50	3.49	5.08
Radxa Rock5B		RK3588 A76 2.4GHz x 4 A55 1.8GHz x 4	3.65	5.41	3.37
Orange Pi 5		RK3588S A76 2.4GHz x 4 A55 1.8GHz x 4	3.83	5.75	3.57
ZYSJ RK3588		ARM Mali-G610	7.09	9.16	5.88
ZYSJ RK3588		RK3588 A76 2.4GHz x 4 A55 1.8GHz x 4	7.57	11.01	7.95
Raspberry Pi 5B		BCM2712 A76 2.4GHz x 4	8.56	11.32	4.56
M4N DOCK		AX650 A55 1.7Ghz x 8	10.61	11.86	10.64
NVIDIA Jetson Nano		Maxwell 921MHz x 128	9.03	15.39	10.61
Khadas VIM4		ARM Mali-G52 800MHz x MP8(8EE)	12.77	20.17	12.81
Unisoc Tiger T710		Unisoc Tiger T710 A75 1.82GHz x 4 A55 1.82GHz x 4	16.25	20.31	10.59
Radxa Rock3A		RK3568 A55 2.0GHz x 4	23.20	30.78	19.49
Khadas VIM3		ARM Mali-G52 800MHz x MP4(6EE)	19.54	31.62	17.91
Panther X2		RK3566 A55 1.8GHz x 4	25.91	33.17	21.81
NanoPi M4		ARM Mali-T864	24.57	35.86	33.90
Radxa RockPi X		Intel HD Graphics (Cherry Trail) 500 MHz	29.91	36.41	24.54
Khadas VIM4		A311D2 A73 2.2GHz x 4 A53 2.0GHz x 2	24.27	37.65	19.79
NVIDIA Jetson Nano		A57 1.43GHz x 4	28.55	40.25	19.44
Khadas VIM3		A311D A73 2.2GHz x 4 A53 1.8GHz x 2	30.98	42.57	21.62
Lichee Pi 4A		T-Head TH1520 RV64GCV 2.0GHz x 4	42.29	47.73	120.13
NanoPi Fire3		S5P6818 A53 1.4GHz x 8	41.81	52.33	33.54
Debix Model A		i.MX8M Plus A53 1.6GHz x 4	40.15	54.01	29.33
Odroid XU4		Exynos 5422 A15 2GHz x 4 A7 1.3GHz x 4	35.73	54.17	25.75
Tinker Board S R2.0		ARM Mali-T764	41.48	60.28	55.87
Radxa Zero		S905Y2 A53 1.8GHz x 4	46.82	60.49	35.18
Raspberry Pi 4B		BCM2711B0 A72 1.5GHz x 4	46.28	60.74	32.91
NanoPi M4		RK3399 A72 1.8GHz x 2 A53 1.5GHz x 4	43.73	64.28	27.43
BeagleBone AI 64		TDA4VM A72 2.0GHz x 2	42.23	65.78	29.97
NanoPi K2		S905 A53 1.5GHz x 4	56.96	75.25	38.70
Radxa RockPi X		Atom x5-Z8350 x86-64 1.92 GHz x 4	50.22	80.12	37.96
Tinker Board S R2.0		RK3288CGW A17 1.8GHz x 4	52.86	82.19	38.17
NanoPi R2S		RK3328 A53 1.3GHz x 4	62.20	82.88	52.34
EAIDK 310		RK3228H A53 1.3GHz x 4	61.87	83.88	45.83
Radxa Zero		ARM Mali-G31 850 MHz x 2	57.95	98.62	51.71
Raspberry Pi 3B+		BCM2837B0 A53 1.4GHz x 4	84.74	107.84	58.38
Banana Pi M3		AllWinner A83T A7 1.8GHz x 8	88.35	109.72	80.91
MangoPI MQ-Quad		AllWinner H616 A53 1.5GHz x 4	99.60	129.70	66.90
Raspberry Pi Zero 2W		BCM2710A1 A53 1.0GHz x 4	119.52	162.60	72.72
EASY EAI Nano		RV1126 A7 1.5GHz x 4	105.16	163.82	68.79
Radxa RockPi S		RK3308 A35 1.3GHz x 4	142.02	214.56	102.44
StarFive VisionFive V2		SiFive’s U74 RV64GFC 1.5GHz x 4	150.07	256.21	91.14
RVB-ICE		T-Head C910 RV64GC 1.2GHz x 2	161.10	274.39	118.61
Loongson Pie Lite		Loongson 2K1000 GS264 1.0GHz x 2	185.28	276.03	124.39
Ingenic X2000		Ingenic X2000 XBurst2 1.2GHz x 2	175.99	282.25	118.4
Raspberry Pi 2B		BCM2836 A7 900MHz x 4	179.50	296.97	132.49
Banana Pi M2 Zero 2		AllWinner H2+ A7 1.2GHz x 4	218.73	303.92	138.43
Raspberry Pi 4B		Broadcom VideoCore VI	286.71	365.04	162.14
PICO-PI-IMX7		i.MX7D A7 1.0GHz x 2	220.10	366.92	137.02
Microphase Z7-Lite		XC7Z020CLG400 A9 766MHz x 2	389.18	623.71	212.20
MangoPI MQ-Pro		D1 C906 RV64GCV 1GHz x 1	385.21	691.47	487.67
NeZha D1		D1 C906 RV64GCV 1GHz x 1	396.18	727.97	495.87
StarFive VisionFive V1		SiFive’s U74 RV64GFC 1.5GHz x 2	609.41	993.87	358.98
Raspberry Pi B+		ARM1176JZF-S 700MHz x 1	4285.24	7684.05	2102.22

ncnn-small-board's Issues

The time of model inference is based on how large the image is tested.

The time of model inference is based on how large the image is tested.

bench？

up的方案是vk fp16 cpu 谁有就优先跑哪个吗

ROCK pi N10 benchmark ?

The running time with vulkan in jetson nano is much slower than yours

Trying to run benchncnn in jetson nano (Jetpack 4.6) with vulkan.

ncnn version is latest.

But the running time is much slower than yours

Here are my result

Any idea?