Does iPhoneOS armv7 support neon dot and neon v8? about xnnpack

[Maratyszcza]

I imagine the neon_dot build error is the same as tensorflow/tensorflow#48464, but what is failing in neon_v8 microkernels? It is plausible that Apple toolchain for ARMv7 doesn't support neon_dot because the only Apple SoCs that support it are ARM64-only, but IMO neon_v8 should still compile fine, at there are AArch32-compatible Apple SoCs with ARMv8.

[Maratyszcza]

The fix for NEONDOT microkernels is in #1404

[Maratyszcza]

@larryliu0820 Bazel build use

XNNPACK/BUILD.bazel

Lines 3971 to 3974 in 6e8c0ce

	apple_aarch32_copts = [
	"-mcpu=cyclone",
	"-mtune=generic",
	],

for NEONV8 microkernels. Could you try these options with CMake?

[Maratyszcza]

Good news is

 apple_aarch32_copts = [ 
     "-mcpu=cyclone", 
     "-mtune=generic", 
 ],

Fixes issue for neonv8 microkernels.

Would you create a PR?

[Maratyszcza]

No, this code is in AArch32-specific section

[Maratyszcza]

Currently, there is no way to exclude neon_dot and neon_v8 microkernels from the build. I'm looking into a possible solution.

[larryliu0820]

Are armv8 SoCs passing CMakeLists.txt check: IOS_ARCH MATCHES "^armv7" ?

[Maratyszcza]

I'd expect so. "armv7" really means AArch32 with ARMv7 or higher instruction set.

[larryliu0820]

"armv7" really means AArch32 with ARMv7 or higher instruction set.

armv7 is failing arm_neon.h:

#if __ARM_ARCH >= 8 && defined(__ARM_FEATURE_DIRECTED_ROUNDING)

hence a lot of vrndz vrndu symbols are not found

I imagine the neon_dot build error is the same as tensorflow/tensorflow#48464

The error signature looks similar. I've got:

XNNPACK/src/qs8-gemm/gen/4x16c4-minmax-neondot.c:102:18: error: assigning to 'int32x4_t' (vector of 4 'int32_t' values) from incompatible type 'int'
      vacc0x0123 = vdotq_lane_s32(vacc0x0123, vb0123x0123, va0x01234567, 0);
                 ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[Maratyszcza]

hence a lot of vrndz vrndu symbols are not found

Do you actually get compilation errors about these symbols? XNNPACK sets " -march=armv8-a -mfpu=neon-fp-armv8 " when compiling ARMv8 NEON microkernels, which should make Clang include the declarations of these intrinsics.

[larryliu0820]

Yes the compiler errors looks like this:

XNNPACK/src/f32-vrnd/gen/vrndu-neonv8-x4.c:31:23: error: initializing 'const float32x4_t' (vector of 4 'float32_t' values) with an expression of incompatible type 'int'
    const float32x4_t vy0123 = vrndpq_f32(vx0123);
                      ^        ~~~~~~~~~~~~~~~~~~

I've made sure the compiler flags are there.

[larryliu0820]

For neondot I ran into these errors after pulling latest commit:

stderr: Undefined symbols for architecture armv7:
  "_xnn_qs8_igemm_minmax_ukernel_4x8c4__neondot", referenced from:
      _init in libXNNPACKApple-2065260873.a(init.c.o)
  "_xnn_qs8_igemm_minmax_ukernel_1x8c4__neondot", referenced from:
      _init in libXNNPACKApple-2065260873.a(init.c.o)
  "_xnn_qs8_gemm_minmax_ukernel_1x8c4__neondot", referenced from:
      _init in libXNNPACKApple-2065260873.a(init.c.o)
  "_xnn_qs8_gemm_minmax_ukernel_4x8c4__neondot", referenced from:
      _init in libXNNPACKApple-2065260873.a(init.c.o)

I'm going to see if I can build with cmake

[larryliu0820]

Good news is

 apple_aarch32_copts = [ 
     "-mcpu=cyclone", 
     "-mtune=generic", 
 ],

Fixes issue for neonv8 microkernels.

[Maratyszcza]

For neondot I ran into these errors after pulling latest commit:

stderr: Undefined symbols for architecture armv7:
  "_xnn_qs8_igemm_minmax_ukernel_4x8c4__neondot", referenced from:
      _init in libXNNPACKApple-2065260873.a(init.c.o)
  "_xnn_qs8_igemm_minmax_ukernel_1x8c4__neondot", referenced from:
      _init in libXNNPACKApple-2065260873.a(init.c.o)
  "_xnn_qs8_gemm_minmax_ukernel_1x8c4__neondot", referenced from:
      _init in libXNNPACKApple-2065260873.a(init.c.o)
  "_xnn_qs8_gemm_minmax_ukernel_4x8c4__neondot", referenced from:
      _init in libXNNPACKApple-2065260873.a(init.c.o)

I'm going to see if I can build with cmake

Fix coming in #1405

[larryliu0820]

if (!XNN_PLATFORM_IOS && cpuinfo_has_arm_neon_dot()) {

Wouldn't that disable iphoneos arm64?