arm-software / ethos-n-driver-stack

Driver stack (including user space libraries, kernel module and firmware) for the Arm® Ethos™-N NPU

Arm® Ethos™-N Driver Stack

About the Arm Ethos-N Neural Processing Unit (NPU)

The Arm Ethos-N NPUs improve the inference performance of neural networks. The NPUs target 8-bit integer quantized Convolutional Neural Networks (CNN).

For more information, see https://www.arm.com/solutions/artificial-intelligence.

Supported networks

• We have tested this release with the following networks:

  • VGG16
  • MobileNet v1 0.25 224
  • SSD MobileNet v1
  • InceptionV3
  • InceptionV4
  • FSRCNN
  • Yolo V3
  • ResNet v2-50
  • SRGAN
  • U-Net
  • EfficientNet Lite

  Note: Running other networks may result in parts of the network being run by the Arm NN CPU reference backend.

About the Ethos-N driver stack

The Ethos-N driver stack targets the Ethos-N78 NPU and consists of open-source and other software components.

The open-source components are:

• Arm NN: A software library that enables machine learning workloads on power efficient devices. On Linux®, applications can link directly to Arm NN. On Android™, you can use Arm NN as a backend for the Android NNAPI or applications can link directly to Arm NN.
• Arm NN Android neural network driver: Supports the Android NNAPI on the NPU. The Arm NN Android neural network driver is optional.
• Ethos-N NPU driver: Contains the user-space component of the driver.
• Ethos-N NPU kernel module: Contains the kernel space component of the driver.
• Arm NN Ethos-N NPU backend: Contains the Ethos-N NPU backend for Arm NN.

The software component that is available under an Arm proprietary license is:

• Ethos-N NPU firmware binaries file: Contains the firmware that runs on the NPU.

Arm NN and the Arm NN Android neural network driver are external downloads and links are provided in this README file. All other components are part of this driver stack release.

Target platform requirements

Your target platform must meet specific requirements to run the Ethos-N NPU driver. Your platform must have:

• An Armv8-A application processor
• An Arm Ethos-N NPU
• At least 4GB of RAM
• At least 16GB of free storage space

Secure mode and TZMP1

Depending on how the hardware has been configured, the NPU boots up in either Secure or Non-secure mode.

To use the NPU in Secure mode, the target platform must have a Trusted Firmware-A (TF-A) that is built with Arm Ethos-N NPU support.

NPU support is available in the TF-A source tree, along with a reference implementation showing how to use it on the Arm Juno platform. For instructions on how to get and build the TF-A source, see the TF-A documentation. The following commit of TF-A has been tested: 4796d2d9bb4a1c0ccaffa4f6b49dbb0f0304d1d1.

The build flag required to enable NPU support for the Arm Juno platform is available at TF-A Arm Platform Build Options.

For information about boot up in Secure or Non-secure modes, see the Arm Ethos-N78 NPU Technical Reference Manual.

For information about how to port TF-A to another platform, see the TF-A Porting Guide.

If TZMP1 support is required, the TF-A must be built with the TZMP1 build option as well as the NPU build option.

The kernel module must also be built with appropriate flags depending on the security level required.

Please refer to the following table for the supported configurations:

System security level	NPU hardware configuration	TF-A build configuration	Kernel module build configuration
Non-secure	Non-secure	No flags	Non-secure
Secure	Secure	NPU	Secure
TZMP1	Secure + SMMU + Single-core	NPU + TZMP1	TZMP1

All other combinations of configurations are not supported and may lead to errors or unexpected behaviour.

Build tools

To build the Ethos-N NPU software, some tools must be installed on the platform used for the compilation. The platform can be either your target platform or the host platform when cross compiling. We have only tested building the driver stack on the Ubuntu 20.04 LTS x86 64-bit Linux distribution. The required tools are:

• A Linux distribution. An open-source operating system.
• Git A version control system that software developers use for source code management. We recommend version 2.25.1.
• SCons An open-source software construction tool. We recommend v3.1.2.
• Make A build automation tool. We recommend version 4.2.1.
• Sparse A semantic parser for C. We recommend v0.6.3.
• GNU C and C++ and compilers Open-source tools for Arm processors. We recommend version 9.4.0.

The build platform must have at least:

• 8GB of RAM
• 4GB of free storage space

Install the build tools

You must use specific tools to build the Ethos-N NPU driver. To install these build tools, you can use a package manager. For example, to install the build tools on Ubuntu 20.04 to cross compile, enter the following commands:

sudo apt install git \
    scons \
    make \
    gcc-aarch64-linux-gnu \
    g++-aarch64-linux-gnu \
    gcc \
    g++ \
    bison \
    flex \
    libssl-dev \
    bc \
    rsync \
    python3

You must build and install sparse. We recommend v0.6.3. To build and install sparse, enter the following commands:

git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git <path_to_sparse>/sparse --branch v0.6.3
cd <path_to_sparse>/sparse
sudo make PREFIX=/usr install

Also, if you want to build unit tests for the Ethos-N NPU user-space libraries, you must install Catch2. We recommend v2.13.8. To install Catch2, enter the following commands:

git clone --depth 1 https://github.com/catchorg/Catch2.git --branch v2.13.8 <path_to_catch>/Catch2

Install the Linux source tree

The Ethos-N driver stack source code depends on the Linux source tree to build the kernel module. You must configure the kernel to build the kernel module. Arm has tested versions 5.4, and 5.10 of the Linux source tree in System Memory Management Unit (SMMU) configurations.

To configure the kernel:

1. Download version 5.4, or 5.10 of the Linux source tree from www.kernel.org.

2. Configure the memory system for the NPU:

   • If you run the NPU with an SMMU, because the SMMU driver cannot simultaneously enable the SMMU v1 or v2 and the SMMU v3, you must disable the CONFIG_ARM_SMMU configuration key and enable the CONFIG_ARM_SMMU_V3 configuration key:

     CONFIG_ARM_SMMU_V3=y
     CONFIG_ARM_SMMU=n

   • If you run the NPU without an SMMU, you must create a reserved memory area. The reserved memory area stores working data for the NPU, for example the firmware code and network data. The size of the reserved memory area depends on your specific use case and several factors. These factors include the number of NPU cores and the size of the networks being used. We recommend that you test to ensure the chosen size is suitable.

   There are restrictions on using the reserved memory area. These restrictions are detailed in Limitations.

3. How you compile the driver affects how you configure the Linux kernel source tree:

   • If you compile the driver natively, enter the following commands:

     make -C <path_to_kernel> defconfig
     make -C <path_to_kernel> modules_prepare

   • If you cross compile the driver, enter the following commands:

     make -C <path_to_kernel> ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig
     make -C <path_to_kernel> ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- modules_prepare

   Note: The <path_to_kernel> directory is where the Linux kernel tree is stored.

Build the Ethos-N driver stack

The Ethos-N driver stack is written using portable C++14 and the build system uses scons. It is therefore possible to build for a wide variety of target platforms, from a wide variety of host environments.

Download the Ethos-N driver stack

To build the Ethos-N driver stack, you must download Arm NN and the Ethos-N driver stack components. The Ethos-N driver stack download contains the Ethos-N NPU driver, kernel module, backend, and firmware binaries file.

To download the components, enter the following commands:

mkdir driver_stack
cd driver_stack
git clone https://github.com/Arm-software/armnn --branch v24.02
git clone https://github.com/Arm-software/ethos-n-driver-stack --branch 23.11

_Note: The default branch on GitHub has changed to main._

Build the Ethos-N NPU driver

You must follow specific steps to build the Ethos-N NPU driver. You must build the Ethos-N NPU driver, Ethos-N NPU kernel module, and Arm NN. Depending on your system, appropriate privileges are required to run some of the following commands:

1. Copy the <path_to>/driver_stack/ethos-n-driver-stack/firmware/ethosn.bin file into the /lib/firmware/ folder of the target system that runs the Ethos-N NPU driver.

   Note: The <path_to> directory is where the driver_stack directory is stored.

2. Build the Ethos-N NPU kernel module.

   Note: By default, the kernel module is built for an NPU running in Secure mode. If Non-secure mode is required, add the EXTRA_CCFLAGS=" -DETHOSN_NS" flag to the following make commands. If TZMP1 mode is required, add the EXTRA_CCFLAGS=" -DETHOSN_TZMP1" flag to the following make commands. If disabling all SMC calls is required when using using Non-secure mode, add the EXTRA_CCFLAGS=" -DETHOSN_NO_SMC" flag to the following make commands. Secure mode still makes use of SMC calls regardless of this flag. Please see the above Secure mode and TZMP1 section for more details.

   How you compile the driver affects how you build the Ethos-N NPU kernel module:

   • If you compile the driver natively, enter the following commands:

     cd <path_to>/driver_stack/ethos-n-driver-stack/kernel-module
     make -C <path_to_kernel> M=$PWD modules

   • If you cross compile the driver, enter the following commands:

     cd <path_to>/driver_stack/ethos-n-driver-stack/kernel-module
     make -C <path_to_kernel> M=$PWD ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- modules

   Note: The <path_to_kernel> directory is where the Linux kernel tree is stored.

   We recommend you strip unnecessary symbols from the driver:

   • If you compile the driver natively, enter the following command:

     strip --strip-unneeded <path_to>/driver_stack/ethos-n-driver-stack/kernel-module/ethosn.ko

   • If you cross compile the driver, enter the following command:

     aarch64-linux-gnu-strip --strip-unneeded <path_to>/driver_stack/ethos-n-driver-stack/kernel-module/ethosn.ko

3. Copy the kernel module <path_to>/driver_stack/ethos-n-driver-stack/kernel-module/ethosn.ko to the system that runs the Ethos-N NPU driver.

4. To load the kernel module on the target system, enter the following command:

   insmod ethosn.ko

   • The kernel module supports the following parameters:

     • firmware_log_severity : INTEGER. Default value: 3. Used to set the log level of firmware. The following log levels are supported.

       • PANIC = 0
       • ERROR = 1
       • WARNING = 2
       • INFO = 3
       • DEBUG = 4
       • VERBOSE = 5

     • firmware_log_to_kernel_log : BOOL. Default value: true. Used to enable/disable sending firmware log prints to kernel dmesg output.

     • queue_size : INTEGER. Default value: 65536 bytes. Used to set the queue size of ethosn firmware mailbox.

     • profiling: BOOL. Default value: false. Used to enable/disable firmware profiling. The profiling data is available in

     • /sys/kernel/debug/ethosn<N>/core<X>/firmware_profiling, where N is the ethosn device number and X is the core number.

     • clock_frequency: INTEGER. Default value: 1000 MHz. This paramter is used only in tests and doesn't set the clock frequency of the device.

     • stashing: BOOL. Default value: true. Used to enable/disable stashing in hardware.

5. To build the user-space libraries of the Ethos-N NPU driver:

   • If you compile the driver natively, enter the following command:

     cd <path_to>/driver_stack/ethos-n-driver-stack/driver
     scons

   • If you cross compile the driver, enter the following command:

     cd <path_to>/driver_stack/ethos-n-driver-stack/driver
     scons platform=aarch64

   To include dependencies from non-standard locations and to install files into non-standard locations, use the configuration options. To see all configuration options, enter the following command:

   scons --help

6. To install the user-space libraries of the Ethos-N NPU driver:

   • If you compile the driver natively, enter the following command:

     cd <path_to>/driver_stack/ethos-n-driver-stack/driver
     scons install_prefix=<install_directory> install

   • If you cross compile the driver, enter the following command:

     cd <path_to>/driver_stack/ethos-n-driver-stack/driver
     scons platform=aarch64 install_prefix=<install_directory> install

   Note: The <install_directory> directory is where files built from the Ethos-N NPU driver will be stored.

7. Set up the Arm NN directory structure and link the Ethos-N NPU backend to the Arm NN source tree:

   cd <path_to>/driver_stack
   mkdir armnn_build
   cd armnn_build
   mkdir source
   mkdir build
   cd ..
   mv armnn armnn_build/source
   cd <path_to>/driver_stack/armnn_build/source/armnn/src/backends
   ln -s <path_to>/driver_stack/ethos-n-driver-stack/armnn-ethos-n-backend ethos-n

8. Build Arm NN for TensorFlow Lite. For instructions about building Arm NN, see the README.md file in the armnn/build-tool directory.

   cd <path_to>/driver_stack/armnn_build
   sudo ./source/armnn/build-tool/scripts/install-packages.sh
   ./source/armnn/build-tool/scripts/setup-armnn.sh --tflite-parser --target-arch=aarch64

   The following build options are required to use the Ethos-N backend:

   -DBUILD_TESTS=1
   -DARMNNREF=1
   -DETHOSN_SUPPORT=1
   -DETHOSN_ROOT=<install_directory>

   ./source/armnn/build-tool/scripts/build-armnn.sh  --target-arch=aarch64 --tflite-parser --ref-backend --armnn-cmake-args="-DBUILD_TESTS=1,-DARMNNREF=1,-DETHOSN_SUPPORT=1,-DETHOSN_ROOT=<install_directory>"

   Running the above command will produce a compressed folder containing all the files required to use Arm NN.

Exercise the Ethos-N NPU driver

There are multiple ways to exercise the Ethos-N NPU driver:

1. Running the Arm NN Ethos-N NPU backend unit tests.

   Note: You must have built Arm NN and the Ethos-N NPU driver before running the backend unit tests.

   If you have cross compiled the driver, copy the following files onto the target platform:

   • All *.so* files built from Arm NN.
   • UnitTests built from Arm NN.
   • libEthosNSupport.so built from the Ethos-N NPU driver inside the <install_directory>/lib/ directory.
   • libEthosNDriver.so built from the Ethos-N NPU driver inside the <install_directory>/lib/ directory.

   Note: You may need to copy extra .so files depending on your toolchain and its runtime dependencies.

   Note: You may need to set the library path so that the supplied libraries can be found.

   To set the library path and run the backend unit tests for the Ethos-N NPU, enter the following command:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries>:<path_to_armnn_libs> ./UnitTests --test-suite=*Ethos*

2. Running the Ethos-N NPU driver user-space unit tests.

   Before running the user-space unit tests, you must have built the driver with testing enabled. To enable testing:

   • Add tests=1 to your scons commands to build the user-space component unit tests.
   • Make sure your CPATH scons variable, which is specified on the command-line, points to <path_to_catch>/Catch2/single_include/catch2/.
   • Make sure that your unit_test_kernel_dir scons variable, which is specified on the command-line, points to the Linux kernel source tree used to build the Ethos-N NPU Linux kernel module. The Linux kernel's user-space headers will be used when building and running the unit tests to determine what features that are supported and should be tested. Note: In order to support all the unit tests, the Linux kernel source tree used must be version 5.6 or higher. The Linux kernel source tree should be the same as the one used to compile the Linux kernel module. Note: If the Linux kernel source tree does not contain the Linux kernel's generated user-space headers, scons will generate them using the Linux kernel's headers_install make target.

   If you have cross compiled the driver, copy the following files onto the target platform:

   • UnitTests built for the Ethos-N NPU support library inside <path_to>/driver_stack/ethos-n-driver-stack/driver/support_library/build/release_<platform>/tests.
   • UnitTests built for the Ethos-N NPU command stream inside <path_to>/driver_stack/ethos-n-driver-stack/driver/support_library/command_stream/build/release_<platform>/tests.
   • UnitTests built for the Ethos-N NPU driver library inside <path_to>/driver_stack/ethos-n-driver-stack/driver/driver_library/build/release_<platform>_kmod/tests.
   • libEthosNSupport.so built from the Ethos-N NPU driver inside <install_directory>/lib/.
   • libEthosNDriver.so built from the Ethos-N NPU driver inside <install_directory>/lib/.
   • ethosn.ko built from the Ethos-N NPU driver inside <path_to>/driver_stack/ethos-n-driver-stack/kernel-module/.

   Note: You may need to copy additional .so files depending on your toolchain and its runtime dependencies.

   Note: You may need to set the library path so that the supplied libraries can be found.

   To set the library path and run the user-space unit tests for the Ethos-N NPU, enter the following command:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries> ./UnitTests

3. Run the ExecuteNetwork program provided by Arm NN. This program supports the running of TensorFlow Lite models.

   If you have cross compiled the driver, you must copy the following files onto the target platform:

   • All *.so* files built from Arm NN.
   • ExecuteNetwork built from Arm NN in the tests/ folder.
   • libEthosNSupport.so built from the Ethos-N NPU driver inside <install_directory>/lib/.
   • libEthosNDriver.so built from the Ethos-N NPU driver inside <install_directory>/lib/.

   Note: You may need to copy additional .so files depending on your toolchain and its runtime dependencies.

   The ExecuteNetwork program requires parameters passed in. To find details about these parameters, enter the following command:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries>:<path_to_armnn_libs> ./tests/ExecuteNetwork --help

   The minimum set of required parameters are:

   • -d The path to a whitespace separated list of values for the input.
   • -m The path to the model to be run.
   • -c The device you wish to run this inference on.

   An example of running mobilenet_v1_0.25_224_default_minmax.tflite on the Ethos-N NPU is:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries>:<path_to_armnn_libs> ./tests/ExecuteNetwork -f tflite-binary -i input -y qasymmu8 -o MobilenetV1/Predictions/Reshape_1 -z qasymmu8 -d input_data.txt -m mobilenet_v1_0.25_224_default_minmax.tflite -c EthosNAcc -c CpuRef

4. Run the Ethos-N NPU System Tests. This program supports the running of TensorFlow Lite models, networks described in Ggf format, and contains a suite of built-in tests.

   Arm NN is required to build the system tests.

   To build the system-tests executable, use the following command:

    ```sh
    cd <path_to>/driver_stack/ethos-n-driver-stack/driver
    scons tests=1 ../tools/system_tests LPATH=`<path_to>/driver_stack/armnn_build/build/armnn/x86_64_build/` CPATH=`<path_to_catch>/Catch2/single_include/catch2/:<path_to>/driver_stack/armnn_build/source/armnn/include/`
    ```
   

   • If you are cross compiling the system tests, add platform=aarch64 to the scons command and change x86_64_build to aarch64_build
   • Make sure that your unit_test_kernel_dir scons variable, which is specified on the command-line, points to the Linux kernel source tree used to build the Ethos-N NPU Linux kernel module. The Linux kernel's user-space headers will be used when building and running the unit tests to determine what features are supported and should be tested. Note: In order to support all the unit tests, the Linux kernel source tree used must be version 5.6 or higher. The Linux kernel source tree should be the same as the one used to compile the Linux kernel module. Note: If the Linux kernel source tree does not contain the Linux kernel's generated user-space headers, scons will generate them using the Linux kernel's headers_install make target.

   This generates an executable for system tests and system test unit tests:

   • system-tests in <path_to>/driver_stack/ethos-n-driver-stack/tools/system_tests/build/release_<platform>/
   • UnitTests in <path_to>/driver_stack/ethos-n-driver-stack/tools/system_tests/build/release_<platform>/tests

   If you have cross compiled the driver, you must copy the following files onto the target platform:

   • All *.so* files built from Arm NN.
   • system-tests and UnitTests from the Ethos-N NPU driver mentioned above.
   • libEthosNSupport.so built from the Ethos-N NPU driver inside <install_directory>/lib/.
   • libEthosNDriver.so built from the Ethos-N NPU driver inside <install_directory>/lib/.

   The system-tests program requires parameters passed in. To find details about these parameters, enter the following command:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries>:<path_to_armnn_libs> ./system-tests -?

   Note: As system tests is built using the Catch2 framework, this will also display the built-in options for the Catch2 framework.

   To run networks using system tests, you can use TfLiteRunner which accepts TensorFlow Lite models, or GgfRunner which accepts networks described in a text format. For examples of how .ggf files are laid out, see <path_to>/driver_stack/ethos-n-driver-stack/tools/system_tests/graphs. For more information, check GgfParser.cpp.

   An example of using TfLiteRunner to run mobilenet_v1_0.25_224_default_minmax.tflite in system tests is:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries>:<path_to_armnn_libs> ./system-tests TfLiteRunner --tflite-file mobilenet_v1_0.25_224_default_minmax.tflite

   An example of using GgfRunner to run conv1x1_nhwc.ggf in system tests is:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries>:<path_to_armnn_libs> ./system-tests GgfRunner --ggf-file conv1x1_nhwc.ggf
   

   Note: TfLiteRunner and GgfRunner compare the inference result against a reference result generated using Arm NN with the CpuRef backend. To skip this check, use the --skip-ref option.

   To run other tests within system tests, use the following command:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries>:<path_to_armnn_libs> ./system-tests <test_case>
   

   If <test_case> is empty, system tests will run every test.

   Note: Some test cases are dependant on specific Ethos-N NPU configurations to run successfully.

   To run the unit tests for system tests, use the following command:

   LD_LIBRARY_PATH=<path_to_ethosn_libraries>:<path_to_armnn_libs> ./UnitTests <test_case>
   

Power Management

To allow efficient power usage, the Ethos-N NPU supports the following power management features:

• Suspend: Puts the NPU in a low-power state. The state of the NPU is stored in RAM. This feature is sometimes referred to as 'suspend to RAM'.
• Hibernate: Puts the NPU in a low-power state. The state of the NPU is saved to disk. This feature is sometimes referred to as 'suspend to disk'.
• Sleep: This is a runtime power management feature that dynamically puts the NPU in a low-power state when the NPU is not being used. The rest of the system still functions normally.

The Ethos-N NPU kernel module implements the Linux Power Management (PM) callbacks. This gives the flexibility to the system integrator to integrate the Ethos-N NPU in any power domain.

There are some restrictions on using the power management features. These restrictions are detailed in Limitations.

Firmware Binary

The ethosn.bin has been compiled with the following security-related flags:

-Werror -Wall -Wextra -Wformat=2 -Wno-format-nonliteral -Wctor-dtor-privacy -Woverloaded-virtual -Wsign-promo -Wstrict-overflow=2 -Wswitch-default -Wconversion -Wold-style-cast -Wsign-conversion -Wno-missing-braces

Build the Ethos-N driver stack in Android

To build the Ethos-N driver stack in Android, you must first install the Android NN driver with Arm NN in Android. To install the Android NN driver in Android, follow the instructions: https://github.com/ARM-software/android-nn-driver/blob/main/docs/IntegratorGuide.md. You must then put the downloaded Ethos-N project folder in the Android build folder.

Note: The Android build folder is the same folder that you put the Android NN driver.

After you install the Android NN driver with Arm NN in Android, install the Ethos-N driver stack using the setup_android.sh script. You can edit the paths in the script and execute it, or use the script as reference if you create your own script. The script performs the following steps:

1. The code in armnn-ethos-n-backend is an Arm NN backend which is built by Arm NN. For Arm NN to build the backend, a symbolic link is needed from the Arm NN src backend folder to the Ethos-N NPU backend folder:

   ln -s ../../../../ethos-n-driver-stack/armnn-ethos-n-backend ../android-nn-driver/armnn/src/backends/ethos-n

2. Build the Ethos-N kernel module, which will be treated as a pre-built by the Android build.

3. Update device.mk with instructions to include Arm NN Ethos-N backend and add Ethos-N kernel module and firmware

   We need to add a few config lines in device.mk. Arm NN needs a build flag to include the Ethos-N driver.

   • ARMNN_ETHOSN_ENABLE := 1

   In an Android build, we recommend adding the vendor-specific drivers to the /vendor file system so that the /system and /vendor file systems can be updated independently. This can be achieved by adding the kernel module and firmware to your device.mk, for example:

   • BOARD_VENDOR_KERNEL_MODULES += vendor/arm/ethos-n-driver-stack/kernel-module/ethosn.ko
   • PRODUCT_COPY_FILES += vendor/arm/ethos-n-driver-stack/firmware/ethosn.bin:$(TARGET_COPY_OUT_VENDOR)/lib/firmware/ethosn.bin

4. When booting, the kernel module must be started. To load the kernel module at boot, add the following to your init board file (for example, init.juno.rc):

   on post-fs
       insmod /vendor/lib/modules/ethosn.ko

5. To ensure the device tree contains the correct nodes for your Ethos-N hardware, the Ethos-N dts files must be merged with your dts file.

Build the Android image. Flash and run the image according to your normal target instructions.

_Note: If you get "No firmware found." in the kernel log, make sure your kernel searches for firmware files in /vendor/lib/firmware. For example, this can be done by adding the kernel argument "firmware_class.path=/vendor/lib/firmware/" when booting. If you want to use a different folder for your firmware files, you can also edit the build to make sure the ethonsn.bin file is moved to the different folder._

To use the Android NN driver and Arm NN, please see the instructions in the "Testing" section of the integration guide: https://github.com/ARM-software/android-nn-driver/blob/main/docs/IntegratorGuide.md#testing.

To specify the Ethos-N backend, use "-c EthosNAcc", for example:

adb shell /vendor/bin/hw/[email protected] -v -c EthosNAcc

Limitations

The following features and feature combinations have known limitations in this Ethos-N driver stack release.

Backend limitations

Custom allocators can only be used for importing intermediate tensors, not inputs and outputs.

Memory limitations

For systems that do not implement an Arm SMMU, the driver expects a reserved memory area to be associated with the NPU device. There are several restrictions on the properties of the reserved memory area. If these restrictions are not met, then the kernel module will not load successfully or the NPU will behave unexpectedly. The restrictions are:

1. The reserved memory area must begin on a 512MB aligned address.
2. The reserved memory area must not be larger than 512MB.
3. The reserved memory area must not be smaller than 4MB.
4. The size of the reserved memory area must be a power-of-two.
5. If the reserved memory area is smaller than 512MB, the NPU may still perform speculative memory reads to addresses up to 512MB from the starting address, which must not fail. The values returned from these speculative reads will not affect the behavior of the NPU. This means that the NPU must have read access to a full 512MB region, however the portion of the 512MB region which is not in the reserved memory area does not need to be backed by physical memory.

For systems that do implement a SMMU, a memory footprint of 3MB is required to create the page translations for the NPU memory accesses. This is to guarantee that speculative memory accesses to unmapped memory addresses cannot occur. During a short interval, when the SMMU page tables are reconfigured to map new buffers' address ranges, speculative accesses to unmapped addresses will result in a bus error, hence an error interrupt and any running inference will fail.

For more information on memory requirements and limitations, see the documentation for your SoC.

Power Management limitations

The hibernate power management feature is only supported with SMMU configurations.

Security limitations

The security guarantees provided by the NPU hardware and software stack depend on the configuration of the system. In particular, configurations that do not use an SMMU (and therefore use a reserved memory area) have weaker security because there is no SMMU that can be used to prevent the NPU from accessing memory that it should not be accessing (e.g. that belonging to inferences from other processes). A malicious userspace process might be able to cause the NPU to read or write to such memory by sending a maliciously crafted command stream. If security is of concern, we recommend that you use an SMMU.

License

The Ethos-N driver stack is composed of multiple components, each with their own license. The components are:

• The Arm Ethos-N NPU driver, which is the collection of user-space libraries.
• The Arm NN backend, which interfaces the Ethos-N NPU driver to Arm NN.
• The Arm Ethos-N NPU kernel module, to be used with the Linux kernel.
• The Arm Ethos-N NPU firmware, which will be loaded by the kernel module onto the NPU.

License for the Ethos-N NPU driver and Arm NN backend

The Arm Ethos-N NPU driver and the Arm NN backend are provided under the Apache-2.0 license. For more information, see LICENSE and LICENSE. Contributions to this project are accepted under the same license.

Copyright 2018-2023 Arm Limited

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Individual files contain the following tag instead of the full license text:

SPDX-License-Identifier: Apache-2.0

This enables machine processing of license information based on the SPDX License Identifiers that are available here: http://spdx.org/licenses/.

License for the Ethos-N NPU kernel module

The Arm Ethos-N NPU kernel module is provided under the GPL v2 only license. For more information, see LICENSE. Contributions to this project are accepted under the same license.

Individual files contain the following tag instead of the full license text:

SPDX-License-Identifier: GPL-2.0-only

EULA and TPIP for the Ethos-N NPU firmware

The Ethos-N NPU firmware binary is released under an EULA.

The Ethos-N NPU firmware binary was compiled against the CMSIS library, which is released under the Apache-2.0 license. For more information, see apache-2.0.txt.

Trademarks and copyrights

Arm and Ethos are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Android is a trademark of Google LLC.

Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

ethos-n-driver-stack's Issues

Build fails on Centos7/manylinux2014 and Ubuntu:latest

The build on Centos7/manylinux2014 fails at build stage:

g++ -o support_library/build/release/src/Support.o -c -std=c++14 -O3 -Werror -Wall -Wextra -Wformat=2 -Wno-format-nonliteral -Wctor-dtor-privacy -Woverloaded-virtual -Wsign-promo -Wstrict-overflow=2 -Wswitch-default -Wlogical-op -Wnoexcept -Wstrict-null-sentinel -Wconversion -fPIC -Icommand_stream/include -Iutils/include -Isupport_library/build/release/src -Isupport_library/src -Isupport_library/build/release/include -Isupport_library/include support_library/src/Support.cpp

With warnings that include:

In file included from support_library/src/Graph.hpp:9, from support_library/src/Compiler.hpp:10, from support_library/src/Support.cpp:9: /opt/rh/devtoolset-9/root/usr/include/c++/9/ext/new_allocator.h: In instantiation of 'void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = ethosn::support_library::Operand::Consumer; _Args = {ethosn::support_library::Operation&, const long unsigned int&}; _Tp = ethosn::support_library::Operand::Consumer]': /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/alloc_traits.h:482:2: required from 'static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = ethosn::support_library::Operand::Consumer; _Args = {ethosn::support_library::Operation&, const long unsigned int&}; _Tp = ethosn::support_library::Operand::Consumer; std::allocator_traits<std::allocator<_CharT> >::allocator_type = std::allocator<ethosn::support_library::Operand::Consumer>]' /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/vector.tcc:115:30: required from 'void std::vector<_Tp, _Alloc>::emplace_back(_Args&& ...) [with _Args = {ethosn::support_library::Operation&, const long unsigned int&}; _Tp = ethosn::support_library::Operand::Consumer; _Alloc = std::allocator<ethosn::support_library::Operand::Consumer>]' support_library/src/Network.hpp:46:50: required from here support_library/src/Network.hpp:28:19: error: but 'constexpr ethosn::support_library::Operand::Consumer::Consumer(ethosn::support_library::Operation&, size_t)' does not throw; perhaps it should be declared 'noexcept' [-Werror=noexcept] 28 | constexpr Consumer(Operation& operation, const size_t inputIndex) | ^~~~~~~~

Resulting in

cc1plus: all warnings being treated as errors scons: *** [support_library/build/release/src/Support.o] Error 1 scons: building terminated because of errors.

Similar warnings appear regardless of gcc toolchain version installed (7 or 8).

Kernel support

Hello!

The readme says that the ethos-n-driver has been tested with Kernels v5.4 and v5.10.

Is there a roadmap for newer kernels to be supported/tested?

Thanks, Chris

SPA test generates warning "Layer of type is not supported"

Hi,
We have a license to use a SPA program, and we analyze modified BiSeNetv2 models with the SPA program for evaluating NPU. During the analysis process of modified BiSeNetv2 int8 and uint8 model, we met the following warning respectively:

(uint8): Layer of type Resize is not supported on request backend EthosNAcc for input data type QAsymmU8 and output data type QAsymmU8

(int8): Layer of type DepthwiseConvolution2d is not supported on request backend EthosNAcc for input data type QAsymmS8 and output data type QAsymmS8

We want to ask:

1. whether the warning messages above affect the results when using a physical NPU?
2. The warning messages informs the layer is falling back to the next backend, will the unsupported layer be executed on GPU or CPU?

Best Regards,
Rahn

Orange Pi 5B no driver file created after initializing the kernel module

Using Ubuntu 22.04 (5.10.160-rockchip kernel) with the SMMU config options as per your tutorial.

The firmware file was copied to the right folder, the ethosn.ko file initialized with sudo insmod ethosn.ko and it works:

dmesg | tail shows:
[ 8232.407774] Registering ethosn-main_allocator
[ 8232.410038] Registering ethosn-memory
[ 8232.411936] Registering ethosn-core

and lsmod shows:
ethosn 110592 0

and yet there is no device file:
$ ls /dev/ethosn*
ls: cannot access '/dev/ethosn*': No such file or directory

and so unit tests don't work etc.

What could be wrong?

Best practise to create a quantized tflite model

Can you guys share a recommended way fo producing quantised tflite model for the SPA tool to consume? I have followed the instructions from Tensorflow to quantize a simple mnist model, but the tflite model generated can not be processed by the SPA tool(AnalyzeNetowork command):

The example code can be found here. I used tensoflow 2.3.4 to run the code, which matches the version I compiled the SPA with:armnn 21.05 and tensorflow r2.3. I have tried both tf.lite.TFLiteConverter.from_keras_model and tf.lite.TFLiteConverter.from_saved_model, neither of the quantized model can be consumed by the SPA tool. Can you guys advice?

compilation error with scons

Hi,

when I compile ethos-n driver with scons, I met the error
support_library/src/nonCascading/Strategies.cpp:737:66: error: '*((void*)(& bestParams)+4).ethosn::support_library::Strategy6::TrySetupAnyBlockConfig(ethosn::support_library::TensorConfig&, ethosn::support_library::SramAllocator&, const TensorShape&, const TensorShape&, const TensorShape&, ethosn::support_library::DataFormat, const TensorShape&, const std::vector<ethosn::command_stream::BlockConfig>&, const ethosn::support_library::HardwareCapabilities&, const ethosn::support_library::utils::ShapeMultiplier&, const ethosn::support_library::utils::ShapeMultiplier&, std::pair<bool, unsigned int>, ethosn::support_library::CompilerMceAlgorithm, uint32_t)::Strategy6Params::blockHeight' may be used uninitialized in this function [-Werror=maybe-uninitialized] tensorConfig.blockHeight = bestParams.value().blockHeight; ^ support_library/src/nonCascading/Strategies.cpp:736:65: error: '*((void*)(& bestParams)+4).ethosn::support_library::Strategy6::TrySetupAnyBlockConfig(ethosn::support_library::TensorConfig&, ethosn::support_library::SramAllocator&, const TensorShape&, const TensorShape&, const TensorShape&, ethosn::support_library::DataFormat, const TensorShape&, const std::vector<ethosn::command_stream::BlockConfig>&, const ethosn::support_library::HardwareCapabilities&, const ethosn::support_library::utils::ShapeMultiplier&, const ethosn::support_library::utils::ShapeMultiplier&, std::pair<bool, unsigned int>, ethosn::support_library::CompilerMceAlgorithm, uint32_t)::Strategy6Params::blockWidth' may be used uninitialized in this function [-Werror=maybe-uninitialized] tensorConfig.blockWidth = bestParams.value().blockWidth; ^ cc1plus: all warnings being treated as errors scons: *** [support_library/build/release_native/src/nonCascading/Strategies.o] Error 1 scons: building terminated because of errors.
Does anyone know how to solve these errors?

Support for Raspberry or jetson nano ?

Hi ,

Does Rpi4 or Jetson nano have NPU ? Which board has the NPU and where to buy from ? Totally clueless

Conv2D could not be estimated

Hi,

We have a license to the SPA tool (compiled with 21.05 armnn and tensorflow r2.3 specifically). We tried to evaluated a quantized network. The AnalyzeNetwork command executed without exceptions, but the the perfreport it generated listed 3 unsupported operators：

.
However, the Conv2D it complains about is a normal Convolution layer just like the rest of the network Conv2D layers, which can be seen in the attached. I have attached the model here.

Can you guys let me know what I am missing?

The converged pad and convolution NPU are not supported

pad: pad_left: 1 pad_right: 1 pad_top: 1 pad_bottom: 1
convolution: dialation: 1 padding: VALID stride: 2 kernel_size: 3x3
After actual testing，the above convolution NPU is supported, but the NPU does not support the fusion of the two operators. The problem is that when the two operators are fused, the pad mode of convolution is SAME, which is supported by NPU documentation.

The NPU documentation is described as follows：

Convolution 2D

• HWIO format weights are supported.
• The supported kernel heights and widths (the kernel does not have to be square) are: { 1, 2, 3, 5, 7, 9 }.
  - The supported strides (the height and width stride have to match) are: { 1, 2 }.
• For kernels with height or width >7, only a stride of 1 is supported.
  - SAME and VALID padding are supported.
• I*W/O must be between 2.33e-10 and 1, where:
  • I is the input quantization scale.
  • W is the weight quantization scale.
  • O is the output quantization scale.

Support of elementwise mulitiplication

Hi,
Arm NN Ethos-N backend 21.08 supports multiplication of a variable by a scalar constant. Does this means Ethos-N backend 21.08 currently does not support the operation of elementwise multiplication for multiplying two tensors?

Best regard,
Rahn

Error message while executing inference with NPU

Hi,

We adopted two methods to compile our ArmNN inference code, the first method is compiling via Makefile and the other one is via CMakeLists. We logged the execution time from these compiling approaches and compared the results based on the same inference code, test image, and test model, and the logs are shown in the attachment.
logs.zip

However, the execution results contain the following error message:

Error: An error occurred attempting to execute a workload: An error has occurred waiting for the inference of a pre-compiled object: Timed out while waiting for the inference to complete
Why does the execution of NPU produce the error above? Since it takes time to reset the status of NPU, how to resolve the issue?

Best Regards

Inference result of int8 model on NPU

Hi,
We've tested an int8 model with ArmNN 21.08 and Compute Library 21.08 patch, which solves the issue of wrong result in int8 models.

To further evaluate the results on NPU, we want to ask that if we apply the same tflite model on NPU, will the inferred results contain noise or the results be identical to TFLite API?

Best regards,
Rahn

Which type of quantization does ARM NPU support?

I want to know ARM NPU can support both symmetric and asymmetric quantization or just symmetric quantization.

Ethos-N driver + simulated Ethos-N NPU?

Hi all,

I want to introduce some security protections to Arm NPU. It requires to modify the NPU driver to collaborate with some access control in TF-A. Currently, I want to implement on Arm Ethos-N NPU driver, but I do not have a real Ethos-N NPU.

Thus, if I use some NPU simulator, such as SCALE-Sim (https://github.com/ARM-software/SCALE-Sim), to simulate the NPU, will the driver works with the simulator?

Execution error of Convolution2d on EthosNAcc mode

Hi,
We have an int8 tflite model based on BiSeNetv2 architecture. When we applied the model on NPU, we met the following error:
WARNING: Layer of type Convolution2d is not supported on requested backend EthosNAcc for input data type QAsymmS8 and output data type QAsymmS8(reason: Convolution: Overall scale (of the input* weights/ output) should be in range [2.328306e-10, 1)), falling back to the next backend.
Warning: ERROR: Layer of type Convolution2d is not supported on any preferred backend[EthosNAcc]
terminate called after throwing an instance of 'armnn::InvalidArgumentException'
what(): Failed to assign a backend to each layer

However, the same model could be executed in GPUAcc mode and SPA tool (with NPU evaluation). Does the error above mean the weight value of the convolution2d layer must in the range [2.328306e-10, 1)?

Best Regards,
Rahn

An exception occurred using the cache network function

I first changed the optimization options to save the compiled network, and then changed the optimization options again to load the network that was serialized to the file in the previous step, but there was an exception, which seems to be due to a format check error during deserialization

The execution log is as follows：

./object_detection_example --model-file-path ./22W27-yolov5s-conv3-conv2-relu_640-int8.tflite --label-path ./label.txt --preferred-backends EthosNAcc --img-path ./test.jpg --num-runs 1

Info: ObjectDetection example is running...
Info: ObjectDetection example profiling is running...
Info: ArmNN v29.0.0
Info: Initialization time: 0.41 ms.
Info: Network parsing time: 303.61 ms.
Debug: OptimizerOptions:
ReduceFp32ToFp16: 0
ReduceFp32ToBf16: 0
Debug: 0
ShapeInferenceMethod: ValidateOnly
ImportEnabled: 0
ProfilingEnabled: 0
ModelOptions:
Backend: EthosNAcc
Option: SaveCachedNetwork
Value: false
Backend: EthosNAcc
Option: CachedNetworkFilePath
Value: /mnt/run/EthosN-CachingEndToEnd-TempFile.bin

Info: Optimization time: 304.33 ms.
Error: An error occurred when preparing the network workloads: Not a serialized CompiledNetwork
Error: An error occurred when preparing the network workloads: Not a serialized CompiledNetwork
terminate called after throwing an instance of 'armnn::Exception'
what(): An error occurred when preparing the network workloads: Not a serialized CompiledNetwork
Aborted

[BUG]Unable to build tvm with ethos-n-driver-stack support

I want to build the Ethos-N backend with tvm in a minimal way, I just build the driver lib and it generates four files: libEthosNDriver.a, libEthosNDriver.so, libEthosNSupport.a, libEthosNSupport.so. And install into the default directory, with the most updated version(20.11) of Ethos-N Driver stack. Another problem arises, I get the following errors:

/home/dyn/tvm/src/relay/backend/contrib/ethosn/codegen.cc: In member function ‘tvm::relay::contrib::ethosn::NetworkWithIDs tvm::relay::contrib::ethosn::ConstructNetworkVisitor::Construct(const tvm::relay::Function&)’:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/codegen.cc:202:32: error: too few arguments to function ‘std::shared_ptr<ethosn::support_library::Network> ethosn::support_library::CreateNetwork(const std::vector<char>&)’
  202 |   network_ = sl::CreateNetwork();
      |                                ^
In file included from /home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api_version.h:20,
                 from /home/dyn/tvm/src/relay/backend/contrib/ethosn/capabilities.h:34,
                 from /home/dyn/tvm/src/relay/backend/contrib/ethosn/codegen.cc:27:
/usr/local/include/ethosn_support_library/Support.hpp:1116:26: note: declared here
 1116 | std::shared_ptr<Network> CreateNetwork(const std::vector<char>& caps);
      |                          ^~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/codegen.cc: In static member function ‘static ethosn::support_library::CompilationOptions tvm::relay::contrib::ethosn::EthosnCompiler::CreateOptions()’:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/codegen.cc:564:64: error: no matching function for call to ‘ethosn::support_library::CompilationOptions::CompilationOptions(std::vector<char>&)’
  564 |   sl::CompilationOptions options(variants[cfg.value()->variant]);
      |                                                                ^
In file included from /home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api_version.h:20,
                 from /home/dyn/tvm/src/relay/backend/contrib/ethosn/capabilities.h:34,
                 from /home/dyn/tvm/src/relay/backend/contrib/ethosn/codegen.cc:27:
/usr/local/include/ethosn_support_library/Support.hpp:69:8: note: candidate: ‘constexpr ethosn::support_library::CompilationOptions::CompilationOptions()’
   69 | struct CompilationOptions
      |        ^~~~~~~~~~~~~~~~~~
/usr/local/include/ethosn_support_library/Support.hpp:69:8: note:   candidate expects 0 arguments, 1 provided
/usr/local/include/ethosn_support_library/Support.hpp:69:8: note: candidate: ‘ethosn::support_library::CompilationOptions::CompilationOptions(const ethosn::support_library::CompilationOptions&)’
/usr/local/include/ethosn_support_library/Support.hpp:69:8: note:   no known conversion for argument 1 from ‘std::vector<char>’ to ‘const ethosn::support_library::CompilationOptions&’
/usr/local/include/ethosn_support_library/Support.hpp:69:8: note: candidate: ‘ethosn::support_library::CompilationOptions::CompilationOptions(ethosn::support_library::CompilationOptions&&)’
/usr/local/include/ethosn_support_library/Support.hpp:69:8: note:   no known conversion for argument 1 from ‘std::vector<char>’ to ‘ethosn::support_library::CompilationOptions&&’
/home/dyn/tvm/src/relay/backend/contrib/ethosn/codegen.cc:579:55: error: cannot convert ‘const bool’ to ‘ethosn::support_library::CompilationOptions::DebugLevel’ in assignment
  579 |   options.m_DebugInfo.m_DumpDebugFiles = cfg.value()->dump_debug_files;
      |                                          ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
      |                                                       |
      |                                                       const bool
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:638:27: error: ‘IsDepthwiseConvolutionSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  638 |         *rv = !err && sl::IsDepthwiseConvolutionSupported(params.bias_info, params.weights_info,
      |                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:641:27: error: ‘IsConvolutionSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  641 |         *rv = !err && sl::IsConvolutionSupported(params.bias_info, params.weights_info,
      |                           ^~~~~~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:651:25: error: ‘IsFullyConnectedSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  651 |       *rv = !err && sl::IsFullyConnectedSupported(params.bias_info, params.weights_info,
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:660:25: error: ‘IsPoolingSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  660 |       *rv = !err && sl::IsPoolingSupported(params.pool_info, params.input_info);
      |                         ^~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:668:25: error: ‘IsPoolingSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  668 |       *rv = !err && sl::IsPoolingSupported(params.pool_info, params.input_info);
      |                         ^~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:676:25: error: ‘IsReshapeSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  676 |       *rv = !err && sl::IsReshapeSupported(params.new_shape, params.input_info);
      |                         ^~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:684:25: error: ‘IsAdditionSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  684 |       *rv = !err && sl::IsAdditionSupported(params.lhs_info, params.rhs_info,
      |                         ^~~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:693:25: error: ‘IsSigmoidSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  693 |       *rv = !err && sl::IsSigmoidSupported(params.input_info);
      |                         ^~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:701:25: error: ‘IsConcatenationSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  701 |       *rv = !err && sl::IsConcatenationSupported(params.input_infos, params.concat_info);
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:709:25: error: ‘IsSplitSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  709 |       *rv = !err && sl::IsSplitSupported(params.input_info, params.split_info);
      |                         ^~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:717:25: error: ‘IsDepthToSpaceSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  717 |       *rv = !err && sl::IsDepthToSpaceSupported(params.input_info, params.depth_info);
      |                         ^~~~~~~~~~~~~~~~~~~~~~~
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc: In lambda function:
/home/dyn/tvm/src/relay/backend/contrib/ethosn/ethosn_api.cc:725:25: error: ‘IsReluSupported’ is not a member of ‘tvm::relay::contrib::ethosn::sl’
  725 |       *rv = !err && sl::IsReluSupported(params.relu_info, params.input_info);
      |                         ^~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/tvm_objs.dir/build.make:4340: CMakeFiles/tvm_objs.dir/src/relay/backend/contrib/ethosn/ethosn_api.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [CMakeFiles/tvm_objs.dir/build.make:4327: CMakeFiles/tvm_objs.dir/src/relay/backend/contrib/ethosn/codegen.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:193: CMakeFiles/tvm_objs.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

When I check out the 20.08 version of Ethos-N driver stack and build it, I get the following errors:

g++ -o support_library/build/release/src/Support.o -c -std=c++14 -O3 -Werror -Wall -Wextra -Wformat=2 -Wno-format-nonliteral -Wctor-dtor-privacy -Woverloaded-virtual -Wsign-promo -Wstrict-overflow=2 -Wswitch-default -Wlogical-op -Wnoexcept -Wstrict-null-sentinel -Wconversion -fPIC -Icommand_stream/include -Iutils/include -Isupport_library/build/release/src -Isupport_library/src -Isupport_library/build/release/include -Isupport_library/include support_library/src/Support.cpp
In file included from support_library/src/Graph.hpp:9,
                 from support_library/src/Compiler.hpp:10,
                 from support_library/src/Support.cpp:9:
/usr/include/c++/9/ext/new_allocator.h: In instantiation of 'void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = ethosn::support_library::Operand::Consumer; _Args = {ethosn::support_library::Operation&, const long unsigned int&}; _Tp = ethosn::support_library::Operand::Consumer]':
/usr/include/c++/9/bits/alloc_traits.h:482:2:   required from 'static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = ethosn::support_library::Operand::Consumer; _Args = {ethosn::support_library::Operation&, const long unsigned int&}; _Tp = ethosn::support_library::Operand::Consumer; std::allocator_traits<std::allocator<_CharT> >::allocator_type = std::allocator<ethosn::support_library::Operand::Consumer>]'
/usr/include/c++/9/bits/vector.tcc:115:30:   required from 'void std::vector<_Tp, _Alloc>::emplace_back(_Args&& ...) [with _Args = {ethosn::support_library::Operation&, const long unsigned int&}; _Tp = ethosn::support_library::Operand::Consumer; _Alloc = std::allocator<ethosn::support_library::Operand::Consumer>]'
support_library/src/Network.hpp:46:50:   required from here
support_library/src/Network.hpp:28:19: error: but 'constexpr ethosn::support_library::Operand::Consumer::Consumer(ethosn::support_library::Operation&, size_t)' does not throw; perhaps it should be declared 'noexcept' [-Werror=noexcept]
   28 |         constexpr Consumer(Operation& operation, const size_t inputIndex)
      |                   ^~~~~~~~
/usr/include/c++/9/ext/new_allocator.h: In instantiation of 'void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = ethosn::support_library::Network; _Args = {}; _Tp = ethosn::support_library::Network]':
/usr/include/c++/9/bits/alloc_traits.h:482:2:   required from 'static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = ethosn::support_library::Network; _Args = {}; _Tp = ethosn::support_library::Network; std::allocator_traits<std::allocator<_CharT> >::allocator_type = std::allocator<ethosn::support_library::Network>]'
/usr/include/c++/9/bits/shared_ptr_base.h:548:39:   required from 'std::_Sp_counted_ptr_inplace<_Tp, _Alloc, _Lp>::_Sp_counted_ptr_inplace(_Alloc, _Args&& ...) [with _Args = {}; _Tp = ethosn::support_library::Network; _Alloc = std::allocator<ethosn::support_library::Network>; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]'
/usr/include/c++/9/bits/shared_ptr_base.h:679:16:   required from 'std::__shared_count<_Lp>::__shared_count(_Tp*&, std::_Sp_alloc_shared_tag<_Alloc>, _Args&& ...) [with _Tp = ethosn::support_library::Network; _Alloc = std::allocator<ethosn::support_library::Network>; _Args = {}; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]'
/usr/include/c++/9/bits/shared_ptr_base.h:1344:71:   required from 'std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<ethosn::support_library::Network>; _Args = {}; _Tp = ethosn::support_library::Network; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]'
/usr/include/c++/9/bits/shared_ptr.h:359:59:   required from 'std::shared_ptr<_Tp>::shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<ethosn::support_library::Network>; _Args = {}; _Tp = ethosn::support_library::Network]'
/usr/include/c++/9/bits/shared_ptr.h:701:14:   required from 'std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = ethosn::support_library::Network; _Alloc = std::allocator<ethosn::support_library::Network>; _Args = {}]'
/usr/include/c++/9/bits/shared_ptr.h:717:39:   required from 'std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = ethosn::support_library::Network; _Args = {}]'
support_library/src/Support.cpp:405:38:   required from here
support_library/src/Network.hpp:81:5: error: but 'ethosn::support_library::Network::Network(bool)' does not throw; perhaps it should be declared 'noexcept' [-Werror=noexcept]
   81 |     Network(bool estimatePerformance = false)
      |     ^~~~~~~
/usr/include/c++/9/ext/new_allocator.h: In instantiation of 'void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = ethosn::support_library::Network; _Args = {bool}; _Tp = ethosn::support_library::Network]':
/usr/include/c++/9/bits/alloc_traits.h:482:2:   required from 'static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = ethosn::support_library::Network; _Args = {bool}; _Tp = ethosn::support_library::Network; std::allocator_traits<std::allocator<_CharT> >::allocator_type = std::allocator<ethosn::support_library::Network>]'
/usr/include/c++/9/bits/shared_ptr_base.h:548:39:   required from 'std::_Sp_counted_ptr_inplace<_Tp, _Alloc, _Lp>::_Sp_counted_ptr_inplace(_Alloc, _Args&& ...) [with _Args = {bool}; _Tp = ethosn::support_library::Network; _Alloc = std::allocator<ethosn::support_library::Network>; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]'
/usr/include/c++/9/bits/shared_ptr_base.h:679:16:   required from 'std::__shared_count<_Lp>::__shared_count(_Tp*&, std::_Sp_alloc_shared_tag<_Alloc>, _Args&& ...) [with _Tp = ethosn::support_library::Network; _Alloc = std::allocator<ethosn::support_library::Network>; _Args = {bool}; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]'
/usr/include/c++/9/bits/shared_ptr_base.h:1344:71:   required from 'std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<ethosn::support_library::Network>; _Args = {bool}; _Tp = ethosn::support_library::Network; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]'
/usr/include/c++/9/bits/shared_ptr.h:359:59:   required from 'std::shared_ptr<_Tp>::shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<ethosn::support_library::Network>; _Args = {bool}; _Tp = ethosn::support_library::Network]'
/usr/include/c++/9/bits/shared_ptr.h:701:14:   required from 'std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = ethosn::support_library::Network; _Alloc = std::allocator<ethosn::support_library::Network>; _Args = {bool}]'
/usr/include/c++/9/bits/shared_ptr.h:717:39:   required from 'std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = ethosn::support_library::Network; _Args = {bool}]'
support_library/src/Support.cpp:410:42:   required from here
support_library/src/Network.hpp:81:5: error: but 'ethosn::support_library::Network::Network(bool)' does not throw; perhaps it should be declared 'noexcept' [-Werror=noexcept]
cc1plus: all warnings being treated as errors
scons: *** [support_library/build/release/src/Support.o] Error 1
scons: building terminated because of errors.

How to fix this problem?
This is a continueing discuss of apache/tvm#7261.