fraunhofer-ims / aifes_for_arduino

This is the Arduino® compatible port of the AIfES machine learning framework, developed and maintained by Fraunhofer Institute for Microelectronic Circuits and Systems.

AIfES for Arduino®

AIfES (Artificial Intelligence for Embedded Systems) is a platform-independent and standalone AI software framework optimized for embedded systems. The Feedforward Neural Networks (FNN) and Convolutional Neural Networks (CNN) implemented in AIfES can be freely parameterized, trained, modified or reloaded at runtime. In this version, it is optimized for the Arduino IDE and compatible to almost any Arduino board. AIfES is developed in the C programming language and uses only standard libraries based on the GNU Compiler Collection (GCC). AIfES thus runs on almost any hardware from 8-bit microcontrollers over Raspberry PI to smartphones or PCs. Not only inference of FNN and CNN is possible, but also training directly in the device. Furthermore, compatibility to other AI software frameworks such as Keras or TensorFlow is also given.

AIfES was developed by the Fraunhofer Institute for Microelectronic Circuits and Systems IMS. Additional information can be found at <www.aifes.ai>

The Fraunhofer IMS with AIfES and Arduino preapare to enter a partnership.

We are also at the Arduino Project Hub. Here you can find our new AIfES projects and examples: https://create.arduino.cc/projecthub/aifes_team

Follow us on LinkedIn to get all the latest news about AIfES: https://www.linkedin.com/showcase/aifes-ai

Dual License

For Maker and Open-Source-Projects (GNU AGPLv3):

For private projects or developers of Free Open Source Software (FOSS) under version 3 of the GNU Affero General Public License (AGPL), the AIfES version offered here can be used free of charge.

For use in commercial applications:

If AIfES is to be combined and distributed with commercially licensed software and/or if you do not wish to distribute the AIfES source code for the commercially licensed software under version 3 of the GNU Affero General Public License (AGPL), you must conclude a license agreement with Fraunhofer IMS. For more information and contact, refer to our homepage

Compatibility

AIfES in the current version supports almost all microcontroller types and Arduino boards:

• 8-Bit-Microcontroller
• 16-Bit-Microcontroller
• 32-Bit-Microcontroller
• 64-Bit-Microcontroller

Installation guides for various IDEs

We have already put the first guides to integrate AIfES in various IDEs in our GitHub repository which you can find here.

Guides for the following IDEs are available by now:

• Code::Blocks
• STMCubeIDE
• Wokwi - the simulator for Arduino / ESP32

You can help us by downloading the *.docx template and write a manual for your favourite IDE. Send it as a PDF to aifes(at)ims.fraunhofer.de

Wokwi - Online Arduino and ESP32 Simulator

If you don't have any hardware available, you can also test some AIfES examples in Wokwi. The simulation is slower, of course. Here are a few examples:

• AIfES_Express_XOR_F32_Inference (Arduino UNO 8-Bit)
• AIfES_Express_XOR_F32_training (Arduino Mega 8-Bit)
• AIfES_Express_XOR_F32_training (ESP32)
• TicTacToe_Q7 (Arduino UNO 8-Bit)

ARM CMSIS

AIfES also supports the Cortex Microcontroller Software Interface Standard (CMSIS) of the ARM Cortex controllers. This standard can speed up inference and training for large FNNs. Please read the README.md in https://github.com/Fraunhofer-IMS/AIfES_for_Arduino/tree/main/src/CMSIS for more information on how to add the CMSIS library.

Python

To get you startet we developed the AIfES-Converter. It can export your FNN from Python and create the suitable AIfES code. Furthermore, it helps you with the quantization of your neural network. You can install it via pip with:

pip install AIfES-Converter

The quantization can also be done manually by using the provided Python tools in this repository. You can install them via pip with:

pip install https://github.com/Fraunhofer-IMS/AIfES_for_Arduino/raw/main/etc/python/aifes_tools.zip

Please read the README.md in https://github.com/Fraunhofer-IMS/AIfES_for_Arduino/tree/main/etc/python for more information on the AIfES pytools.

We also created some examples to show you how a tf.Keras and a PyTorch neural network can be quantized for AIfES, using the pytools https://github.com/Fraunhofer-IMS/AIfES_for_Arduino/tree/main/etc/python/examples.

Features

Have a look at our documentation for a detailed overview of the provided features with code examples.

Data types and quantization

AIfES supports the following data types:

• F32: 32 Bit float
• Q31: 32 Bit integer
• Q7: 8 Bit integer

Neural network types AIfES supports in the current version:

• Feedforward Neural Network (FNN) inference and training
• Other network types are in progress (see roadmap)

The number of neurons and the number of different layers can be adapted individually. Inference and training can be performed on the controller.

Inference layer

Layer	f32	q31	q7
Dense	ailayer_dense_f32_default() ailayer_dense_f32_cmsis() ailayer_dense_f32_avr_pgm()	ailayer_dense_q31_default()	ailayer_dense_q7_default() ailayer_dense_wt_q7_default() ailayer_dense_wt_q7_cmsis() ailayer_dense_q7_avr_pgm() ailayer_dense_wt_q7_avr_pgm()
Input	ailayer_input_f32_default()	ailayer_input_q31_default()	ailayer_input_q7_default()
ReLU	ailayer_relu_f32_default()	ailayer_relu_q31_default()	ailayer_relu_q7_default() ailayer_relu_q7_avr_pgm()
Sigmoid	ailayer_sigmoid_f32_default()	ailayer_sigmoid_q31_default()	ailayer_sigmoid_q7_default() ailayer_sigmoid_q7_avr_pgm()
Softmax	ailayer_softmax_f32_default()	ailayer_softmax_q31_default()	ailayer_softmax_q7_default() ailayer_softmax_q7_avr_pgm()
Leaky ReLU	ailayer_leaky_relu_f32_default()	ailayer_leaky_relu_q31_default()	ailayer_leaky_relu_q7_default() ailayer_leaky_relu_q7_avr_pgm()
ELU	ailayer_elu_f32_default()	ailayer_elu_q31_default()	ailayer_elu_q7_default() ailayer_elu_q7_avr_pgm()
Tanh	ailayer_tanh_f32_default()	ailayer_tanh_q31_default()	ailayer_tanh_q7_default() ailayer_tanh_q7_avr_pgm()
Softsign	ailayer_softsign_f32_default()	ailayer_softsign_q31_default()	ailayer_softsign_q7_default() ailayer_softsign_q7_avr_pgm()
Conv2D	ailayer_conv2d_f32_default()
Batch Normalization	ailayer_batch_norm_f32_default()
MaxPool2D	ailayer_maxpool2d_f32_default()
Reshape	ailayer_reshape_f32_default()
Flatten	ailayer_flatten_f32_default()

Training layer

Layer	f32	q31	q7
Dense	ailayer_dense_f32_default() ailayer_dense_f32_cmsis() ailayer_dense_f32_avr_pgm()	ailayer_dense_q31_default()
Input	ailayer_input_f32_default()	ailayer_input_q31_default()	ailayer_input_q7_default()
ReLU	ailayer_relu_f32_default()	ailayer_relu_q31_default()	ailayer_relu_q7_default()
Sigmoid	ailayer_sigmoid_f32_default()	ailayer_sigmoid_q31_default()
Softmax	ailayer_softmax_f32_default()	ailayer_softmax_q31_default()
Leaky ReLU	ailayer_leaky_relu_f32_default()	ailayer_leaky_relu_q31_default()
ELU	ailayer_elu_f32_default()	ailayer_elu_q31_default()
Tanh	ailayer_tanh_f32_default()	ailayer_tanh_q31_default()
Softsign	ailayer_softsign_f32_default()	ailayer_softsign_q31_default()
Conv2D	ailayer_conv2d_f32_default()
Batch Normalization	ailayer_batch_norm_f32_default()
MaxPool2D	ailayer_maxpool2d_f32_default()
Reshape	ailayer_reshape_f32_default()
Flatten	ailayer_flatten_f32_default()

Loss:

Loss	f32	q31	q7
Mean Squared Error (MSE)	ailoss_mse_f32_default()	ailoss_mse_q31_default()
Crossentropy	ailoss_crossentropy_f32_default() ailoss_crossentropy_sparse8_f32_default()

Optimizer:

Optimizer	f32	q31	q7
Stochastic Gradient Descent (SGD)	aiopti_sgd_f32_default()	aiopti_sgd_q31_default()
Adam	aiopti_adam_f32_default()

Installation

You can download and install AIfES® (search for "aifes") with the Arduino library manager. https://www.arduino.cc/en/guide/libraries

Alternatively, the manual download is also possible: Download the AIfES repository as a ZIP archive and follow these instructions: https://www.arduino.cc/en/guide/libraries

Citation

If you use this software in your work please cite it.

For your scientific work you can cite the following paper:

Plain Text:

L. Wulfert et al., "AIfES: A Next-Generation Edge AI Framework," in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2024.3355495.
keywords: {Training;Data models;Artificial intelligence;Support vector machines;Hardware acceleration;Libraries;Performance evaluation;Machine Learning Framework;Edge AI Framework;On-Device Training;Embedded Systems;Resource-Constrained Devices;TinyML}

BibTex:

@ARTICLE{10403985,
  author={Wulfert, Lars and Kühnel, Johannes and Krupp, Lukas and Viga, Justus and Wiede, Christian and Gembaczka, Pierre and Grabmaier, Anton},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
  title={AIfES: A Next-Generation Edge AI Framework}, 
  year={2024},
  volume={},
  number={},
  pages={1-16},
  keywords={Training;Data models;Artificial intelligence;Support vector machines;Hardware acceleration;Libraries;Performance evaluation;Machine Learning Framework;Edge AI Framework;On-Device Training;Embedded Systems;Resource-Constrained Devices;TinyML},
  doi={10.1109/TPAMI.2024.3355495}}

Roadmap

The AIfES team at Fraunhofer IMS is constantly working on new features and network types.

aifes_for_arduino's Issues

Linear regression training problem

Hello, I would like to ask some questions, forgive my English is not good. When I was doing the linear regression training, the value of the Loss function appeared to be fixed at a certain value. For my case, it will always be 14.25. Of course, the trained model did not succeed, and the the predicted value of the model before training is also always 0. But after several failures, I accidentally touched the analog pin during setup(), and then the training was successful. The following is the log of my training model, one succeeded (random seed: 641), the other failed (random seed: 17)
Training fail:
training_fail.txt

Training success:
training_success.txt

I think it should be the random seed that caused the initial model weight setting problem. Is my idea correct?

I won against the trained TicTacToe AI (Q7)

Hello,

I uploaded the example code to Arduino UNO board and managed to win.

This is the win patter:

Me: b2
AI: a1
Me: a2
AI: c2
Me: b1

here AI plays c1 instead of b3 because I can win at b3.

I tested the same thing for F32 and the AI made the correct move there.

Feature Request: Support for Arduino PortentaH7 Vision and Sound

Really impressed with this Arduino Library so far.

Any plans for future support for the Portenta Vision Shields?

I hve some draft examples using Edge Impulse or my own generic ML @rocksetta methods converting from TensorflowJS to TensorflowMicro c header files.

AifES compiling issue with gcc on PC.

issue is resolved.

Recurrent NNs

Really appreciate the work you guys are doing here. Library is great and a strong alternative to Tensorflow Lite Micro - arguably even better given how easy it is to load and update new weights. Since a lot of sensor data makes sense with time as a dimension, it begs the question though; is there any chance we are getting support for recurrent neural networks? More specifically GRUs and LSTMs.

Regards.

LSTM Support

Hi, seems like a great library! Is there a support for LSTM or any RNN model?

Fix bug in aifes_config.h

Hello,

While trying to compile AIfES on an STM32 Nucleo-64 board (STM32L476RG), I got the following error, after including the library in my project:

Description	Resource	Path	Location	Type
expected identifier or '(' before '=' token	ailoss_crossentropy_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/ailoss	line 28	C/C++ Problem
expected identifier or '(' before '=' token	ailoss_crossentropy_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/ailoss	line 52	C/C++ Problem
expected identifier or '(' before '=' token	aimath_f32_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/aimath	line 29	C/C++ Problem
expected identifier or '(' before '=' token	aimath_f32_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/aimath	line 30	C/C++ Problem
expected identifier or '(' before '=' token	aimath_q31_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/aimath	line 28	C/C++ Problem
expected identifier or '(' before '=' token	aimath_q31_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/aimath	line 29	C/C++ Problem
expected identifier or '(' before '=' token	aimath_q31_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/aimath	line 443	C/C++ Problem
expected identifier or '(' before '=' token	aimath_q7_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/aimath	line 28	C/C++ Problem
expected identifier or '(' before '=' token	aimath_q7_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/aimath	line 29	C/C++ Problem
expected identifier or '(' before '=' token	aimath_q7_default.c	/AIfES_TestBug/AIfES_for_Arduino/src/basic/default/aimath	line 30	C/C++ Problem
make: *** [AIfES_for_Arduino/src/basic/default/ailoss/subdir.mk:22: AIfES_for_Arduino/src/basic/default/ailoss/ailoss_crossentropy_default.o] Error 1	AIfES_TestBug		 	C/C++ Problem
make: *** [AIfES_for_Arduino/src/basic/default/aimath/subdir.mk:25: AIfES_for_Arduino/src/basic/default/aimath/aimath_f32_default.o] Error 1	AIfES_TestBug		 	C/C++ Problem
make: *** [AIfES_for_Arduino/src/basic/default/aimath/subdir.mk:25: AIfES_for_Arduino/src/basic/default/aimath/aimath_q31_default.o] Error 1	AIfES_TestBug		 	C/C++ Problem
make: *** [AIfES_for_Arduino/src/basic/default/aimath/subdir.mk:25: AIfES_for_Arduino/src/basic/default/aimath/aimath_q7_default.o] Error 1	AIfES_TestBug		 	C/C++ Problem
make: *** Waiting for unfinished jobs....	AIfES_TestBug		 	C/C++ Problem

Going back to the sources, I found that this is caused by an incomplete macro inside:

AIfES_for_Arduino/src/aifes_config.h
line 96:  #define AISTRING_STORAGE_WRAPPER(S) 

Replacing line 96 with:

#define AISTRING_STORAGE_WRAPPER(S)     const char S[]

seems to fix the problem (which occurs at line 29, 30 of "src/basic/default/aimath/aimath_f32_default.c").

Help with inference on Arduino (outputs are 0)

Hi! I have a sequential model from Keras and I'm trying to deploy it to Arduino Uno using AIfES.

This is my model in Keras:

Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input (Dense)               (None, 10)                510       
                                                                 
 dense_2 (Dense)             (None, 10)                110       
                                                                 
 output (Dense)              (None, 5)                 55        
                                                                 
=================================================================
Total params: 675
Trainable params: 675
Non-trainable params: 0
________________________________________________________________

However, my outputs are 0. Can you check if my configuration in Arduino is correct?

// Tensor for the input data
  uint16_t input_shape[] = {DATA_COUNT, INPUTS};                      // Definition of the input shape
  aitensor_t input_tensor = AITENSOR_2D_F32(input_shape, input_data); // Creation of the input AIfES tensor with two dimensions and data type F32 (float32)

  // Tensor for the output data
  float output_data[DATA_COUNT * OUTPUTS];          // Output data
  uint16_t output_shape[2] = {DATA_COUNT, OUTPUTS}; // Definition of the output shape
  aitensor_t output_tensor = AITENSOR_2D_F32(output_shape, output_data);

  // ---------------------------------- Layer definition ---------------------------------------

  uint16_t input_layer_shape[] = {1, INPUTS};                                  // Definition of the input layer shape (The 1 must remain here regardless of the number of data sets)
  ailayer_input_f32_t input_layer = AILAYER_INPUT_F32_M(2, input_layer_shape); // Creation of the AIfES input layer
  ailayer_dense_f32_t dense_layer_1 = AILAYER_DENSE_F32_M(NEURONS_1, W0_data, b0_data);
  ailayer_relu_f32_t relu_layer_1 = AILAYER_RELU_F32_M(); // Hidden activation function
  ailayer_dense_f32_t dense_layer_2 = AILAYER_DENSE_F32_M(NEURONS_2, W1_data, b1_data);
  ailayer_relu_f32_t relu_layer_2 = AILAYER_RELU_F32_M(); // Output activation function
  ailayer_dense_f32_t output_layer = AILAYER_DENSE_F32_M(OUTPUTS, W2_data, b2_data);
  ailayer_relu_f32_t relu_output_layer = AILAYER_RELU_F32_M();

  ailoss_mse_t mse_loss; // Loss: mean squared error

  // --------------------------- Define the structure of the model ----------------------------

  aimodel_t model; // AIfES model
  ailayer_t *x;    // Layer object from AIfES, contains the layers

  // Passing the layers to the AIfES model
  model.input_layer = ailayer_input_f32_default(&input_layer);
  x = ailayer_dense_f32_default(&dense_layer_1, model.input_layer);
  x = ailayer_relu_f32_default(&relu_layer_1, x);
  x = ailayer_dense_f32_default(&dense_layer_2, x);
  x = ailayer_relu_f32_default(&relu_layer_2, x);
  x = ailayer_dense_f32_default(&output_layer, x);
  model.output_layer = ailayer_relu_f32_default(&relu_output_layer, x);

  // Add the loss to the AIfES model
  model.loss = ailoss_mse_f32_default(&mse_loss, model.output_layer);

  aialgo_compile_model(&model); // Compile the AIfES model

Thank you!!!

BiDirectional networks

we would like to try out the AIfES framework to evaluate the execution of a model built with Keras.
Can BiDirectional network models from Keras be translated into an implementation that runs, for example, on a Portenta H7?

Support for Sequential Models

Hi! Are sequential models supported in this library?

support for hardim function

Is it planned to add support for hardim activation layer?

Impressed with the weights loading

It has been a while since I was loading weights using TensorflowJS https://www.rocksetta.com/tensorflowjs/beginner-examples/tfjs11-load-weights.html

I will have to try loading a saved TensorflowJS set of weights into an AIfES program.

Presently I think the most important example would be showing how to Serial print a trained model set of weights (or save to an SD card) and then to have a different sketch that is pre-programmed to load the generated set of weights. (Or load them from the SD card). Kind of like how Tensforflow-Micro allows the user to save a c header file (a converted TFlite file).

I actually prefer just saving the weights than a TFLITE file (which contains the model structure) as it makes more sense to me to just load the weights since I already know the structure of my model.

Any opinions about generating weights and re-loading those weights?

Wrong macro usage in CNN example

After merging #18 the wrong macro is used when initializing the sigmoid layer in the example "0_Universal/6_CNN/1_CNN_Training_F32/". However, this does not affect the result of the training!

In line 141, instead of

ailayer_sigmoid_f32_t sigmoid_layer_3 = AILAYER_RELU_F32_A();

should be

ailayer_sigmoid_f32_t sigmoid_layer_3 = AILAYER_SIGMOID_F32_A();

should be used.

https://github.com/Fraunhofer-IMS/AIfES_for_Arduino/blob/07a438213f67e8f9d0271417a9f16be97f1597d2/examples/0_Universal/6_CNN/1_CNN_Training_F32/1_CNN_Training_F32.ino#L141C2-L141C65

Which #include <arm_math.h> for nano33BleSense CMSIS example

On both the Portenta or the Nano33BleSense my Arduino IDE 1.18.15 compiler dies on the line

#include <arm_math.h>

When I try to install it I only get the option to install the

"Arduino_CMSIS-DSP" which does not work

if I google arm_math I get several libraries such as:

https://github.com/ARM-software/CMSIS/blob/master/CMSIS/Include/arm_math.h

https://github.com/PaulStoffregen/arm_math

etc, etc. Any suggestions?

Multiclasstraining on PC

Hi I want to train a NN for the MNIST dataset on the PC.
How can I implement a training of a NN with multiple output neurons.
When I am editing the example (link) and change the output layer to 10 neurons the inference of the model stops with an error (Process returned -1073741819 (0xC0000005) execution time : 0.485 s)
Are there any changes I need to be especially aware of?